Senior Data Engineer

Job Responsibilities

• Microsoft Fabric Expertise: Serve as the primary subject matter expert for Microsoft Fabric, including but not limited to Lakehouse, Data Factory, Synapse Data Engineering (Spark), Data Warehousing, Real-Time Analytics, and Power BI integration.

• Data Solution Design & Development: Lead the end-to-end design, development, and implementation of robust, scalable, and efficient data pipelines and solutions within the Microsoft Fabric ecosystem. This includes data ingestion, transformation, orchestration, and consumption layers.

• Architectural Leadership: Contribute to and drive the architectural vision for our data platform on Microsoft Fabric, ensuring best practices for performance, security, reliability, and cost-effectiveness.

• Independent Project Management: Take full ownership of data engineering projects, including planning, resource allocation (where applicable), timeline management, risk mitigation, and successful delivery.

• Stakeholder Collaboration: Work closely with data analysts, business stakeholders, and other engineering teams to understand data requirements, translate them into technical specifications, and deliver impactful data solutions.

• Data Governance & Quality: Implement and enforce data governance, quality, and security best practices within the Microsoft Fabric environment.

• Performance Optimization & Troubleshooting: Proactively monitor, troubleshoot, and optimize data pipelines and data models for performance and efficiency.

• Mentorship & Best Practices: Champion best practices in data engineering, data modeling, and Microsoft Fabric utilization. Potentially mentor junior team members and foster a culture of continuous learning.

• Documentation: Create and maintain comprehensive technical documentation for data pipelines, data models, and architectural designs.

Job Requirements

• Minimum 5+ years of experience in data engineering, with a strong focus on building and maintaining enterprise-grade data platforms.

• Demonstrable expert-level proficiency in Microsoft Fabric, with hands-on experience across multiple components (Lakehouse, Data Factory, Synapse Data Engineering/Spark, Data Warehousing, Real-Time Analytics).

• Proven track record of successful project management in data-related initiatives, including planning, execution, and delivery.

• Extensive experience with Spark (PySpark/Scala) for data transformation and processing within a distributed environment.

• Strong understanding of data warehousing concepts, dimensional modeling, and data lake architectures.

• Proficiency in SQL for data manipulation and analysis.

• Experience with Azure ecosystem components (e.g., Azure Data Lake Storage, Azure DevOps, Azure Functions) is a strong plus.

• Experience with version control systems (e.g., Git).

• Excellent problem-solving, analytical, and critical thinking skills.

• Exceptional communication and interpersonal skills, with the ability to articulate complex technical concepts to non-technical stakeholders and work effectively in a team-oriented environment.

• Highly self-motivated and able to work independently with minimal supervision, taking initiative and ownership of tasks.

• Microsoft Certified: Azure Data Engineer Associate, or other relevant Microsoft Azure/Fabric certifications.

• Experience with real-time data streaming technologies.

• Familiarity with CI/CD practices for data pipelines.

• Experience with Power BI for data visualization and reporting.