Architect and Design: Develop scalable data architectures using Databricks, Unity Catalog, Azure Data Lakehouse, Medallion Architecture, and Azure Data Factory.
Leadership: Lead and mentor a team of data engineers, full stack developers, and QA teams, fostering a collaborative environment.
Delivery Management: Oversee the entire lifecycle of data engineering projects, ensuring successful data extraction, transformation, and end-user consumption.
Collaboration: Work with cross-functional teams, including data scientists, analysts, and business stakeholders, to meet data requirements.
Problem-Solving: Apply strong analytical and problem-solving skills.
Communication: Exhibit excellent verbal and written communication skills.
Adaptability: Adapt to changing business needs in a fast-paced environment.
Innovation: Stay updated with industry trends and integrate new technologies into data engineering practices.
Technical Expertise:
Extensive experience with Databricks, Unity Catalog, Azure Data Lakehouse, Medallion Architecture, and Azure Data Factory.
Proficiency in Python, PySpark, SQL, and Scala.
Strong understanding of data modeling, ETL/ELT processes, and data warehousing.
Experience with Microsoft Azure services for optimal performance and scalability.
Design and implement use cases for Unity Catalog.
Ensure data quality, integrity, and security.
Familiarity with big data technologies like Hadoop, Spark, and Kafka.
Experience with DevOps practices and CI/CD tools.
Knowledge of full stack web development technologies (React, .Net, etc.).
Qualifications:
Bachelor s or master s degree in computer science, Information Technology, or a related field.
10+ years of experience in data engineering, focusing on the specified technologies and architectures.