Data Engineering Lead

Job Summary

We are seeking a Data Engineering Lead to spearhead the agile development of our data platform, which is crucial for our company's digital transformation. This role involves leading the data engineering team, administering Databricks environments, crafting and executing ETL processes through a Medallion architecture, and ensuring the seamless integration of data into our Neo4j knowledge graph. The Data Engineering Lead will play a pivotal role in our data-driven initiatives reporting to the Senior Data Manager and working closely with the Data Analytics Lead.

Essential Functions

Data Platform Development: Build and maintain a robust data platform, ensuring timely access to high-quality data. Pipeline Construction: Architect and uphold data pipelines from various sources, including internal databases and SaaS applications. Code Excellence: Write clean, efficient, and maintainable code. Embrace and implement the DataOps philosophy to promote operational excellence in data management. System Expansion: Strategically plan and implement system enhancements to support business growth and analytical demands. Best Practices: Champion data engineering best practices, focusing on improving data quality, efficiency, and value. Code Standards: Elevate and enforce coding standards, focusing on maintainability and performance. Guide the team in creating merge requests and managing issues effectively. Independent Delivery: Lead by example, delivering medium to large features independently and fostering a culture of accountability and innovation within the team.

Minimum Requirements

Experience: At least 6 years of hands-on experience in deploying production-quality code, with a strong preference for experience in Python, Java, or Scala for data processing (Python preferred). Technical Proficiency: Advanced knowledge of data-related Python packages and a profound understanding of SQL and Databricks. Graph Database Expertise: Solid grasp of Cypher and experience with graph databases like Neo4j. ETL/ELT Knowledge: Proven track record in implementing ETL (or ELT) best practices at scale and familiarity with data pipeline tools.

Preferred Qualifications

Professional experience using Python, Java, or Scala for data processing (Python preferred)

Working Conditions And Physical Requirements

Ability to work for long periods at a computer/deskStandard office environment

About The Organization

Fullsight is an integrated brand of our three primary affiliate companies – SAE Industry Technologies Consortia, SAE International and Performance Review Institute – and their subsidiaries. As a collective, Fullsight enables a robust resource of innovative programs, products and services for industries, their engineers and technical experts to work together on traditional and emergent complex issues that drive their future progress.

SAE Industry Technologies Consortia® (SAE ITC) enables organizations to define and pilot best practices. SAE ITC industry stakeholders are able to work together to effectively solve common problems, achieve mutual benefit for industry, and create business value.

The Performance Review Institute® (PRI) is the world leader in facilitating collaborative supply chain oversight programs, quality management systems approvals, and professional development in industries where safety and quality are shared values.

SAE International® (SAEI) is a global organization serving the mobility sector, predominantly in the aerospace, automotive and commercial-vehicle industries, fostering innovation, and enabling engineering professionals. Since 1905, SAE has harnessed the collective wisdom of engineers around the world to create industry-enabling standards. Likewise, SAE members have advanced their knowledge and understanding of mobility engineering through our information resources, professional development, and networking.