Role: Data Engineer (AWS and Appflow)
Job Description:
We are seeking a skilled and motivated Data Engineer with expertise in AWS (Amazon Web Services) and Appflow to join our dynamic and innovative team. As a Data Engineer, you will play a crucial role in designing, building, and maintaining our data pipelines and infrastructure to enable efficient data integration, transformation, and analysis. You will collaborate with various teams, including data scientists, software engineers, and business analysts, to deliver robust and scalable data solutions that power data-driven decision-making across the organization.
Data Pipeline Design and Development: Design, build, and maintain data pipelines and workflows using AWS services and Appflow to extract, load, and transform data from various sources, ensuring data quality, reliability, and performance.
Data Integration: Implement seamless integration of data from different systems, databases, APIs, and third-party platforms into our data ecosystem, ensuring compatibility and consistency of data.
Data Transformation and ETL: Develop data transformation processes and ETL (Extract, Transform, Load) workflows to cleanse, enrich, and prepare data for analytics and reporting purposes.
Data Modeling: Design and maintain data models that support analytical needs and provide a solid foundation for business insights.
Performance Optimization: Monitor and optimize data pipelines for performance, efficiency, and scalability, taking into account data volume and processing requirements.
Data Governance and Security: Ensure compliance with data governance policies and best practices for data security, privacy, and data retention.
Data Quality Management: Implement and maintain data quality checks and validations to ensure the accuracy and reliability of data.
Cloud Infrastructure Management: Work closely with DevOps teams to manage and optimize the AWS infrastructure supporting the data ecosystem.
Collaboration and Documentation: Collaborate effectively with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand requirements and deliver on data engineering initiatives. Document data processes, data lineage, and architecture for reference and future enhancements.
Stay Updated with Emerging Technologies: Stay abreast of industry trends, best practices, and new AWS and Appflow features to continually improve data engineering capabilities and suggest innovative solutions.
Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field.
•Proven experience as a Data Engineer with hands-on expertise in AWS and Appflow services
(Development using PySpark and big data processing on AWS)
- Knowledge in writing stored procedures in Redshift and optimize SQL queries.
- Create data flows using AWS Appflow
- Setup control-M jobs for orchestration
- SSIS, SQL and C# (ETL functions)
- Redshift, Athena, Python (AWS)
- Control-M (event monitoring)
- Strong analytical and problem-solving skills, with the ability to troubleshoot and resolve data-related issues.
- Excellent communication skills and the ability to work collaboratively in a team-oriented environment.
Join us in this exciting opportunity to shape the future of our data infrastructure and make a significant impact on our organization's data-driven decision-making process. We offer a competitive salary, a stimulating work environment, and opportunities for personal and professional growth. If you are passionate about data engineering and have a strong background in AWS and Appflow, we encourage you to apply and become a valued member of our data engineering team.
Job Type: Full-time
Salary: $90,000.00-$125,000.00 per year
Benefits:
Flexible Language Requirement:
Schedule:
Supplemental pay types:
Ability to commute/relocate:
- Montréal, QC H3B 4W8: reliably commute or plan to relocate before starting work (required)
Experience:
- AWS: 7 years (preferred)
- Redshift: 7 years (preferred)
- Python: 7 years (preferred)
Work Location: In person
