Data Engineer - Python/AWS

We are seeking a skilled Data Engineer to join our client's team in Charlotte, NC. This person will be responsible for the design and development of data/batch processing, data manipulation, data mining, and data extraction/ transformation/ loading into large data domains using Python/Pyspark and AWS tools.

Client Details

The employer is a well-established organization in the Financial Services industry, known for its strong market presence and innovative approach. As part of a large organization, they offer a professional environment that emphasizes excellence and technical advancement.

Description

• Lead end-to-end delivery of data and application solutions, including scoping, estimating, planning, design, development, testing, deployment, and production support while adhering to SDLC standards.

• Design, develop, and maintain scalable ETL pipelines and batch processing solutions, collaborating with developers, business analysts, and stakeholders to translate requirements into technical designs.

• Produce and present technical design documentation and solution alternatives that align with business objectives, ensuring data quality, defect tracking, and risk management.

• Monitor project progress, coordinate vendors and contractors, resolve or escalate issues, and apply industry best practices and emerging technologies to ensure successful delivery.

MPI does not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, disability, veteran status, marital status, or based on an individual's status in any group or class protected by applicable federal, state or local law. MPI encourages applications from minorities, women, the disabled, protected veterans and all other qualified applicants.

Profile

• Data engineering background with hands-on development experience using Python, PySpark, and ETL tools, including designing mappings, HLD/LLD, coding, and unit testing

• Extensive experience with AWS services (Glue, Lambda, MSK/Kafka, S3, Step Functions, RDS, EKS), relational databases, and advanced SQL development and performance tuning.

• Proven ability in data modeling, data analysis, data profiling, REST API integration, UNIX scripting, and workload automation tools such as Control-M and Autosys.

• Experience working in Agile/Scrum environments with CI/CD DevOps tools (GitHub, Jenkins), familiarity with multiple ETL platforms, and knowledge of MDM, data warehousing, and analytics.

Job Offer

• Opportunity to work in a large organization within the Financial Services industry.
• Access to comprehensive benefits through Michael Page.
• Exposure to innovative technology projects in a professional environment.

MPI does not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, disability, veteran status, marital status, or based on an individual's status in any group or class protected by applicable federal, state or local law. MPI encourages applications from minorities, women, the disabled, protected veterans and all other qualified applicants.