Data Engineer (Python)

The Role:

• Integrate with internal/external LLM APIs (e.g., OpenAI, Azure OpenAI), including prompt engineering and pre/post-processing as required.
• Build and maintain data analysis workflows using Pandas for data transformation and insight delivery.
• Develop RESTful APIs using FastAPI or Flask for data and document management.
• Design and implement clean, efficient, and modular Python codebases for backend services, data pipelines, and document processing workflows.
• Support the team in onboarding new data sources, integrating with Azure services, and ensuring smooth cloud deployments.
• Collaborate with product, data science, and engineering teams to translate business requirements into technical solutions.
• Write unit tests and contribute to CI/CD pipelines for robust, production-ready code.
• Stay up to date with advances in Python, LLM, and cloud technologies.

Qualifications:

Bachelor’s or master’s in computer science, Engineering, or related quantitative discipline.

Experience:

6 to 9 years of hands-on experience in data engineering or backend development with Python.

Technical Competencies:

• Exposure to LLM integration (prompt design, API integration, handling text data).
• Strong experience in Python with focus on data analysis (Pandas) and scripting.
• Hands-on experience in building REST APIs (FastAPI or Flask).
• Experience in developing data pipelines, data cleaning, and transformation.
• Working knowledge of Azure cloud services (Azure Functions, Blob Storage, App Service, etc.)

(Nice to have) Experience integrating MongoDB with Python for data storage, modelling, or reporting.