Data Engineer

Linux Foundation Limited

Design and build data infrastructure: Develop and maintain data pipelines, ETL (Extract, Transform, Load) processes, and data warehouses or lakes to handle large-scale data.
Manage and process data: Clean, combine, and transform raw data from multiple sources into a structured format that can be analyzed.
Ensure data quality and reliability: Create data validation methods and monitor pipeline performance to maintain data integrity and accuracy.
Collaborate with stakeholders: Work with data scientists, analysts, and other teams to understand business needs and provide them with the data they require.
Deploy and optimize systems: Deploy machine learning algorithms and build tools for data analysis, while also optimizing existing systems for better performance.
Ensure data governance and security: Implement and maintain data governance and security policies to ensure compliance.