Company Description
PT. Bumi Amartha Teknologi Mandiri (AMARTEK) is an IT Consulting company based in Jakarta, Indonesia. We are committed to delivering cost-effective and innovative solutions in Data & Analytics, Talent Augmentation, Regulatory Reporting, and Outcome-based services. Our goal is to provide the best value and highest quality of services to our clients by leveraging proven domain knowledge and professional expertise.
Role Description
- Utilize Google Cloud Data Fusion, Composer/Airflow, and DataProc to develop and manage scalable data pipelines that integrate data from various sources.
- Write and optimize Python and PySpark scripts to process, clean, and analyze large datasets.
- Develop and manage BigQuery solutions, including creating and executing queries, writing procedures, and using the BigQuery API to interface with data. Optimize query performance and ensure efficient data storage.
- Leverage Google Cloud Pub/Sub to implement real-time data streaming solutions. Design and maintain pub/sub topics and subscriptions to handle data ingestion and processing.
- Identify and resolve issues related to data processing and pipeline performance. Continuously seek opportunities to optimize and enhance data engineering practices.
Qualifications
- Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent work experience.
- Minimum of 2 years of experience as a Data Engineer or in a similar role, with hands-on experience in Google Cloud Platform and associated tools.
- Proficient in Google Cloud Data Fusion, Composer/Airflow, DataProc, and Pub/Sub.
- Strong experience with Python and PySpark for data processing and analysis.
- Expertise in BigQuery, including API usage, writing procedures, and optimizing queries.
- Experience with Google Kubernetes Engine (GKE) for deploying and managing containerized applications.
- Knowledge of data warehousing and ETL best practices.
