We are looking for a Senior Engineer who will join our team to be responsible for building out the data pipelines (data ingestion framework and ETL processing) that are scalable and reusable, also be responsible for transitioning data workloads to the Cloud. You will take responsibilities for all things related to system maintenance, application support, deployment and pipeline engineering and support. This includes learning and understanding upstream processes, pipelines and source systems. This role will work with other cross functional teams to help build and support the next generation of data platform.
Responsibilities:
We’re seeking the candidates who are comfortable and have some experience with the AWS eco-system (or even certified for foundation and associate level, stretching this to professional).
This includes but is not limited to:
- Distributed Data Stores (specifically Aurora, ElasticSearch, ElastiCache/Redis, Neptune)
- Distributed streaming data flow / integration (specifically Flink, Glue, Kinesis family ..Streams/Firehose/Analytics)
- Distributed/parallel data processing workflows (specifically EKS, ECS, Fargate, Lambda)
- Query engines (specifically Athena, Redshift/Spectrum)
- Machine Learning (specifically EMR, SageMaker)
- Monitoring/Alerting/Telemetry/Triage (specifically CloudWatch, X-Ray, Kibana, Grafana)
Besides this we have a few technologies/systems to be deployed on-premise, which serve as edge- computing for our cloud env:
- Distributed data processing workflows (experience with Apache Spark and/or Apache Flink with Apache Beam is great)
- Resource Manager (experience with Apache Hadoop Yarn and/or Kubernetes would be fantastic)
- Distributed file-storage/object-storage (experience with HDFS or Ceph or GlusterFS or MinIO)
- Database (PostgreSQL)
- Monitoring, Alerting (Prometheus, Grafana)
WHY BECOME IT/DATA EXPERTS AT TECHCOMBANK?