Senior Data Engineer / Architect

Senior Data Engineer / Architect

A giant in the gaming industry is looking for an expert who can design the architecture as well as build the infrastructure.

Responsibilities

• Develop, construct, test, and maintain the data architecture
• architecting and building the big data stack from scratch
• Ensure that the architecture will support the requirements of the business
• Discover opportunities for data acquisition
• Develop data set processes for data modelling, mining and production
• Implement secure and scalable API endpoints to make available the computed data
• Recommend ways to improve data reliability, efficiency and quality
• Collaborate with data analysts/-scientists and the Business Intelligence Team to implement new ELT pipelines

Qualifications

• Experience in architecting and building the big data stack from scratch with minimal support highly desired
• Understanding of Cloud architecture in the big data and data warehousing space
• Preferred proficiency with Python/Java, shell scripting, system diagnostic and automation tooling
• Expertise in building and maintaining Hadoop clusters in a Hybrid environment
• Extensive experience in Big Data technologies and solutions (Spark, Hadoop, Hive, HDFS)
• Experience in building Data Streaming pipelines (Spark, Flume, Kafka, Kinesis, Flink, etc.)
• Strong experience in designing and implementing highly available high-volume messaging infrastructure with Apache Kafka
• At least 5 years of professional experience in data engineering , data science or another related field with experience in manipulating and processing huge data sets