Big Data Infrastructure Engineer

RESPONSIBILITIES
 ? Responsible for design and development core modules in our big data platform infrastructures (hosting in Google Cloud, based on Hadoop ecosystem, Spark Core/Streaming/SQL, Scala, Python, AngularJS, Node.js, Kafka, Impala, Elasticsearch, Google Cloud Machine Learning Engine and TensorFlow) – our big data platform handles huge amount of information, through complex processing in batch and real time modes, complex data searches, using pluggable services and UI frameworks.  
? Responsible for our production environments and for new developments.
? Responsible for research, analysis and performing proof of concepts for new technologies, tools and design concepts.  
 
REQUIREMENTS
 5+ years of experience working in Java/Scala/Python or other high level programming language.  
? Strong understanding of software architecture paradigms (OOP, FP) and data structures.  
? Strong understanding of design patterns and clean architecture, infrastructure development skills.  
? Experience working on large scale distributed systems and distributed programing.  
? Experience working with Linux OS systems and Bash scripting languages.  
? Experienced with building scalable stream-processing and/or ETL batch processing – using solutions such as Spark/Spark Streaming.  
? Experience working with development testing methodologies.  
? Strong SQL skills and experienced with NoSQL databases (such as HBase, Cassandra, MongoDB and Impala) and relational databases (such as MySQL and SQL Server).  
? Experience working with CI/CD tools (Jenkins, Chef etc). Bachelor’s degree in Computer Science or related.  
 
DESIRED
1. ? Experience working with cloud service (AWS, GCP, Azure) – an advantage.  
2. ? Proven experience working with GitHub – an advantage.  
3. ? Experience working with Node.js – an advantage.  
4. ? Experience working with Docker/Kubernetes Engine – an advantage.  
5. ? Knowledge in hadoop ecosystem- an advantage.