Job Description:
Design & implement the product’s core elements & data pipeline(s)
Help design & implement a cutting edge, scalable, data-based rules engine!
Responsible for the product’s core – its underlying data lake & storage
Work in collaboration with the rest of the team to integrate with the product backend
Qualifications:
5+ Years experience as a data engineer in a SaaS environment
Experience in architecting pipelines for processing streaming data (Spark/Storm/Flink/Beam, etc) – must
Experience in coding , Python – Must
Experience in orchestrating batch processing jobs over large amounts of data (e.g. with AirFlow) – must
Experience with major cloud providers – must
Experience & familiarity with big data stores such as BigQuery, Athena, Snowflake – advantage
Experience with Graph databases such as Neo4j – strong advantage
Experience in Cyber Security companies – advantage