Data Engineer, Big Data, New York New York, NY, United States, Full-Time
Dailymotion is one of the world’s leading video-sharing platforms. It attracts over 300 million unique visitors and 3 billion videos views worldwide per month by offering the best content from users, independent content creators and premium partners. As part of its growing activities, Dailymotion is looking to enforce its programmatic and monetization product by building its own video ad stack to deliver new monetization solution for its own ecosystem around online, mobile and TV, and provide innovative marketing solutions for advertisers.
As a Data Engineer for Big Data you will design and build highly-scalable data pipelines and data stores using cutting edge big data technologies, to help power Dailymotion's fast-growing digital advertising business.
- Design, build, and maintain petabyte-scale datastores, and high-performance, horizontally-scalable processing pipelines to support them. - Create processes for large-scale ingest and export of data, as well as enabling fast data query capabilities. - Build and maintain a centralized user profile datastore to enable precision audience targeting. - Monitor, analyze, and improve the efficiency, scalability, and stability of data collection, storage, retrieval processes, and KPI’s. - Work closely with other teams in the group to integrate systems – including Data Science team for building, testing, and productionalizing data analysis algorithms, Ops team for deployment and production support, Data Architecture team for ingestion of data, etc.
- Minimum of 5 years software development experience. - B.S. degree in Computer Science or related discipline (M.S. is a plus). - A solid understanding of core computer science concepts, and fundamentals of distributed computing. - Hands-on experience using the Hadoop MapReduce and/or Spark distributed computing frameworks. - Experience working with relational databases (e.g., PostgreSQL, MySQL) as well as with one or more NoSQL, distributed, or columnar databases (e.g., Cassandra, Redis, Aerospike, RedShift, Vertica). - Strong skills with the SQL and HQL (Hive) data query languages.Strong coding skills in Java. (Python or Scala a plus.) - Proficiency working in a Unix/Linux environment, with strong experience in shell script programming. - Experience working in a cloud computing environment (e.g., AWS, Google Cloud, Azure) - Exposure to streaming technologies (e.g., Spark Streaming, Apache Storm, Flink) and/or containerization technologies (e.g., Mesos, Docker, Kubernetes) is a plus. - Previous Ad Tech experience is a big plus.
Salary commensurate with experience. We offer great health / dental / vision insurance, 401k (company Safe Harbor contribution) paid life / AD&D with additional life insurance / AD&D insurance (optional), summer hours, paid vacation, sick time, paid holidays, company sponsored Training Opportunities (upon eligibility) and much more