Description
At SmartLens, we start with and end with data. Data is everything. As a data engineer at SmartLens, you will be a part of our engineering team, working with our data scientists, web/UI engineers, products and business. You are in very early stage of the company, and your decision and effort will make significant impact on the future of the company.
In this role, you will have ownership and responsibility of the company's data and its pipelines, making it up-to-date, accurate, accessible and actionable by our data scientists as well as our clients.
Please send your resume (PDF, doc, docx) to job@smartlensanalytics.com
Responsibilities
Own unified data in our data warehouse and ETL pipelines that process various data coming from search engine platforms, 3rd parties, clients, and public data.
Continuously evolve data models and ETL pipelines to meet increasing demand and supply of data.
Own API communications with search engine platforms such as Google AdWords, Bing Ads to automate many of our daily SEM operations.
Design, develop and own SmartLens data APIs that allow our clients to feed their internal data into our system in easy, secure and scalable manner.
Closely work with our data science team to automate some of the proprietary bidding algorithms in scale.
Qualifications
Hadoop (or similar) ecosystem: Spark, MapReduce, HDFS, Presto, Hive, Parquet.
SQL and data models: MySQL, Redshift, Hive, Presto, AWS Athena/Glue Catalog, S3, Elasticsearch.
Workflow (data pipeline) tools: Apache Airflow, AWS Data Pipeline, Luigi...
Programming languages: Python, Java/Scala
Great if you have: AWS experience (e.g. EMR, Redshift, Athena/Glue, Lambda, SQS), machine learning in Python or R