|
Current Statistics
1,581,556 Total Jobs
240,909 Jobs Today
17,821 Cities
222,734 Job Seekers
146,855 Resumes |
|
|
 |
|
 |
 |
Platform Data Engineer - San Jose California
Company: Iconma LLC
Location: San Jose, California
Posted On: 01/30/2025
Location: San Francisco, Seattle, LA or PSTEnterprise Data Platform to enable timely, effective and safe sharing of data to multiple engineering, operations and business teams for building world class data products.Responsibilities Build data ingestion and processing pipelines to enable data analytics and data science use-cases in areas of digital commerce, service operations, charging, reliability, finance, capex, warranty, customer service and others. Build modular set of data services using Python, SQL, AWS Glue, lambdas, API Gateway, Kafka, data build tool (dbt), Apache Spark on EMR among others. Build automated unit and integration testing pipelines using frameworks like PySpark. Create and manage CICD pipelines with Gitlab CI and AWS Code Pipeline/CodeDeploy. Automate and schedule jobs using Managed Airflow. Build the ODS and reporting schemas and load the data into AWS Redshift or Snowflake. Design and build data quality management services with Apache Deequ and data observability tools like Splunk, DataDog, CloudWatch. Provide a variety of query services with REST, Athena/Presto, server sent events. Configure and setup the enterprise data lineage and meta data management and data catalog support using tools like Collibra/Alation. Assist the data scientist within the data engineering team as well as other software engineering teams with data cleansing, wrangling and feature engineering. Ensure green builds for deployment and work with program management and senior leads to burn down planned deliverables in a sprint cycle.Qualifications At least 5+ years building data and analytics platforms using AWS Cloud, Python and SQL. Knowledge of AWS technologies specifically MSK, EMR, Athena, Glue, lambdas, API Gateway as well as Python, SQL is a must. Knowledge of modern data tools like dbt (data build tool) and Airflow orchestration is highly desired. Ability to assist SQL analysts and Tableau developers in business teams in creating the right set of materialized views in a SQL data warehouse like Redshift/Snowflake. Knowledge of automation and CICD best practices. Familiarity with machine learning and data science ecosystems especially AWS Sagemaker and Databricks is highly preferred. Hands-on experience in building and maintaining production data applications, current experience in both relational and distributed columnar data stores. Deep experience using SQL, Python, and Spark. Hands-on experience with Big-data technologies (e.g. Redshift, Athena, Glue, EMR, Kinesis, Step Function, or equivalent in other web services). Familiarity with timeseries database, data streaming applications, Kafka, Flink, and more is a plus. Familiarity with modern data science and product analytics tools and techniques such as R, Machine Learning, and advanced statistics is a plus.
#J-18808-Ljbffr More...
Register an account with us and set up job agents! We'll email you immediately when jobs like this are posted on our site.
|
 |
 |
 |
 |
|
|
RSS Job Feeds