girl looking into her desktop
Back to search results

Data Engineer

Charlotte, North Carolina;

Job Description:

Responsible for performing complex analysis and is engaged in the development of modeling that maximizes profits and asset growth and minimizes credit and operating losses and other risk exposures. Provides analytical support on various product strategies to ensure company goals are met. Coordinates the production of performance reports for Senior Management. Reviews and analyzes trends in current population distributions and recommends strategies. May participate in or develop complex program models to extract data and use databases to provide statistical and financial modelling.

Responsible for performing more complex analysis and modeling for multiple products with the goal of maximizing profits and asset growth and minimizing risk and operating losses and/or other financial and marketing exposures. Develops complex program models to extract data and uses multiple databases to acquire statistical and financial data. Utilizes portfolio trends to propose policy/procedural changes within segmentation structure to produce optimal results. Excels at risk/reward trade off. Build relationships with external agencies. Participates in the rollout of company-wide pilot programs developed as a result of programmed models. Duties primarily include the regular use of discretion, independent judgment, the ability to communicate with multiple levels of management and the utilization of core leadership behaviors. SAS and/or MicroStrategy experience preferred; 2-5 years experience preferred.

Required and desired skills/qualifications:

• Have strong technical foundation with in-depth knowledge in Big Data Hadoop, Data Reporting, Data Design, Data Analysis, Data governance, Data integration and Data quality.
• Experience in setting, configuring and monitoring of Hadoop cluster of Cloudera, Hortonworks distribution.
• Deep and extensive knowledge with HDFS, Spark, MapReduce, Hive, HBase, Sqoop, Yarn, Oozie.
• Thorough knowledge on Hadoop architecture and various components such as HDFS, Name Node, Data Node, Application Master, Resource Manager, Node Manager, Job Tracker, Task Tracker and MapReduce programming paradigm.
• Good understanding on Hadoop MR1 and MR2 (YARN) Architecture.
• Experience in developing scalable solutions using NoSQL databases including HBASE and COSMOS DB.
• Efficient in working with Hive data warehouse tool creating tables, data distributing by implementing Partitioning and Bucketing strategy, writing and optimizing the HiveQL queries.
• Experienced in performing analytics on structured data using Hive queries, operations, Joins, tuning queries, SerDe's and UDF.
• Good experience working with different Hadoop file formats like Sequence File, RCFile, ORC, AVRO and Parquet.
• Experience in using modern Big-Data tools like SparkSQL to convert schema-less data into more structured files for further analysis. Experience in Spark Streaming to receive real time data and store the stream data into HDFS.
• Good Knowledge on Hadoop Cluster architecture and monitoring the cluster.
• In-depth understanding of Data Structure and Algorithms.
• Experience in managing and reviewing Hadoop log files.
• Excellent understanding and knowledge of NOSQL databases like HBase.
• Implemented in setting up standards and processes for Hadoop based application design and implementation.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
• Experience in working with Apache Sqoop to import and export data to and from HDFS and Hive.
• Good working experience in designing Oozie workflows for cleaning data and storing into Hive tables for quick analysis.
• Primarily responsible for designing, implementing, Testing, and maintaining database solution for Azure.
• Primarily involved in Data Migration process using Azure by integrating with Github repository and Jenkins.
• Hands on experience with Real time streaming using Kafka, Spark streaming into HDFS.
• Developed analytical components using SparkSql and Spark Stream.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Spark SQL using Scala.
• Good knowledge streaming data using Flume and Kafka from multiple sources into HDFS.
• Knowledge of processing and analyzing real-time data streams/flows using Kafka and HBase.
• Experience with Informatica Power Center Big Data Edition (BDE) for high-speed Data Ingestion and Extraction.
• Hands on experience with Amazon EMR, Cloudera (CDH4 & CDH5), and Horton Works Hadoop Distributions
• Proficient in all phases of software development including design, configuration, testing, debugging, implementation, release, and support of large-scale, Pega platform applications. Candidate must have 2-5 years of experience building and implementing model-driven, enterprise-level business solutions using Pega PRPC.

Qualified candidate will also need to be able to interface closely with our data science team on deeper analytical needs. The individual is also expected to be able to communicate with multiple levels of management and able to succinctly summarize complex and highly technical requests.

Job Band:

H5

Shift: 

1st shift (United States of America)

Hours Per Week:

40

Weekly Schedule:

Referral Bonus Amount:

0

Job Description:

Responsible for performing complex analysis and is engaged in the development of modeling that maximizes profits and asset growth and minimizes credit and operating losses and other risk exposures. Provides analytical support on various product strategies to ensure company goals are met. Coordinates the production of performance reports for Senior Management. Reviews and analyzes trends in current population distributions and recommends strategies. May participate in or develop complex program models to extract data and use databases to provide statistical and financial modelling.

Responsible for performing more complex analysis and modeling for multiple products with the goal of maximizing profits and asset growth and minimizing risk and operating losses and/or other financial and marketing exposures. Develops complex program models to extract data and uses multiple databases to acquire statistical and financial data. Utilizes portfolio trends to propose policy/procedural changes within segmentation structure to produce optimal results. Excels at risk/reward trade off. Build relationships with external agencies. Participates in the rollout of company-wide pilot programs developed as a result of programmed models. Duties primarily include the regular use of discretion, independent judgment, the ability to communicate with multiple levels of management and the utilization of core leadership behaviors. SAS and/or MicroStrategy experience preferred; 2-5 years experience preferred.

Required and desired skills/qualifications:

• Have strong technical foundation with in-depth knowledge in Big Data Hadoop, Data Reporting, Data Design, Data Analysis, Data governance, Data integration and Data quality.
• Experience in setting, configuring and monitoring of Hadoop cluster of Cloudera, Hortonworks distribution.
• Deep and extensive knowledge with HDFS, Spark, MapReduce, Hive, HBase, Sqoop, Yarn, Oozie.
• Thorough knowledge on Hadoop architecture and various components such as HDFS, Name Node, Data Node, Application Master, Resource Manager, Node Manager, Job Tracker, Task Tracker and MapReduce programming paradigm.
• Good understanding on Hadoop MR1 and MR2 (YARN) Architecture.
• Experience in developing scalable solutions using NoSQL databases including HBASE and COSMOS DB.
• Efficient in working with Hive data warehouse tool creating tables, data distributing by implementing Partitioning and Bucketing strategy, writing and optimizing the HiveQL queries.
• Experienced in performing analytics on structured data using Hive queries, operations, Joins, tuning queries, SerDe's and UDF.
• Good experience working with different Hadoop file formats like Sequence File, RCFile, ORC, AVRO and Parquet.
• Experience in using modern Big-Data tools like SparkSQL to convert schema-less data into more structured files for further analysis. Experience in Spark Streaming to receive real time data and store the stream data into HDFS.
• Good Knowledge on Hadoop Cluster architecture and monitoring the cluster.
• In-depth understanding of Data Structure and Algorithms.
• Experience in managing and reviewing Hadoop log files.
• Excellent understanding and knowledge of NOSQL databases like HBase.
• Implemented in setting up standards and processes for Hadoop based application design and implementation.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
• Experience in working with Apache Sqoop to import and export data to and from HDFS and Hive.
• Good working experience in designing Oozie workflows for cleaning data and storing into Hive tables for quick analysis.
• Primarily responsible for designing, implementing, Testing, and maintaining database solution for Azure.
• Primarily involved in Data Migration process using Azure by integrating with Github repository and Jenkins.
• Hands on experience with Real time streaming using Kafka, Spark streaming into HDFS.
• Developed analytical components using SparkSql and Spark Stream.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Spark SQL using Scala.
• Good knowledge streaming data using Flume and Kafka from multiple sources into HDFS.
• Knowledge of processing and analyzing real-time data streams/flows using Kafka and HBase.
• Experience with Informatica Power Center Big Data Edition (BDE) for high-speed Data Ingestion and Extraction.
• Hands on experience with Amazon EMR, Cloudera (CDH4 & CDH5), and Horton Works Hadoop Distributions
• Proficient in all phases of software development including design, configuration, testing, debugging, implementation, release, and support of large-scale, Pega platform applications. Candidate must have 2-5 years of experience building and implementing model-driven, enterprise-level business solutions using Pega PRPC.

Qualified candidate will also need to be able to interface closely with our data science team on deeper analytical needs. The individual is also expected to be able to communicate with multiple levels of management and able to succinctly summarize complex and highly technical requests.

Shift:

1st shift (United States of America)

Hours Per Week: 

40

Learn more about this role

Full time

JR-21065291

Band: H5

Manages People: No

Travel: No

Manager:

Talent Acquisition Contact:

Taylor Pitre

Referral Bonus:

0