Data Engineer.

Data Engineer.

Job Id : 6453
Posted : 2022-05-13
Industry : Information Technology and Services
Employment Type : Full Time, Permanent
Required Skills : Japanese3, English3, Data Engineer, HDFS, Hive, HBase, Phoenix, Scala , MS-SQL, MongoDB, Linux, Kerberos, networking
City : Tokyo
State : Tokyo
Country : Japan
Annual Salary : ¥6,000,000 ~ ¥10,500,000

Job Description

Position Name: Data Engineer

Responsibilities:

  • Collaborate with users and project team on requirements and expectations
  • Analyze business requirements, and create proper data solutions that meet functional and non-functional requirements
  • Define the data architecture, structure, storage, and security model of data in HDFS, Hive, HBase, Solr, RestAPI
  • Design and implement highly scalable data pipelines in Scala/Spark/Java for processing of extremely large amounts of data
  • Create and maintain reusable frameworks for data ingestion, validation, normalization, and transformation
  • Automate Unit testing and release with CI/CD pipeline
  • Create scripts for operations, releases, data migrations, data validations
  • Maintain data lineage information
  • Provide technical support for releases, issues, and incidents

Qualification:

  • Minimum 3- 4 years of working experience as a developer or data engineer
  • Solid knowledge and experience of the Hadoop ecosystem: HDFS, Hive, HBase, Phoenix, Solr, Oozie, Impala, Knox, Zookeeper, YARN, Spark, Livy, HUE, Zeppelin
  • Strong programming skills in Scala and Spark
  • Strong capabilities in designing Database schemas on data platform
  • Good knowledge of Oracle database, MS-SQL Server, MongoDB, Kerberos, Linux, networking
  • Proficient in SQL, shell scripting, git, unit testing, CI/CD tools
  • Understanding of design principles, coding standards, and best practices
  • Good problem-solving skills
  • Good data analytics skills
  • Excellent communication, presentation, and interpersonal skills

Preferred Qualification:

  • Understanding of REST API, micro-services, ETL, and Salesforce is preferred
  • Hand-on experience with Cloud services is preferred
  • Understanding of streaming data processing and change data capture is preferred
  • Experience with BI tools is preferred

Language:

  • Japanese: intermediate (N3 and above)
  • English: intermediate (N3 and above)

Company Description 

One of the world's leading professional services companies, transforming clients' business, operating and technology models for the digital era. 
Their unique industry-based, consultative approach helps clients envision, build and run more innovative and efficient businesses. 
Headquartered in the U.S., this company is one of the Fortune 500 companies and is consistently listed among the most admired companies in the world.

 [Passive smoking measures]
Indoor smoking
Designated smoking area

 


AI Job Matching Girl icon

All you have to do is upload your resume!
AI will find the list of jobs matching your career history.

POPULAR JOBS