Data Engineer (Cloudera) - #1099505

Company Description

NCS is a leading technology services firm that operates across the Asia Pacific region in over 20 cities, providing consulting, digital services, technology solutions, and more. We believe in harnessing the power of technology to achieve extraordinary things, creating lasting value and impact for our communities, partners, and people. Our diverse workforce of 13,000 has delivered large-scale, mission-critical, and multi-platform projects for governments and enterprises in Singapore and the APAC region.

Job Description

Responsibilities:

1. Cloudera Private Cloud Deployment & Management:

Install, configure, and maintain Cloudera Private Cloud (CDP) clusters.

Manage cluster services (HDFS, YARN, Spark, Hive, Impala, Kafka, etc.).

Monitor cluster health, performance, and capacity planning.

2. Security & Access Control:

Configure and manage Kerberos, LDAP/Active Directory integration, and TLS/SSL encryption. Implement Role-Based Access Control (RBAC) for users and services.

Ensure compliance with security best practices and audit logging.

3. Performance Tuning & Troubleshooting:

Optimize cluster performance by tuning configurations (YARN, HDFS, Spark).

Diagnose and resolve issues related to storage, compute, and networking.

Analyze logs and metrics using Cloudera Manager and other monitoring tools.

4. Data Processing & Optimization:

Develop, optimize, and maintain Spark, Hive, Impala, Nifi, and Kafka applications.

Write efficient SQL queries and optimize Hive/Impala performance.

Build data pipelines for batch and real-time processing

Requirements:

Must-Have Skills:

Linux & Command Line Proficiency: o Strong experience with Linux commands (ls, grep, awk, sed, chmod, ssh).

Managing file permissions, ownership, and shell scripting (Bash).

Programming Languages:

Python (for scripting and automation).

SQL (for database queries and management).

Good-to-Have Skills:

Programming Languages:

Java/Scala (nice to have for application support).

Networking & Security:

Understanding of TLS/SSL, certificates, and data-in-transit encryption.

Knowledge of networking protocols (HTTPS, DNS, NTP) and port management.

Familiarity with Microsoft Active Directory and LDAP authentication.

Scripting & Automation:

Experience with Ansible/Chef/Puppet for infrastructure automation. o Knowledge of GitLab/Bamboo for DevOps and CI/CD pipelines.

Academic Qualifications:

1. Bachelor’s Degree in:

Computer Science / Information Technology

Software Engineering

Data Analytics / Big Data Technologies.

OR

Diploma in:

Computer Science / Information Technology

Software Engineering o Data Analytics / Big Data Technologies

We are driven by our AEIOU beliefs—Adventure, Excellence, Integrity, Ownership, and Unity—and we seek individuals who embody these values in both their professional and personal lives. We are committed to our Impact: Valuing our clients, Growing our people, and Creating our future.