Location:
CN-Shenzhen-HyQ
Shift:
Scheduled Weekly Hours:
40
Worker Type:
Permanent
Job Summary:
Job Duties:
Key Responsibilities:
- Lead a high-performing data transformation POD of 8–10 engineers, driving delivery and technical excellence.
- Collaborate closely with data analysts and business stakeholders to gather requirements and deliver scalable, reliable data solutions.
- Design, build, and maintain robust data pipelines using SparkSQL and Python to process large-scale datasets efficiently.
- Manage and optimize data storage and retrieval within the Hadoop Distributed File System (HDFS).
- Utilize Trino to query and analyze data, supporting business intelligence and reporting needs.
- Ensure data quality, integrity, and security across all systems and processes.
- Monitor and fine-tune pipeline performance and query execution to ensure scalability and operational efficiency.
- Troubleshoot and resolve data-related issues promptly to minimize business impact.
- Manage production incidents and coordinate with the Site Reliability Engineering (SRE) Lead to oversee incident investigation and postmortem analysis.
Mandatory Skills and Qualifications:
- Minimum 8 years of experience in data engineering with minimum 3 years of data engineering management role.
- Strong expertise in SparkSQL for large-scale data processing and querying.
- Proven hands-on experience with HDFS for distributed data storage and management.
- Deep understanding of Trino for distributed SQL query execution.
- Proficiency in Python for scripting, automation, and data manipulation.
- Solid grasp of ETL architecture, data modeling, and pipeline design.
- Familiarity with distributed computing principles and big data ecosystems
- Basic proficiency in English and Mandarin, including both written and verbal communication skills.
Preferred Skills:
- Experience with Tableau for data visualization and dashboard development.
- Exposure to cloud platforms such as AWS or Huawei HCS.
- Knowledge of SQL performance tuning and query optimization techniques.
Company Introduction:
ITD SZ
港交所科技(深圳)有限公司,是2016年12月28日于深圳市前海自贸区成立的外商独资企业。
作为港交所的技术子公司,港交所科技(深圳)有限公司主要是为集团及其附属公司提供计算机软件、计算机硬件、信息系统、云存储、云计算、物联网和计算机网络的开发、技术服务、技术咨询、技术转让;经济信息咨询、企业管理咨询、商务信息咨询、商业信息咨询、信息系统设计、集成、运行维护;数据库管理、大数据分析;以承接服务外包方式提供系统应用管理和维护、信息技术支持管理、数据处理等信息技术和业务流程外包服务。