This is a hybrid role, based in G-Research’s office in Dallas.

G-Research is Europe’s leading quantitative finance research firm. We hire the brightest minds in the world to tackle some of the biggest questions in finance. We pair this expertise with machine learning, big data, and some of the most advanced technology available to predict movements in financial markets.

Opened in 2022, the Dallas office is a key infrastructure hub where we work on the latest technologies in a cutting-edge environment.

The role

The Big Data Platform Engineering team is expanding and is looking for a DevOps focused Infrastructure Engineer with demonstrable experience in HDFS, YARN and the Hadoop ecosystem.

The team manages a significant distributed HDFS storage footprint and have also started to explore Apache Ozone deployment on Kubernetes. Recent Airflow adoption and support is an exciting growth area for the team, while there is also an overlap in technology support within the team, in products such as Trino, Spark, Grafana, Hive, Nifi and HBase.

You will help us deliver automation, feature enhancements and best practice security on our existing hybrid cloud platforms, as well as explore new technologies and integration with other Open Source products within our ecosystem.

There is significant focus on configuration management, container orchestration, Infrastructure as Code (IaC) and CI/CD. In return, you will gain exposure to the latest hardware and software technologies in a forward- thinking company, which values innovation, personal development and training.

Key responsibilities of the role include:

Developing, deploying and maintaining big data solutions within the Hadoop ecosystem, such as HDFS, YARN and Ozone
Deploying and maintaining Airflow
Delivering engineering solutions ensuring they are scalable, stable, performant, and operationally efficient
Providing metrics, documentation, and self-service infrastructure to help our customers work at pace and get the most out of the platforms
Proactively identifying and addressing system performance bottlenecks, security vulnerabilities, and areas for improvement to ensure systems are stable and robust
Participating in the out of hours rota, providing support and maintenance for critical systems across the BigData ecosystem

Who are we looking for?

Whatever your technical background, good coding ability is essential for this role.

The ideal candidate will have the following skills and experience:

Strong HDFS, YARN and Hadoop ecosystem knowledge, preferably on the Cloudera Distribution
Good coding skills, preferably Python
Experience working with CI\CD pipelines, such as Jenkins and ArgoCD
Experience working with Configuration Management tools, such as Ansible and Terraform
Strong Linux, Hardware and Networking knowledge
Experience deploying and running applications on Docker and Kubernetes, including the creation of Helm charts
Experience working with distributed systems

Highly desirable but not essential:

Experience working with Ozone
Experience working with other Big Data technologies, such as Spark, Trino, Hive, Airflow orYARN
Data Engineering experience, such as Spark application development and SQL
Team Lead experience

Why should you apply?

Market-leading compensation plus annual discretionary bonus
Informal dress code and excellent work/life balance
Excellent paid time off allowance
Sick days, military leave, and family and medical leave
Generous 401(k) plan
12-weeks’ fully paid parental leave
Medical and Prescription, Dental, and Vision insurance
Life and Accidental Death & Dismemberment (AD&D) insurance
Employee Assistance and Wellness programs
Generous relocation allowance and support
Great selection of office snacks, and hot and cold drinks
On-site gym and car park

G-Research is committed to cultivating and preserving an inclusive work environment. We are an ideas-driven business and we place great value on diversity of experience and opinions.

We want to ensure that applicants receive a recruitment experience that enables them to perform at their best. If you have a disability or special need that requires accommodation please let us know in the relevant section

Big Data Infrastructure Engineer