Senior Data Engineer (Python)
What’s the challenge?
We have an exciting opportunity to join us at comparethemarket.com as a Senior Data Engineer (Python). Your role will be pivotal in delivering and owning high performance data applications using cutting edge technology.
At CTM our Data Engineering team is at the heart of everything that we do. The team manages and works on multiple streams and projects that have high impact and importance for the business.
This role sits within the core Data Engineering team which is responsible for data processing and provision for the entirety of CTM. More specifically this role is within the current generation platform domain and is initially going to support the delivery of a regulatory project, followed by opportunities to get involved with wider projects for the business that will be focused within the data platform domain. As a Senior Engineer in our team, you will be driving the development of quality solutions for our customers and be proactive in adopting and championing best practices.
You’ll make a real impact by taking an active role in the team’s agile practices, technical decision making and development, generating value and continuously striving to improve the quality and reliability of our data.
What will it take to be a successful Senior Data Engineer (Python)?
Strong experience in Data Engineering - building strong, robust pipelines to ingest data from a variety of sources with a focus on real-time streaming (we process millions of events a day and deal with 100 TB’s of data here)
Experience in Python & Kafka
Extensive experience with Software Engineering Testing practices using TDD, BDD, ideally with Behave framework
Strong appreciation for modern software engineering principles (Agile, CI/CD, TDD/BDD)
Active learner, keeps up to date with industry knowledge and keen to learn new languages and tools
Self-motivated, natural leader and motivator. Passionate about Technology
Great communicator, influencer and negotiator
Ability to use your wealth of experience to find the right solution for any given task
AWS (awareness of Redshift, Data pipelines, IAM, S3, SQS etc.)
Experience with GOCD
Terraform – just to note that good engineering testing practices are key for this role
Experience mentoring junior engineers
Why choose us?
You might already know us - we're the ones with the Meerkats! Compare the Market has helped revolutionise the insurance market and we have been growing rapidly since starting in 2006.
We are on a mission to transform the comparison industry, moving beyond endless forms and generic search results, to a highly personalised, service-led experience. In order to do so, we recognise that we need to consistently deliver seamless, personalised customer experiences through a connected data and technology ecosystem.
Founded in 1992, the BGL Group has grown significantly in size and capability. From 30 to 3,000 people globally we are a leading digital distributor of insurance and household financial services.
We’re growing and we are home to such brands as; Les Furets, Beagle Street, Junction, Budget Insurance and Dial Direct. We are trusted by more than 10million customers in the UK alone, with this foundational success we continually grow and invest in our people and their ideas.
We enjoy what we do, and we have fun doing it!
Diversity and Inclusion
We're a diverse community of dedicated, innovative and talented professionals. With an inclusive and open workplace, we encourage our people to create and share ideas – supporting their growth and celebrating their uniqueness.
We understand the importance of achieving a healthy lifestyle balance, whether it’s working remotely or flexibly we have an environment where people are empowered to embrace flexible working in a way that works for them and for the business.
'We are committed to offering equal chances to you throughout the application process, so if you require any special adjustments to be made, please let us know. We will work with you to make any appropriate arrangements.’