My client is looking for an experienced data-engineer to further mature its data-platform. With the platform, the company generates data with application in finance (capital markets). The platform focusses on large-scale text data processing to generate data for asset managers to make investment decisions.
Role and responsibilities
- Developing, testing and maintaining data loading and processing solutions (in the domain of large scale text-processing);
- Implement solutions to improve data reliability, efficiency and quality
- Apply and guard architecture principles;
- Build reliable, efficient and quality controlled solutions to support data scientists and business stakeholders;
- Organize work in an agile environment (task definition, sprint planning, etc).
Experience
- Testing, continuous integration, and in general DevOps;
- Making design decisions;
- Developing solutions for data ingestion, processing, and storage;
- Orchestration of data-pipeline components into workflows;
- Developing solutions in a cloud based environment;
- Experience with (and deep understanding of) large scale (text) data driven technologies.
Behaviour and knowledge:
- Know the strengths and weaknesses of each tool/technology and what it’s best used for;
- Previously demonstrated quickly gaining an understanding of a data domain or has Entis data domain (companies, securities, structured and unstructured data associated with them) experience;
- Understand how to deal with issues w.r.t scalability, consistency, reliability, efficiency, and maintainability, and security;
- Agile way of working in practice and knowing when it is appropriate to apply and when not;
- Team player, communicator, supportive, positive mindset, result driven.
Skills (Proficient to Expert level):
Required:
- Python (or one of the nice to have languages where they demonstrated the ability to properly build software in a new language via rapid learning);
- SQL;
- Strong software engineering and design ability w.r.t data intensive systems.
Nice to have:
- Scala/Java or C#;
- Spark/Databricks;
- Airflow;
- Azure Cloud (storage, Kubernetes and containers, SQL databases, Functions, virtual machines, etc).
Do you think we have a match?
Please leave your contact details and we will get in touch.