Data Engineer

Data Engineer | Blockchain | London | Hybrid

Data Engineer with a keen focus on blockchain and distributed ledger technology required for a not-for-profit organisation focused on developing the blockchain ecosystem.

The Data Engineer will be pivotal in managing, curating, optimising, and securing datasets specifically related to cryptocurrency discussions across various platforms.

The ideal candidate will be adept in web scraping, data quality assurance using AI, data integration, ensuring data security and compliance, and maintaining detailed documentation.

What's on offer to you?

• Work with leading academics


• Work with leading blockchain technology


• Be part of an exciting new project with AI

What You Will Be Doing

• Data Collection: Identify relevant chat sources, groups, and forums on platforms discussing particular topics.
  
  Maintain and develop web scraping tools or APIs for periodic data extraction.


• Data Quality Assurance: Develop and implement AI-based procedures for quality control of data and data sources to eliminate inaccuracies and anomalies.
  
  Create tools for monitoring data sources for changes and updates, adapting data collection and cleaning processes accordingly.


• Data Integration: Collaborate with data scientists and analysts to integrate collected data into various projects and analysis tools.
  
  Ensure smooth data flow and integration with other data sources within the organisation.


• Data Security and Compliance: Uphold the security and privacy of collected data in compliance with relevant regulations and company policies.


• Documentation: Maintain clear and comprehensive documentation of data sources, collection methods, and workflows.
  
  Produce reports and documentation for both internal and external stakeholders as required.


• Monitoring and Reporting: Develop and maintain systems to monitor the performance and health of data collection processes.

What You Will Need to Succeed in This Role

• Bachelor's degree in Computer Science, Data Science, or a related field.


• Knowledge of Data Structures and Databases is a must.


• Demonstrable experience in data engineering or a similar role, with a focus on web scraping and data collection.


• Proficient in programming languages such as Python, SQL.


• Knowledge in TypeScript is a must.


• Familiarity with blockchain technology.


• Knowledge of data privacy laws and compliance requirements.


• Strong analytical and problem-solving skills.


• Excellent communication and collaboration abilities.


• Preferred: Advanced degree in a relevant field.


• Preferred: Experience with big data technologies and cloud services.


• Preferred: Proficiency in AI and machine learning techniques for data quality assurance.

Keywords: Data Engineer | AI | Blockchain | SQL | Typescript