Data Engineer

We are seeking a Data Engineer with experience or interest in IoT technologies and cloud-based data engineering to join our team. This role blends the management of high-volume data flows and IoT-specific analytics with general data engineering practices to deliver robust and scalable data solutions. The ideal candidate will balance technical skills and domain knowledge, enabling data-driven insights for utility operations, customer services, and infrastructure optimisation initiatives.

Key Responsibilities:

• Design, develop, and maintain scalable and efficient data pipelines and ETL processes for IoT and enterprise data systems.
• Implement data ingestion workflows from IoT devices and integrate with enterprise platforms using Azure Data Factory or similar tools.
• Ensure data quality through validation, cleansing, and monitoring processes to address issues such as missing data, duplicates, and inconsistencies.
• Define data attributes and formats for IoT device and network data to support seamless integration with existing systems and standards.
• Optimise data storage solutions in Azure Data Lake Storage (or equivalent) for structured and unstructured data.
• Develop APIs and data interfaces for real-time or near-real-time data transfer between IoT components and enterprise platforms.
• Apply advanced analytics techniques to IoT data for performance monitoring, usage profiling, and network management.
• Leverage BI tools (e.g., Power BI) to enable business intelligence and operational insights.
• Implement robust data security and privacy measures, ensuring compliance with relevant regulations.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality, documented solutions.

Required Skills & Qualifications:

• Bachelor’s degree in Computer Science, Information Technology, or a related field. Advanced degrees or certifications are advantageous.
• Academic or industrial experience in data engineering, including SQL Databases and cloud platforms (Azure, AWS, or GCP).
• Experience or interest in IoT technologies and data systems.
• Proficiency in data ingestion tools such as Azure Data Factory and ETL processes.
• Strong programming skills in Python, SQL, and one or more of Java or Scala.
• Familiarity with big data technologies (e.g., Hadoop, Kafka) and enterprise service buses (ESB).
• Knowledge of data management systems and integration with enterprise applications.
• Understanding of operational data flows, business cycles, and regulatory requirements.

Preferred Skills & Qualifications:

• Familiarity with additional Azure ecosystem tools (e.g., Synapse Analytics, Event Hub, Stream Analytics).
• Experience with APIs, data interfaces, and integrating IoT systems with enterprise data platforms.
• Relevant certifications in Microsoft Azure or other recognised credentials.
• Knowledge of DevOps practices and CI/CD pipelines (e.g., Azure DevOps).
• Familiarity with containerisation technologies such as Docker.
• Background in implementing Change Data Capture (CDC) designs and scalable data architectures.

Personal Attributes:

• Excellent communication and collaboration skills.
• Strong analytical and problem-solving mindset.
• Proactive approach to continuous professional development.
• Ability to work effectively in dynamic, fast-paced environments.

This position offers an opportunity to work at the intersection of IoT data systems and modern cloud engineering, supporting innovation and operational excellence across industries.