Metadata-Driven Telemetry: Decoupling Signal Routing from Firmware

In most IoT systems, the server knows what signals to expect because someone hardcoded those definitions into both firmware and backend. When a new sensor is added, you update firmware, server parser, database schema, dashboards - and hope nothing breaks in devices you cannot physically reach.

This is the problem DTCM - Dynamic Telemetry Configuration Middleware - was designed to solve.

The Problem with Hardcoded Signals

• Firmware-server coupling: Adding a sensor means coordinating releases simultaneously
• Blast radius: A parsing bug in one signal can crash processing for all signals
• No gradual rollout: You cannot add a signal for some devices without affecting others
• Testing complexity: Every combination needs integration testing across firmware versions

The DTCM Architecture

DTCM introduces a metadata layer between the raw telemetry stream and application logic. The system reads signal definitions from a configuration store at runtime instead of hardcoding them. Key components include a Signal Registry, Dynamic Parser, Routing Engine using MQTT 5.0, and Configuration API.

{
  "signalId": "fuel_level",
  "deviceType": "FM-4200",
  "byteOffset": 24,
  "dataType": "uint16",
  "unit": "percentage",
  "validation": { "min": 0, "max": 100 },
  "routing": {
    "topic": "telemetry/fleet/{deviceId}/fuel",
    "qos": 1
  }
}

Challenges Addressed

The design went through eight rounds of architectural criticism. We eliminated single points of failure through Redis-backed caching with local fallback, isolated signal parsing failures per signal rather than per packet, and secured the configuration API with RBAC.

In IoT, firmware is the hardest thing to change. Anything you can move into server-side configuration is a win for operational agility.

Fighting firmware coupling? We have been there.