How We Cut IoT Telemetry Memory Usage by 99%

When you are running a telemetry server that maintains 2,000+ concurrent TCP sessions from GPS and RFID devices, memory matters. We discovered our production pipeline was consuming 4.4GB of RAM - for what should have been a lightweight data relay. Here is how we diagnosed the problem and reduced it to 35.9MB.

The Symptom

Our telemetry pipeline follows a straightforward architecture: devices connect via TCP, data flows through a session manager, gets routed through RabbitMQ, and lands in a database handler. During routine monitoring, we noticed the process had ballooned to 4.4GB. For a service that reads bytes from sockets, parses them, and forwards to a message broker, that is orders of magnitude too high.

The Root Cause: Unbounded Deduplication Cache

The culprit was our MessageDeduplicationCache. The original implementation used a ConcurrentDictionary with no eviction policy. Every unique message hash was stored indefinitely. With thousands of devices sending telemetry every few seconds, the dictionary grew without bound.

The Fix: LRU Eviction with TTL

We redesigned the cache with three constraints:

• Entry limit: Maximum 50,000 entries
• TTL: 5-minute expiration per entry
• LRU eviction: Least recently seen hashes evicted first

public class MessageDeduplicationCache
{
    private readonly int _maxEntries = 50_000;
    private readonly TimeSpan _ttl = TimeSpan.FromMinutes(5);
    private readonly ConcurrentDictionary<string, DateTime> _cache = new();

    public bool IsDuplicate(string hash)
    {
        CleanExpired();
        if (_cache.TryGetValue(hash, out _)) return true;
        if (_cache.Count >= _maxEntries) EvictOldest();
        _cache[hash] = DateTime.UtcNow;
        return false;
    }
}

Results

Memory dropped from 4.4GB to 35.9MB. Duplicate detection rate went from about 35% down to about 3.3%, which represented actual retransmissions.

The lesson: any collection that grows with input volume and never shrinks is a memory leak waiting to happen.

What Else We Found

While investigating, we also fixed async void event handlers swallowing exceptions, Thread.Sleep in async contexts blocking the thread pool, and blocking waits on async operations causing thread starvation.

Building IoT infrastructure? Let's talk.