theduykhoa

Building E2E Encrypted Communication Systems with Signal Protocol (in Rust) - Part 1

4 min read

📌 TL;DR

End-to-end encryption isn’t just about keeping secrets from a hacker / eavesdropper — it’s about building systems where users don’t have to trust anyone but themselves, including the service provider who offers the communication service (i.e. the server).

In this series, we’ll build a simple E2E encrypted communication system that is post-quantum secure based on the libsignal library from the Signal team using Rust.

In Part 1 (this post), we’ll cover:

  • 🎯 Why E2E encryption matters (security vs privacy vs trust)
  • 🔐 What the Signal Protocol offers (and its limitations)
  • 🏗️ How the protocol works conceptually (key exchange, encryption layers)
  • 🧠 Theory behind PQXDH and post-quantum cryptography

By the end of this 3-part series, you’ll have implemented:

  • ✅ Post-quantum secure key exchange (PQXDH)
  • ✅ End-to-end encrypted file sharing
  • ✅ A working demo that encrypts and decrypts images

Code coming in parts 2 and 3!

The Problem: Learning E2E Encryption is Hard

These days, I’ve been working on building an end-to-end encryption system for SyftBox — an open-source network for privacy-first, offline-capable AI. However, when I started researching how to implement it based on libsignal, I felt like hitting a wall.

Signal Protocol

The Signal Protocol is the gold standard for implementing E2E encryption systems—it’s what powers Signal, WhatsApp, and many other secure messaging apps. However, the whitepaper is highly technical and rigorous, making it difficult to understand without a strong cryptography background. I was trying to search for good learning materials that bridge theory and practice, but couldn’t find anything that really clicked. So I spent time diving deep into the protocol, implementing it, and visualizing all the concepts to truly understand how it works. This series shares what I learned, and my goal is to help you save time understanding and implementing your own E2E encryption system using libsignal, with clear explanations and visual aids along the way.

Security vs Privacy: Understanding the Difference

While security protects you against external eavesdroppers and attackers, privacy refers to your right to control how your personal information is collected, stored, and used. This distinction is crucial when comparing TLS (the technology behind HTTPS) and end-to-end encryption:

  • TLS provides security against external threats but still allows the service provider (server) to access your data
  • E2EE delivers true privacy by ensuring that not even the service provider can read your messages—only you and your intended recipient hold the keys.

TLS vs E2EE: A Visual Comparison

TLS vs End-to-End Encryption Comparison

In the diagram above:

  • Traditional TLS Encryption (left): The server decrypts and re-encrypts messages, meaning it can read your communications. This protects against external attackers but not against the service provider itself.
  • End-to-End Encryption (right): The server only forwards encrypted data without being able to decrypt it. Only Alice and Bob can read the messages, giving you both security and privacy.

🤔 Why does the server need to decrypt and re-encrypt with TLS?

The server keeps separate encryption keys with Alice and with Bob. When Alice sends a message, the server decrypts it using Alice’s keys, then re-encrypts it using Bob’s keys. This means the server sees the plaintext message in between—protecting you from network hackers, but not from the service provider itself.

The Signal Protocol: Battle-Tested E2E Encryption

We will build our E2E encryption system based on Signal Protocol - the gold standard for E2E encrypted messaging:

  • Used by Signal (obviously), WhatsApp, Facebook Messenger, Google Messages
  • Protects billions of messages every day
  • Open source and academically vetted

Why Signal Protocol?

  1. Forward Secrecy - New keys for every message
  2. Asynchronous - Works even when recipients are offline
  3. Authenticated - You know who sent each message
  4. Deniable - Can’t cryptographically prove who said what
  5. Post-Quantum Ready - Updated to include Kyber for quantum resistance

What is PQXDH?

PQXDH (Post-Quantum Extended Diffie-Hellman) is Signal’s latest key agreement protocol, designed to resist quantum computers. It’s currently implemented in the libsignal library (as of Nov 2025) and will be the foundation of our E2E encryption system.

The basic idea:

  • Alice and Bob each have several keys (identity, signed prekeys, quantum keys)
  • They perform multiple Diffie-Hellman exchanges
  • They combine the results into a shared secret
  • This secret is used to derive encryption keys

We’ll implement a simplified 3-key version in this series (we skip the optional one-time prekeys)

References

Graph View