# Part 1: Building End-to-End Encrypted Group Messaging in React Native with OpenMLS

> A deep dive into end-to-end encryption for mobile apps. Explore Signal's Double Ratchet, why group membership changes get expensive in the pairwise model, and how OpenMLS solves secure group messaging in React Native.

Canonical article: https://margelo.com/blog/building-e2e-encrypted-group-messaging-with-openmls
LLM text: https://margelo.com/blog/building-e2e-encrypted-group-messaging-with-openmls/llms.txt
Author: Adnan Sahinovic
Category: Security
Tags: Security, Nitro, OpenMLS, Chat
Published: 2026-09-10
Reading time: 18 minutes

## Article

Most chat systems are built around trust.

Messages are sent to a backend, stored in plaintext, synchronized across devices, and rendered by clients. That architecture is simple to build, easy to scale, and works well, until you ask a dangerous question:

**Who can actually read the messages?**

The moment your backend can access plaintext conversations, your security model quietly expands far beyond the client application. Logs, backups, admin tooling, analytics pipelines, and every future vulnerability all become part of the trust boundary. All it takes is one weak point. At scale, there's always one.

In this series, we're building a system where that question has a better answer: a fully end-to-end encrypted group chat in React Native, powered by OpenMLS.

[Video: The trust boundary, doing its best.](https://media.giphy.com/media/11fot0YzpQMA0g/giphy.mp4)

End-to-end encryption changes that architecture completely.

Messages are encrypted on-device before they ever touch the network. The server can't read message contents or private keys, but it still plays a critical role: storing and distributing KeyPackages, establishing a canonical order for handshake messages, and validating credentials. The client stops being just a UI. It becomes responsible for cryptographic state, synchronization, and security guarantees.

[Video: MLS demo app: an end-to-end encrypted conversation. Every message is encrypted on-device before it leaves the phone; the server only handles ciphertext and public key material.](https://margelo.com/videos/open-mls-chat.mp4)

This is where **MLS (Messaging Layer Security)** comes in.

MLS is the IETF standard for secure group messaging, specified in [RFC 9420](https://datatracker.ietf.org/doc/rfc9420/). Unlike Signal's Double Ratchet, which was designed around one-to-one conversations, MLS was designed around groups from day one. Instead of managing dozens or hundreds of encrypted relationships independently, MLS treats the entire group as a single synchronized cryptographic system.

That shift becomes critical once groups begin to grow.

We'll build that architecture with:

- [OpenMLS](https://github.com/openmls/openmls): a Rust implementation of the MLS protocol
- **Rust**: the home of all cryptographic state
- [Nitro Modules](https://nitro.margelo.com): fast, typed native bindings
- **JSI**: zero-copy binary data transfer

The goal isn't just to encrypt messages. It's to design a system capable of safely managing synchronized cryptographic state on mobile devices at scale.

But before touching the code, we need to understand why MLS exists at all, because it introduces concepts that are very different from traditional client-server messaging.

![Traditional chat backend vs end-to-end encrypted MLS architecture: in a traditional backend, plaintext reaches the server and the trust boundary expands to logs, backups, admin tools, analytics, and future vulnerabilities; everyone inside can read messages. With E2EE, messages leave the client as ciphertext, the server only stores public KeyPackages and relays ciphertext, and decryption happens only on the recipient's device.](https://margelo.com/img/mls-concept.png)

> Traditional chat backend vs MLS: with a traditional backend, everyone inside
> the server's trust boundary can read messages. With MLS, the server only
> relays ciphertext and public KeyPackages; decryption happens exclusively
> on-device.

## Understanding MLS Before the Code

Most developers already know *what* end-to-end encryption is. What makes MLS different is *how* it solves secure group communication at scale, without turning every membership change into a cryptographic nightmare.

MLS is built from a small set of primitives. Once you understand them, the whole protocol clicks into place.

### Identity

Every device in MLS owns a cryptographic identity, containing:

- signing keys
- credentials
- cryptographic metadata

The identity is persistent and is used to sign every important protocol operation. Think of it as this app installation's permanent cryptographic passport inside the MLS ecosystem.

Note that identity belongs to the *device*, not the user. If a user has three devices, they have three MLS identities. Multi-device support is built on top of this model, not into it.

### KeyPackages

A **KeyPackage** is a public bundle that allows other users to add a device into a group securely. It contains:

- public encryption keys
- supported cipher suites
- protocol metadata
- a signature from the device's identity

KeyPackages are what make MLS *asynchronous*. When Alice wants to start a secure conversation with Bob, she fetches Bob's KeyPackage from the backend; Bob doesn't need to be online. The corresponding private keys stay on Bob's device; the server only holds public bundles.

KeyPackages are also single-use: once one is consumed to add a device to a group, it can't be reused, so each device must keep the server stocked with fresh ones. We'll cover replenishment in Part 2.

This is one of the server's two real jobs: acting as a public directory of KeyPackages. The other, ordering handshake messages, shows up when we get to Commits.

### Groups

MLS treats a group as a **synchronized cryptographic state machine**.

Every participant shares:

- group state
- the current epoch
- ratchet tree state
- encryption secrets

This is the biggest conceptual difference compared to traditional chat systems: messages aren't independent encrypted payloads. They're outputs *derived from* shared group state. If your local state diverges from the group's, you can no longer produce or consume valid messages.

### Epochs

Every time the group's membership or key material changes, MLS advances the **epoch**: a version number for the group's cryptographic state. Epoch changes happen when:

- members are added
- members are removed
- keys are rotated

Each of these lands as a Commit (next up), and each new epoch derives fresh encryption secrets through the MLS key schedule. Application messages are deliberately different: they don't advance the epoch. They advance a *per-sender ratchet* within the current epoch, which keeps day-to-day messaging cheap while membership changes remain the expensive, strictly ordered events.

Once the epoch advances, older secrets are no longer sufficient to participate in future communication. This gives MLS two crucial properties:

- **Forward secrecy**: compromising today's keys doesn't reveal yesterday's messages.
- **Post-compromise security**: if a device's keys are compromised, a later Commit that rotates its leaf keys (or removes it) heals the group and locks the attacker out of future epochs.

Post-compromise security is one of MLS's most important advantages over older group messaging protocols. A breach isn't permanent; the group can heal.

### Commits

**Commits** are the group's state transitions. A Commit applies pending proposals (adds, removes, key updates), rotates the affected parts of the ratchet tree, and moves every member onto the next epoch together.

This is how MLS evolves safely over time: not as a stream of independent key updates, but as an ordered sequence of atomic state transitions. It's also why the server (in MLS terms, the Delivery Service) must establish a canonical order for handshake messages: two conflicting Commits for the same epoch can't both apply, so something has to pick the winner.

### Welcome Messages

When a new participant joins a group, they receive a **Welcome** message: the encrypted information required to reconstruct the current group state. Without it, the new member can't derive the shared secrets needed to decrypt anything.

Crucially, the Welcome only grants access from the current epoch forward. New members can't decrypt the group's history.

## Why MLS Is Different From Signal

**Sender Keys make group messages cheap. MLS makes group changes scale.**

That's the fundamental difference.

Signal's Double Ratchet excels in one-to-one conversations: keys evolve with every message, old keys are discarded, and each message lives in its own cryptographic context. For groups, Signal adds Sender Keys, which make message *fan-out* efficient: each sender encrypts once instead of once per recipient.

But membership changes are where the pairwise model strains. When someone leaves a Sender Keys group, every remaining member must generate a new sender key and distribute it pairwise to everyone else. The cost of a membership change grows with the size of the group, and the state fragments into many independent relationships that all must be kept consistent.

MLS approaches the problem differently: it treats **the group itself as the primitive**.

### The Ratchet Tree

At the core of MLS is a ratcheting binary tree (the construction is called TreeKEM).

Each participant occupies a leaf node, and internal nodes hold key material derived from their children. The secret at the root doesn't encrypt messages directly: it feeds the MLS key schedule, which derives the epoch's actual encryption and authentication secrets.

![The MLS ratchet tree (TreeKEM): eight members (Alice, Bob, Carol, Dave, Eve, Frank, Grace, and Heidi) sit at the leaves of a binary tree whose internal nodes derive shared secrets and whose root secret feeds the MLS key schedule. When Dave rotates his key, only the path from his leaf up to the root updates; every other node in the tree remains unchanged.](https://margelo.com/img/mls-ratchet-tree.png)

The tree structure is what makes MLS scale. When membership changes, only the nodes along the affected member's path to the root rotate; the rest of the tree is untouched. Instead of updating O(n) pairwise relationships, a membership change touches O(log n) nodes.

For a group of 1,000 members, that's the difference between updating \~1,000 relationships and updating \~10 tree nodes. As groups grow, this gap becomes the difference between "works" and "unusable."

Here's what that gap looks like on the same axis: distributing new key material to every member individually grows linearly with group size, while the ratchet-tree path depth barely moves.

![Illustrative scaling of logarithmic tree depth vs a linear baseline: idealized balanced-tree depth (log₂ n) compared with a naive per-member delivery function as group size grows from 1 to 128 on linear axes. The linear baseline climbs to n = 128 while log₂(128) is just 7. Illustrative functions only — not MLS payload, runtime, or total membership-change-cost data (RFC 9420 §§4.1-4.1.2, 7.5-7.6).](https://margelo.com/img/scale-open-mls-graph.svg)

## Why React Native Makes This Hard

MLS immediately breaks many assumptions React Native apps are built around.

Most React Native apps follow a familiar flow: fetch data, render UI, send updates. MLS is nothing like that. The protocol is binary-heavy, stateful, mutation-driven, and sensitive to synchronization errors. A "simple" message send becomes a cryptographic state transition, and every operation can produce multiple binary artifacts: ciphertext, Commits, proposals, ratchet tree updates, and new group state snapshots.

Trying to run this inside JavaScript creates problems immediately:

- **Serialization overhead**: constant conversion of binary data at the bridge
- **Memory copying**: large encrypted blobs duplicated on every crossing
- **Binary inefficiency**: JS isn't built for heavy byte-level manipulation
- **GC unpredictability**: sensitive key material at the mercy of the garbage collector
- **Weak runtime guarantees**: no reliable way to ensure constant-time execution or secure zeroization

We quickly realized we didn't want cryptographic logic (or synchronized MLS state) living inside JavaScript at all.

Instead, we built the entire MLS layer in Rust and exposed it to React Native through Nitro Modules and JSI.

## Architecture Overview

Our stack has four layers:

```
┌─────────────────────────────────────┐
│  TypeScript      developer API      │
├─────────────────────────────────────┤
│  C++ / Nitro     JSI integration    │
├─────────────────────────────────────┤
│  C FFI           stable ABI         │
├─────────────────────────────────────┤
│  Rust + OpenMLS  all cryptography   │
└─────────────────────────────────────┘
```

Why four layers? Because no single runtime can safely solve every problem:

- **Rust** executes cryptography.
- **C** provides a stable ABI.
- **C++** integrates with JSI.
- **TypeScript** exposes a developer-friendly API.

Each layer exists because the layer above it can't safely perform that responsibility.

The most important architectural decision: **all cryptographic operations, and all interpretation of state, stay inside the native Rust layer.** JavaScript only ever handles opaque serialized snapshots. The React Native layer communicates through Nitro Modules and JSI using opaque binary buffers.

This matters because MLS is fundamentally stateful. Every encrypt, decrypt, add, or remove operation mutates synchronized group state. Keeping those operations in Rust avoids bridge serialization, eliminates unnecessary copies, and isolates sensitive cryptographic logic from the JavaScript runtime.

Let's walk through each layer, bottom to top.

## Layer 1: Rust + OpenMLS

All cryptographic logic lives in Rust: identity creation, key generation, encryption, decryption, group evolution, and serialization. Rust gives us deterministic memory management, strong safety guarantees, and a natural environment for long-lived cryptographic state.

```toml
[dependencies]
openmls              = { version = "0.5", features = ["test-utils"] }
openmls_rust_crypto  = "0.2"
serde                = { version = "1", features = ["derive"] }
bincode              = "1"
```

A note on that `test-utils` feature: it's enabled here for our integration tests only. It has no place in a production build.

We pinned OpenMLS to version 0.5 because it supports simpler serde-based group serialization. That costs us improvements from later versions, but the simpler serialization model reduced integration complexity for our use case.

A design decision that shaped everything else: **all MLS state is treated as an opaque serialized blob.** JavaScript never deserializes group state. Rust serializes everything (the group, the key store, the identity) into a single versioned binary snapshot:

```rust
#[derive(Serialize, Deserialize)]
pub struct GroupStateBlob {
    pub version: u32,
    pub mls_group_json: Vec<u8>,
    pub key_store: HashMap<Vec<u8>, Vec<u8>>,
    pub identity: IdentityBlob,
    pub group_id: Vec<u8>,
}
```

This gave us simpler persistence, lower overhead, and native-controlled migrations: the `version` field lets Rust upgrade old snapshots as the format evolves, without JavaScript ever knowing. To JavaScript, group state is just bytes to store and pass back.

This softens, but doesn't eliminate, the GC concern from earlier: JavaScript never operates on raw key material, but the opaque blob it shuttles around does contain private keys. Opaque isn't the same as harmless, so the blob must be treated as a secret (more on storage below).

### Messages Mutate State

If you take one thing away from this post, make it this.

Encryption is *not* simply:

```
plaintext → ciphertext
```

In MLS, encryption looks like this:

```
state + plaintext → new_state + ciphertext
```

Encrypting an application message advances the sender's ratchet within the current epoch and produces the next state this device must hold onto; decrypting advances the matching ratchet on the receiver's side. Commits go further and advance the whole group's state at once. In every case, the old state is consumed. **State persistence becomes security-critical.**

### Encrypting a Message

Our Rust encrypt function makes this explicit:

```rust
pub fn encrypt_message(
    group_state_data: &[u8],
    plaintext: &str,
) -> Result<(Vec<u8>, Vec<u8>), String> {

    // Deserialize the blob back into a live MLS group
    let (mut group, provider, identity, gid) =
        unpack(group_state_data)?;

    let signer = signer_from_identity(&identity)?;

    // Encrypt — this mutates the group's internal state
    let mls_msg_out = group.create_message(
        &provider,
        &signer,
        plaintext.as_bytes(),
    )?;

    let ciphertext =
        tls_codec::Serialize::tls_serialize_detached(&mls_msg_out)?;

    // Serialize the *new* state — the old blob is now stale
    let new_state =
        repack(&group, &provider, identity, gid)?;

    Ok((new_state, ciphertext))
}
```

Notice the return type: the function returns both the **ciphertext** *and* the **updated group state**. Both are required. The old state is stale the moment this function returns. We'll see why that matters when we get to persistence.

## Layer 2: The C FFI Boundary

Rust doesn't expose a stable ABI, so we created a thin C layer between Rust and C++. The FFI layer handles:

- memory ownership
- panic safety
- error propagation
- raw pointer conversion

Our core buffer type is deliberately boring:

```c
typedef struct {
    uint8_t* data;
    size_t len;
} ByteBuffer;
```

The ownership contract is simple: **Rust allocates, C++ frees**. C++ never calls `free()` directly; it triggers deallocation by calling back into Rust's exported `openmls_free_buffer` function, so the buffer is always released by the same allocator that created it.

Every exported function is wrapped with `catch_unwind()`. A Rust panic unwinding across an FFI boundary is undefined behavior, so panics are caught at the edge and converted into error codes instead of crashing the app.

## Layer 3: Nitro Modules + JSI

Traditional React Native bridges are inefficient for cryptographic workloads because they rely on serialization. MLS constantly produces binary artifacts (ciphertext, Commits, Welcome messages, group state snapshots), and pushing all of that through JSON would mean base64-encoding every buffer and copying it multiple times per call.

Nitro Modules solve this with JSI and HybridObjects: instead of serializing data, we pass raw `ArrayBuffer`s directly between native code and JavaScript.

The Nitro spec is plain TypeScript:

```typescript
export interface EncryptResult {
  newGroupStateBlob: ArrayBuffer
  ciphertext: ArrayBuffer
}
```

And the C++ bridge wraps Rust-owned memory directly into a JSI buffer. There's no copy, just a transfer of ownership with a custom deleter:

```cpp
return ArrayBuffer::wrap(buf.data, buf.len, [buf]() {
  openmls_free_buffer(buf);
});
```

When the JS garbage collector releases the `ArrayBuffer`, the deleter runs and Rust's allocator reclaims the memory. This gives us:

- zero-copy memory transfer
- lower CPU overhead
- fewer allocations
- better performance

For an encrypted messaging system that moves binary blobs on every single message, this adds up quickly.

## Layer 4: TypeScript

Finally, we expose a clean API to the application:

```typescript
const { newGroupStateBlob, ciphertext } =
  OpenMLSManager.encryptMessage(groupState, 'Hello world')
```

The application layer stays simple. Underneath that one call, the full MLS stack is executing: OpenMLS, Rust, FFI, C++, Nitro, JSI, ratchet advancement, and state re-serialization.

That's the point of the architecture: the complexity exists, but it lives where it belongs.

## Persisting State Correctly

Remember how encryption returns a new state blob? Here's where that bites.

One of the easiest ways to break an MLS application is failing to persist updated group state. This looks reasonable, and is **wrong**:

```typescript
// ❌ WRONG: the new state is silently discarded
const { ciphertext } =
  OpenMLSManager.encryptMessage(groupState, text)

send(ciphertext)
```

This is correct:

```typescript
// ✅ Persist the new state FIRST, then send
const { newGroupStateBlob, ciphertext } =
  OpenMLSManager.encryptMessage(groupState, text)

// MMKV stores ArrayBuffers directly — no base64 roundtrip
storage.set(`mls_group_${groupId}`, newGroupStateBlob)

await send(ciphertext)
```

The order matters, too: persist before sending. If the app crashes after sending but before persisting, the device has broadcast a message derived from a state it no longer remembers.

In production, go one step further and make the pair atomic: write the new state blob and the outgoing ciphertext in a single transaction (an outbox), then transmit from the outbox. That way, a crash can never separate the state from the message that consumed it.

Where you persist matters as much as when: the group state blob contains private key material, so it can't sit in plain storage. Store it encrypted at rest, with the encryption key protected by the iOS Keychain or Android Keystore. And version what you store: the `version` field in `GroupStateBlob` is what lets the native layer migrate old snapshots as the format evolves.

Concurrency has the same failure mode. Because every operation consumes the current blob and produces the next one, two operations on the same group must never run in parallel; operations per group have to be serialized. We'll cover how in Part 2.

And losing state is severe. This isn't a bug you can retry your way out of: if the updated state is lost, the device's ratchets fall out of sync with the group. Replaying ciphertext from the server won't rebuild the missing secrets, so recovery generally means rejoining: another member re-adds the device using a fresh KeyPackage, and history from before the rejoin stays unreadable, by design.

This is why, in practice, state management is often harder than the cryptography itself.

## The Full Message Flow

Putting it all together, a complete MLS conversation bootstraps like this:

![Sequence diagram of bootstrapping an encrypted MLS conversation between Alice, the server, and Bob: Alice and Bob each upload a public KeyPackage; Alice fetches Bob's KeyPackage, creates the group, and adds Bob; the encrypted Commit is sent to the server, which enforces delivery order; the encrypted Welcome is relayed through the server to Bob, who joins the group from it; encrypted messaging then flows in both directions. The server only ever sees public KeyPackages and ciphertext.](https://margelo.com/img/mls-message-flow.svg)

Notice the server's role in this flow: it stores public KeyPackages, relays opaque messages, and orders handshake traffic. It can't read a private key, a group secret, or a message body. What it can observe is metadata: which devices talk to which groups, when, and how much. E2EE protects content, not traffic patterns; that's a separate problem with separate tools.

After the group exists, every application message advances its sender's ratchet, and processing it advances the matching ratchet on each receiver. Membership changes are different: a Commit moves everyone to the next epoch at once, and there's no partial progress. Either the whole group advances, or nobody does.

Here's that exact flow running live in our demo app, from creating a group and adding a member to exchanging encrypted messages through the full stack:

[Video: MLS demo app: creating a group, adding a member via their KeyPackage, and exchanging encrypted messages. Commits advance the group's epoch; each message advances its sender's ratchet.](https://margelo.com/videos/open-mls-group-create.mp4)

## Performance

So what does this four-layer stack cost at runtime?

For application messages, very little. On-device, with state deserialization, ratchet advancement, and re-serialization happening underneath every call:

| Operation  | Time     |
| ---------- | -------- |
| Encryption | \~0.80ms |
| Decryption | \~0.66ms |

These aren't theoretical benchmarks: they're typical timings for application messages in our demo app (a small group), measured end-to-end through the entire stack: TypeScript call, JSI crossing, FFI boundary, OpenMLS execution, state re-serialization, and zero-copy buffer return. Handshake operations like Commits and joins do more work and scale with group size; we'll measure those separately in Part 2.

For this workload, sub-millisecond crypto puts encryption well below typical rendering and network costs in a messaging UI. The architecture (Rust for the work, zero-copy JSI for the transport) is what keeps it there.

## What Comes Next

We now have the foundation:

- MLS models the group as a single synchronized cryptographic state machine.
- Application messages advance per-sender ratchets; Commits advance the epoch for everyone.
- Epochs, Commits, and the ratchet tree make membership changes scale as O(log n).
- All cryptographic state lives in Rust, crossing into JavaScript only as opaque zero-copy buffers.
- State persistence is security-critical: losing it means losing your seat in the group.

In Part 2, we go deeper into the implementation:

- Building the group lifecycle: creating groups, adding and removing members, processing Welcomes
- Handling Commits, proposals, and epoch synchronization across devices
- Designing the delivery service: ordering guarantees and KeyPackage distribution
- Storage, migrations, and recovering from state corruption

The cryptography is solved. The engineering is where the real work begins.
