# KMS Performance Comparison

**Versions**: `v5.25.0`  
**Generated**: 2026-11-01

---

## Benchmark Environment

| Field | Value |
|---|---|
| Date | 2026-11-01 16:26:14 UTC |
| HSM backend | None (Software) |
| Command | `mise run bench:load` |
| Build | release / non-fips |
| Database | SQLite (temporary, single benchmark run) |
| CPU | AMD EPYC 7R13 Processor @ 3,604 MHz |
| CPU cores | 2 physical / 4 logical (HT) |
| RAM | 7.0 GB |
| OS | Ubuntu 24.04.4 LTS |
| Kernel | 6.17.0-1019-aws |

### Load test parameters

| Parameter | Value |
|---|---|
| Mode | all |
| Protocols | all |
| Measurement window | 20 s per concurrency level |
| Concurrency levels | 1,2,4,8,16 |
| Warm-up | 5 s |
| Cooldown between levels | 2 s |

### CPU detail (`lscpu`)

```text
Architecture:                            x86_64
CPU op-mode(s):                          32-bit, 64-bit
Address sizes:                           48 bits physical, 48 bits virtual
Byte Order:                              Little Endian
CPU(s):                                  4
On-line CPU(s) list:                     0-3
Vendor ID:                               AuthenticAMD
Model name:                              AMD EPYC 7R13 Processor
CPU family:                              25
Model:                                   1
Thread(s) per core:                      2
Core(s) per socket:                      2
Socket(s):                               1
Stepping:                                1
BogoMIPS:                                5300.00
Flags:                                   fpu vme de pse tsc msr pae mce cx8 apic sep pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat npt nrip_save vaes vpclmulqdq rdpid
Hypervisor vendor:                       KVM
Virtualization type:                     full
L1d cache:                               64 KiB (2 instances)
L1i cache:                               64 KiB (2 instances)
L2 cache:                                1 MiB (2 instances)
L3 cache:                                8 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-3
Vulnerability Gather data sampling:      Not affected
Vulnerability Ghostwrite:                Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Old microcode:             Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Mitigation; Safe RET
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; Retpolines; IBPB conditional; IBRS_FW; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Mitigation; Clear CPU buffers
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
```


---

## Architecture

```mermaid
graph TB
    subgraph cli["Local: ckms CLI"]
        ckms["<b>ckms load</b><br/>(concurrent load driver)"]
    end
    
    subgraph srv["KMS Server"]
        kmip["KMIP 2.1<br/>(HTTP/TLS)"]
        crypto["Software Crypto<br/>(OpenSSL 3.6.2)"]
        db["Key Storage<br/>(SQLite/PostgreSQL)"]
    end
    
    ckms -->|HTTP/TLS| kmip
    kmip --> crypto
    crypto --> db
    
    style ckms fill:#4A90E2
    style kmip fill:#7ED321
    style crypto fill:#F5A623
    style db fill:#BD10E0
```

**Components**:
- **ckms**: Configurable concurrent load generator (ops/sec sweep)
- **KMIP Endpoint**: Protocol handler for request/response marshaling
- **Software Crypto**: AES-GCM, RSA-PKCS, ECDSA-P256, EdDSA-Ed25519 via OpenSSL 3.6.2
- **Key Storage**: Key material persistence (SQLite or PostgreSQL)



---

## Protocols

The KMS server was exercised over three distinct wire protocols.
Each benchmark column is labelled with the protocol name it used.

| Protocol | Transport | Encoding | Endpoint | Description |
|---|---|---|---|---|
| **ttlv-json** | HTTP/1.1 | KMIP 2.1 JSON-TTLV | `POST /kmip/2_1` | Primary interoperability protocol — any KMIP 2.1 compliant client can use it |
| **ttlv-bytes** | HTTP/1.1 | KMIP 2.1 binary TTLV | `POST /kmip` | Binary wire format; eliminates JSON parsing overhead — typically 10–30 % faster |
| **jose** | HTTP/1.1 | JWE / JWS (JOSE) | `POST /v1/crypto/` | REST API for OAuth2/OIDC workloads that prefer JWA algorithm identifiers over KMIP |

**KMIP TTLV** (Tag-Type-Length-Value) is the native encoding of the KMIP 2.1 standard (OASIS KMIP Spec v2.1, §9.1). The **JSON** variant wraps every field in a `{"tag": …, "type": …, "value": …}` JSON object and base64-encodes binary values. The **binary** variant uses a compact 8-byte fixed header (3-byte tag, 1-byte type, 4-byte length) per value, removing JSON tokenisation, base64, and UTF-8 overhead entirely.

**JOSE** (JSON Object Signing and Encryption, RFC 7516 / RFC 7515) exposes KMS key material through `/v1/crypto/` REST endpoints. It is used by cloud integrations (Google CSE, Microsoft DKE, Azure EKM) and any workload that speaks JWA algorithm identifiers (A256GCM, RS256, ES384 …) rather than KMIP semantics.


---

## Benchmark Methodology

### Plaintext / payload sizes

All encrypt/decrypt benchmarks use a **fixed-size random payload**. Sizes represent a realistic key-wrapping or small-message encryption workload without introducing significant data-transfer overhead on a loopback connection.

| Algorithm / category | Plaintext size | Notes |
|---|---|---|
| AES-GCM (128 / 192 / 256-bit key) | 64 bytes | FIPS 140-3 |
| AES-GCM-SIV (128 / 256-bit key) | 64 bytes | Non-FIPS |
| AES-XTS (128 / 256-bit AES = 256 / 512-bit key) | 64 bytes | FIPS 140-3; requires 16-byte IV |
| ChaCha20-Poly1305 (256-bit key) | 64 bytes | Non-FIPS |
| ECIES — P-256 / P-384 / P-521 | 64 bytes | Non-FIPS; EC public-key encryption |
| Salsa Sealed Box (X25519) | 64 bytes | Non-FIPS |
| Covercrypt (attribute-based encryption) | 64 bytes | Non-FIPS |
| JOSE JWE — `dir` + AES-GCM (A128GCM / A192GCM / A256GCM) | 64 bytes | Symmetric (direct key agreement) |
| JOSE JWE — RSA-OAEP + AES-GCM (2048 / 4096-bit) | 64 bytes | Asymmetric (RSA-OAEP CEK wrapping) |
| RSA-OAEP (2048 / 3072 / 4096-bit) | 32 bytes | Limited by RSA block size |
| RSA-PKCS#1 v1.5 (2048 / 3072 / 4096-bit) | 32 bytes | Non-FIPS |
| RSA-AES Key Wrap — KWP (2048 / 3072 / 4096-bit) | 32 bytes | FIPS 140-3 |
| Sign / Verify — all algorithms | 32 bytes | Message is hashed internally |
| JOSE JWS / MAC | 32 bytes | |

### Load test (`mise run bench:load`)

The load test sweeps a configurable list of concurrency levels. At each level *N* concurrent async tasks send pre-serialised requests in tight loops for a fixed **measurement window** (default: 20 s), preceded by a **warm-up phase** (default: 5 s) that is excluded from measurements. Pre-serialisation happens once at setup time and the same bytes are reused on every iteration, isolating server-side KMS latency from client-side encoding overhead.
Recorded metrics per *(protocol, operation, concurrency)* triple:

- **Throughput** — requests per second (req/s)
- **p50 / p95 / p99** — round-trip latency percentiles (ms)

### Criterion micro-benchmarks (`ckms bench`)

Criterion (Rust, v0.5) measures the **round-trip latency of a single request** from the ckms client library through the KMS server and back over a loopback TCP connection. The server is started once and kept alive across all benchmarks in the suite.
The reported value is the **mean ± 95 % confidence interval** over a configurable number of samples (preset `quick`: 3 s warm-up + 5 s measurement per benchmark).

> **Infrastructure note:** Both test types use a **local SQLite** backend (temporary, discarded after the run). This isolates pure cryptographic and KMIP serialisation overhead from database I/O. Throughput figures will differ on a production deployment backed by PostgreSQL or Redis-Findex.


---

## Load Tests

### encrypt/aes-gcm

| Concurrency | ttlv-json (req/s) | ttlv-bytes (req/s) | jose (req/s) |
|---|---|---|---|
| 1 | 594 | 3 | 1,216 |
| 2 | 1,118 | 5 | 2,595 |
| 4 | 2,514 | 5 | 10,976 |
| 8 | 3,236 | 6 | 10,567 |
| 16 | 3,675 | 6 | 10,429 |

![Throughput — encrypt/aes-gcm](load/encrypt_aes-gcm.svg)

---

### sign-verify/ecdsa-p256

| Concurrency | ttlv-json (req/s) | ttlv-bytes (req/s) | jose (req/s) |
|---|---|---|---|
| 1 | 3 | 3 | 551 |
| 2 | 5 | 5 | 1,101 |
| 4 | 5 | 5 | 2,298 |
| 8 | 6 | 6 | 2,279 |
| 16 | 6 | 6 | 2,298 |

![Throughput — sign-verify/ecdsa-p256](load/sign-verify_ecdsa-p256.svg)

---

### sign-verify/eddsa-ed25519

| Concurrency | ttlv-json (req/s) | ttlv-bytes (req/s) | jose (req/s) |
|---|---|---|---|
| 1 | 3 | 3 | 1,155 |
| 2 | 5 | 5 | 2,327 |
| 4 | 5 | 5 | 7,851 |
| 8 | 6 | 6 | 7,945 |
| 16 | 6 | 6 | 8,136 |

![Throughput — sign-verify/eddsa-ed25519](load/sign-verify_eddsa-ed25519.svg)

---

### sign-verify/ecdsa-secp256k1

| Concurrency | ttlv-json (req/s) | ttlv-bytes (req/s) |
|---|---|---|
| 1 | 3 | 3 |
| 2 | 5 | 5 |
| 4 | 5 | 5 |
| 8 | 6 | 6 |
| 16 | 6 | 6 |

![Throughput — sign-verify/ecdsa-secp256k1](load/sign-verify_ecdsa-secp256k1.svg)

---

### key-creation/aes-sym

| Concurrency | ttlv-json (req/s) |
|---|---|
| 1 | 409 |
| 2 | 676 |
| 4 | 979 |
| 8 | 1,031 |
| 16 | 1,006 |

![Throughput — key-creation/aes-sym](load/key-creation_aes-sym.svg)

---

### batch/aes-gcm-10

| Concurrency | ttlv-json (req/s) |
|---|---|
| 1 | 3 |
| 2 | 5 |
| 4 | 5 |
| 8 | 6 |
| 16 | 6 |

![Throughput — batch/aes-gcm-10](load/batch_aes-gcm-10.svg)

---

## Criterion Benchmarks

### Symmetric Encryption

| Benchmark | ttlv-json | ttlv-bytes |
|---|---|---|
| aes-gcm-siv/decrypt - 128 | 373.47 ms | 483.06 ms |
| aes-gcm-siv/encrypt - 128 | 389.09 ms | 520.76 ms |
| aes-gcm-siv/encrypt - 256 | 384.72 ms | 517.88 ms |
| aes-gcm/decrypt - 128 | 370.90 ms | 492.78 ms |
| aes-gcm/encrypt - 128 | 387.16 ms | 528.51 ms |
| aes-gcm/encrypt - 192 | 385.53 ms | 518.34 ms |
| aes-xts/decrypt - 128 | 369.57 ms | 449.00 ms |
| aes-xts/encrypt - 128 | 384.34 ms | 510.99 ms |
| aes-xts/encrypt - 256 | 390.11 ms | 528.17 ms |
| chacha20-poly1305/decrypt/256 | 365.03 ms | 479.84 ms |
| chacha20-poly1305/encrypt/256 | 386.10 ms | 528.82 ms |
| salsa-sealed-box/decrypt | 373.25 ms | 444.70 ms |
| salsa-sealed-box/encrypt | 385.78 ms | 512.18 ms |


---

### Asymmetric Encryption

| Benchmark | ttlv-json | ttlv-bytes | jose |
|---|---|---|---|
| covercrypt/decrypt | 401.58 ms | 483.81 ms | — |
| covercrypt/encrypt | 395.61 ms | 523.43 ms | — |
| decrypt - A128GCM | — | — | 1.64 ms |
| decrypt - A192GCM | — | — | 1.54 ms |
| decrypt - A256GCM | — | — | 1.58 ms |
| ecies/decrypt - P-256 | 375.48 ms | 469.69 ms | — |
| ecies/encrypt - P-256 | 382.80 ms | 529.67 ms | — |
| ecies/encrypt - P-384 | 386.00 ms | 529.74 ms | — |
| encrypt - A128GCM | — | — | 1.58 ms |
| encrypt - A192GCM | — | — | 1.59 ms |
| encrypt - A256GCM | — | — | 1.55 ms |
| rsa-aes-kwp/decrypt - 4096 | 723.73 ms | 808.15 ms | — |
| rsa-aes-kwp/encrypt - 4096 | 391.79 ms | 528.39 ms | — |
| rsa-oaep/decrypt - 2048 | — | — | 51.91 ms |
| rsa-oaep/decrypt - 4096 | 726.90 ms | 811.98 ms | 356.49 ms |
| rsa-oaep/encrypt - 2048 | — | — | 1.76 ms |
| rsa-oaep/encrypt - 4096 | 391.90 ms | 520.80 ms | 2.04 ms |
| rsa-pkcs1v15/decrypt - 4096 | 731.48 ms | 810.96 ms | — |
| rsa-pkcs1v15/encrypt - 4096 | 387.87 ms | 526.17 ms | — |


---

### Key Encapsulation (KEM)

| Benchmark | ttlv-json | ttlv-bytes |
|---|---|---|
| configurable/decapsulate - ML-KEM-512 | 366.49 ms | 455.65 ms |
| configurable/encapsulate - ML-KEM-512 | 397.24 ms | 526.64 ms |
| configurable/encapsulate - ML-KEM-768 | 384.94 ms | 516.92 ms |
| pqc/encapsulate - ML-KEM-1024 | 385.28 ms | 524.92 ms |
| pqc/encapsulate - ML-KEM-512 | 380.80 ms | 519.57 ms |
| pqc/encapsulate - ML-KEM-768 | 383.79 ms | 525.24 ms |


---

### Key Creation

| Benchmark | ttlv-json |
|---|---|
| EC - ES256 | — |
| EC - ES384 | — |
| RSA - 2048 | — |
| covercrypt/master-keypair | 54.19 ms |
| ec/ed25519 | 3.39 ms |
| ec/ed448 | 3.64 ms |
| ec/p256 | 3.91 ms |
| ec/p384 | 4.72 ms |
| ec/p521 | 6.27 ms |
| kem/ML-KEM-512 | 3.80 ms |
| kem/ML-KEM-512/P-256 | 3.94 ms |
| kem/ML-KEM-512/X25519 | 8.08 ms |
| kem/ML-KEM-768 | 4.28 ms |
| kem/ML-KEM-768/P-256 | 4.02 ms |
| oct - A128GCM | — |
| oct - A256GCM | — |
| pqc/ML-KEM-1024 | 3.88 ms |
| pqc/ML-KEM-512 | 3.72 ms |
| pqc/ML-KEM-768 | 3.90 ms |
| pqc/X25519MLKEM768 | 3.86 ms |
| pqc/X448MLKEM1024 | 4.56 ms |
| rsa/rsa-4096 | 672.77 ms |
| symmetric/aes-128 | 3.50 ms |
| symmetric/aes-192 | 3.43 ms |
| symmetric/aes-256 | 3.30 ms |
| symmetric/chacha20-256 | 3.41 ms |


---

### Sign / Verify

| Benchmark | ttlv-json | ttlv-bytes | jose |
|---|---|---|---|
| ecdsa-p256/sign | 583.12 ms | 650.82 ms | 2.51 ms |
| ecdsa-p256/verify | 471.88 ms | 581.31 ms | 644.90 ms |
| ecdsa-p384/sign | 540.26 ms | 642.32 ms | 4.05 ms |
| ecdsa-p384/verify | 463.54 ms | 580.58 ms | 655.86 ms |
| ecdsa-p521/sign | 554.14 ms | 644.92 ms | — |
| ecdsa-p521/verify | 462.70 ms | 575.57 ms | — |
| ecdsa-secp256k1/sign | 538.06 ms | 651.48 ms | — |
| ecdsa-secp256k1/verify | 461.60 ms | 577.58 ms | — |
| eddsa-ed25519/sign | 526.98 ms | 656.23 ms | 1.66 ms |
| eddsa-ed25519/verify | 468.63 ms | 588.94 ms | 651.10 ms |
| eddsa-ed448/sign | 539.17 ms | 649.53 ms | — |
| eddsa-ed448/verify | 466.42 ms | 583.15 ms | — |
| ml-dsa/sign - 44 | 529.36 ms | 649.89 ms | — |
| ml-dsa/sign - 65 | 538.40 ms | 654.85 ms | — |
| ml-dsa/verify - 44 | 471.32 ms | 587.70 ms | — |
| rsa-pkcs1v15/sign | — | — | 54.46 ms |
| rsa-pkcs1v15/verify | — | — | 649.82 ms |
| rsa-pss/sign | — | — | 52.77 ms |
| rsa-pss/sign - 4096 | 895.83 ms | 999.18 ms | — |
| rsa-pss/verify | — | — | 649.68 ms |
| rsa-pss/verify - 4096 | 471.01 ms | 561.71 ms | — |
| slh-dsa/sign - SHA2-128f | 533.03 ms | 667.82 ms | — |
| slh-dsa/sign - SHA2-256f | 602.04 ms | 702.82 ms | — |
| slh-dsa/verify - SHA2-128f | 452.51 ms | 566.98 ms | — |


---

