From 3f445eead51b71589cf97a0a01903b631cffcb90 Mon Sep 17 00:00:00 2001 From: Sunil Thakares Date: Sat, 30 May 2026 21:24:29 +0530 Subject: [PATCH] =?UTF-8?q?docs:=20update=20TESTING.md=20for=20v1.0.1=20?= =?UTF-8?q?=E2=80=94=2095=20tests,=20proptest=20section,=20known=20gaps?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/TESTING.md | 291 ++++++++++++++++++++++++++++-------------------- 1 file changed, 171 insertions(+), 120 deletions(-) diff --git a/docs/TESTING.md b/docs/TESTING.md index 746c105..c5d562c 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -2,14 +2,15 @@ ChronoSeal maintains a rigorous, security-first test suite focused on cryptographic correctness, deterministic server ↔ WASM parity, mutation engine integrity, replay resistance, tampering detection, behavioral validation, and storage reliability. -As of **v0.6.1**, the project contains **93 passing tests** across the server, WASM, and shared protocol crates. +As of **v1.0.1**, the project contains **95 passing tests** across the server, WASM, and shared protocol crates. -| Crate | Tests | -| ------------------- | -----: | -| `chronoseal-server` | 33 | -| `chronoseal-wasm` | 24 | -| `shared` | 36 | -| **Total** | **93** | +| Crate | Tests | +| ---------------------------- | ------ | +| `chronoseal-server` | 33 | +| `chronoseal-wasm` | 24 | +| `shared` (unit) | 36 | +| `shared` (property-based) | 2 | +| **Total** | **95** | --- @@ -17,12 +18,12 @@ As of **v0.6.1**, the project contains **93 passing tests** across the server, W ChronoSeal testing prioritizes: -* **Security invariants** over raw coverage metrics -* **Deterministic parity** between server and browser WASM runtimes -* **Negative-path testing** (tampering, replay, malformed input, edge cases) -* **Fuzz-style and randomized testing** for mutation logic -* **Performance regression detection** -* **Long-term protocol stability** +- **Security invariants** over raw coverage metrics +- **Deterministic parity** between server and browser WASM runtimes +- **Negative-path testing** (tampering, replay, malformed input, edge cases) +- **Property-based and fuzz-style testing** for mutation logic and VM robustness +- **Performance regression detection** +- **Long-term protocol stability** Particular emphasis is placed on ensuring that browser-side WASM execution produces identical results to server-side validation. @@ -34,21 +35,21 @@ Particular emphasis is placed on ensuring that browser-side WASM execution produ Configuration tests verify: -* Database backend selection -* TOML configuration parsing -* Command-line override behavior -* Default configuration values -* Runtime initialization logic +- Database backend selection +- TOML configuration parsing +- Command-line override behavior +- Default configuration values +- Runtime initialization logic Supported backends include: -* `sqlite-in-memory` -* `sqlite-in-disk` -* `valkey` +- `sqlite-in-memory` +- `sqlite-in-disk` +- `valkey` (redis-compatible via r2d2 connection pool) Example tests: -```text +``` test_apply_run_args_overrides_db_type test_default_db_type_is_sqlite_in_memory test_toml_parses_db_type_kebab_case @@ -64,17 +65,17 @@ test_valkey_store_operations Session tests validate: -* Session creation -* Public key validation -* Expiration handling -* Replay attack prevention -* Mutation step enforcement -* Commitment verification -* Long-running deterministic parity +- Session creation +- Public key validation +- Expiration handling +- Replay attack prevention +- Mutation step enforcement +- Commitment verification +- Long-running deterministic parity Example tests: -```text +``` test_create_session_rejects_invalid_public_key_length test_expired_session_is_rejected test_replay_attack_is_rejected @@ -93,17 +94,17 @@ The Synthetic Gene Mutation Engine is one of the most security-critical componen Testing focuses on: -* Deterministic server/client parity -* Mutation order execution -* Gene state integrity -* Preview → Commit → Discard lifecycle -* Randomized mutation programs -* Edge-case validation -* Performance regression detection +- Deterministic server/client parity +- Mutation order execution +- Gene state integrity +- Preview → Commit → Discard lifecycle +- Randomized mutation programs +- Edge-case validation +- Performance regression detection Example tests: -```text +``` test_server_client_parity_across_random_orders test_generate_order_is_deterministic_for_seeded_rng test_invalid_positions_wrap_deterministically @@ -118,15 +119,15 @@ test_performance_smoke_mutation_execution Heartbeat validation tests verify: -* Successful state advancement -* Silent rejection behavior -* Commitment validation -* Rate limiting -* Next-state mutation generation +- Successful state advancement +- Silent rejection behavior +- Commitment validation +- Rate limiting +- Next-state mutation generation Example tests: -```text +``` test_handler_success_returns_next_mutation_fields test_handler_tampered_commitment_is_silent_failure test_handler_rate_limit_returns_no_mutation_data @@ -138,16 +139,16 @@ test_handler_rate_limit_returns_no_mutation_data Behavioral validation tests verify: -* Minimum mouse activity -* Minimum movement distance -* Pause detection -* Speed thresholds -* Optional activity requirements -* Fingerprint-related validation paths +- Minimum mouse activity +- Minimum movement distance +- Pause detection +- Speed thresholds +- Optional activity requirements +- Fingerprint-related validation paths Example tests: -```text +``` test_validate_mouse_success test_validate_mouse_insufficient_events test_validate_mouse_insufficient_distance @@ -162,23 +163,28 @@ test_validate_mouse_require_activity_toggle Storage tests verify: -* SQLite in-memory operation -* SQLite disk-backed operation & pool concurrency -* Valkey compatibility mode & pool concurrency operations -* Session CRUD behavior -* Expiration cleanup -* Runtime statistics reporting +- SQLite in-memory operation +- SQLite disk-backed operation and pool concurrency +- Valkey compatibility mode and r2d2 pool concurrency +- Session CRUD behavior including the `opcodes` field +- Expiration cleanup +- Runtime statistics reporting These tests ensure storage implementations remain interchangeable without affecting protocol behavior. Example tests: -```text +``` test_sqlite_pool_concurrency test_valkey_pool_concurrency test_valkey_store_operations ``` +> **Note:** The concurrent write collision path in `update_session` (the `old_last_hash` +> optimistic concurrency guard) is not yet covered by an automated test. Two goroutines +> advancing the same chain simultaneously is a security-relevant race condition. +> A dedicated test is planned for v1.1.0 (see Future Improvements). + --- ## 7. VM Core @@ -187,28 +193,19 @@ The VM core is tested extensively across both WASM and shared crates. Coverage includes: -* ADD -* SUB -* MUL -* XOR -* AND -* OR -* NOT -* HASH -* ROT -* PUSH +- ADD, SUB, MUL, XOR, AND, OR, NOT, HASH, ROT, PUSH Edge cases include: -* Stack underflow -* Truncated instructions -* Unknown opcodes -* Wrapping arithmetic -* Invalid instruction streams +- Stack underflow +- Truncated instructions +- Unknown opcodes +- Wrapping arithmetic +- Invalid instruction streams Example tests: -```text +``` test_add test_add_wrapping test_sub @@ -224,16 +221,38 @@ test_rejects_truncated_instruction --- +## 8. Property-Based Tests + +ChronoSeal uses [`proptest`](https://github.com/proptest-rs/proptest) for property-based testing of core protocol invariants against arbitrary random input. + +Tests live in `shared/tests/proptests.rs` and run as part of `cargo test --workspace`. + +``` +test_vm_execute_never_panics +test_gene_environment_roundtrip_never_panics +``` + +`test_vm_execute_never_panics` feeds arbitrary `Vec` byte sequences into the stack machine +and asserts that execution never panics and that the instruction pointer never exceeds the +program length. This guards against any future VM opcode handler introducing undefined +behaviour on malformed input. + +`test_gene_environment_roundtrip_never_panics` feeds arbitrary bytes into +`gene::decode_environment` and asserts graceful failure rather than a panic, covering the full +space of malformed environment payloads a client could send. + +--- + # Server Test Coverage (`chronoseal-server`) The server crate currently contains **33 tests** covering: -* Configuration -* Runtime initialization -* Session management -* Heartbeat validation -* Rate limiting -* Trust validation +- Configuration +- Runtime initialization +- Session management +- Heartbeat validation +- Rate limiting +- Trust validation The server tests focus heavily on protocol enforcement and security validation. @@ -243,16 +262,16 @@ The server tests focus heavily on protocol enforcement and security validation. The WASM crate currently contains **24 tests** covering: -* VM execution -* Browser-side mutation lifecycle -* Gene initialization -* Mutation preview -* Mutation commit/discard behavior -* Deterministic parity with shared logic +- VM execution +- Browser-side mutation lifecycle +- Gene initialization +- Mutation preview +- Mutation commit/discard behavior +- Deterministic parity with shared logic Example tests: -```text +``` test_preview_commitment_matches_shared_engine test_commit_applies_preview test_discard_preview_keeps_committed_state @@ -265,13 +284,14 @@ These tests ensure browser-generated commitments remain consistent with server e # Shared Crate Coverage (`shared`) -The shared crate currently contains **36 tests** and represents the core protocol implementation used by both server and browser runtimes. +The shared crate currently contains **36 unit tests** and **2 property-based tests**, representing +the core protocol implementation used by both server and browser runtimes. Coverage includes: ### Synthetic Gene Engine -```text +``` test_new_state_with_default_size test_new_state_rejects_invalid_sizes test_commitment_changes_when_gene_or_environment_changes @@ -281,7 +301,7 @@ test_table_driven_randomized_environment_roundtrip ### Mutation Engine -```text +``` test_opcode_insert test_opcode_delete test_opcode_mutate_point @@ -292,7 +312,7 @@ test_mutation_chain ### Validation & Hardening -```text +``` test_rejects_stack_underflow test_rejects_truncated_instruction test_rejects_unknown_opcode @@ -301,7 +321,7 @@ test_zero_length_gene_is_rejected ### Deterministic Parity -```text +``` test_server_client_parity_across_random_orders test_generate_order_is_deterministic_for_seeded_rng test_invalid_positions_wrap_deterministically @@ -309,25 +329,47 @@ test_invalid_positions_wrap_deterministically ### Fuzz & Regression Testing -```text +``` test_fuzz_style_random_program_bytes_do_not_diverge test_performance_smoke_mutation_execution test_vm_instruction_budget_soft_cap ``` +### Property-Based Tests (`shared/tests/proptests.rs`) + +``` +test_vm_execute_never_panics +test_gene_environment_roundtrip_never_panics +``` + +--- + +# Tooling Crates + +## `chronoseal-replay` + +A standalone replay and audit tool for offline verification of recorded ChronoSeal session +chains. It is a developer and forensic utility, not a library, and currently carries no +automated tests. Integration tests against captured session fixtures are planned. + +## `fuzz/` + +Contains libFuzzer targets for deeper coverage of the VM and gene codec. Run separately +via `cargo +nightly fuzz run ` — not part of the standard `cargo test` suite. + --- # Running the Test Suite Run the full workspace: -```bash +``` cargo test --workspace ``` Run individual crates: -```bash +``` cargo test -p shared cargo test -p chronoseal-wasm cargo test -p chronoseal-server @@ -335,7 +377,7 @@ cargo test -p chronoseal-server Display test output: -```bash +``` cargo test -- --nocapture ``` @@ -343,18 +385,22 @@ cargo test -- --nocapture # Critical Security Tests -The following tests protect ChronoSeal's core protocol guarantees and should be treated as **release-blocking** if they fail: +The following tests protect ChronoSeal's core protocol guarantees and should be treated as +**release-blocking** if they fail: -```text +``` test_mutation_commitment_tamper_is_rejected test_replay_attack_is_rejected test_handler_tampered_commitment_is_silent_failure test_server_client_parity_across_random_orders test_deterministic_server_client_parity_across_many_heartbeats test_fuzz_style_random_program_bytes_do_not_diverge +test_vm_execute_never_panics +test_gene_environment_roundtrip_never_panics ``` -These tests directly validate resistance to replay attacks, protocol divergence, mutation tampering, and commitment forgery. +These tests directly validate resistance to replay attacks, protocol divergence, mutation +tampering, commitment forgery, and VM panic on adversarial input. --- @@ -365,40 +411,45 @@ When adding new functionality: 1. Prefer placing protocol logic tests in `shared/` 2. Ensure server ↔ WASM parity is validated 3. Include negative-path test cases -4. Add randomized testing where appropriate +4. Add randomized or property-based testing where appropriate 5. Update this document when introducing major new categories --- # Future Improvements -Planned enhancements include: - -* Property-based testing using `proptest` -* Browser-driven end-to-end integration tests -* Valkey concurrency and failover testing -* Automated benchmark execution in CI -* Expanded mutation-engine fuzzing -* CI-enforced performance regression thresholds +- **Concurrent chain write collision test** — verify that two simultaneous heartbeats for the + same session are handled correctly by the `old_last_hash` optimistic concurrency guard in + `update_session` (security-critical, planned for v1.1.0) +- **Browser-driven end-to-end integration tests** — full Playwright or wasm-bindgen-test + harness exercising the complete init → heartbeat loop in a real browser environment +- **Valkey failover testing** — verify graceful degradation and reconnection under r2d2 pool + exhaustion and server-side connection drops +- **Automated benchmark execution in CI** — enforce performance regression thresholds for + mutation engine and hash chain operations +- **Expanded mutation-engine fuzzing** — additional libFuzzer targets for `vm_extensions` + opcodes introduced in v0.7.0 +- **CI-enforced performance regression thresholds** — gate releases on measured latency bounds --- # Conclusion -ChronoSeal's testing strategy is centered on preserving deterministic behavior, cryptographic correctness, and protocol integrity. +ChronoSeal's testing strategy is centered on preserving deterministic behavior, cryptographic +correctness, and protocol integrity. -The current suite of **93 tests** provides broad coverage across: +The current suite of **95 tests** provides broad coverage across: -* Session security -* Heartbeat validation -* Mutation engine correctness -* Deterministic server/WASM parity -* Trust validation -* Storage abstraction -* Replay resistance -* Protocol hardening +- Session security +- Heartbeat validation +- Mutation engine correctness +- Deterministic server/WASM parity +- Trust and behavioral validation +- Storage abstraction +- Replay resistance +- Protocol hardening +- Property-based VM and gene codec robustness Maintaining and expanding this test suite remains a core project priority as ChronoSeal evolves. -**Last Updated:** May 2026 (v0.6.1) - +**Last Updated:** May 2026 (v1.0.1)