Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
34227efd2b | ||
|
|
5599e1b5c8 | ||
|
|
32a325234a | ||
|
|
d25846c58c | ||
|
|
2f06c8faa9 | ||
|
|
d710deb8b0 | ||
|
|
a6208ef330 |
No files matched your search
@@ -1,519 +0,0 @@
|
||||
# NX9 WireGuard (`nx9-wg`) — Full Technical Architecture & Stack Report
|
||||
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
||||
> **"Software people can own, understand, and control."**
|
||||
> — [NX9 Systems (https://nx9.in)](https://nx9.in)
|
||||
|
||||
---
|
||||
|
||||
## 📑 Table of Contents
|
||||
|
||||
1. [Executive Summary](#1-executive-summary)
|
||||
2. [The NX9 Philosophy in Implementation](#2-the-nx9-philosophy-in-implementation)
|
||||
3. [The Architectural Paradigm Shift](#3-the-architectural-paradigm-shift)
|
||||
4. [Six-Crate Workspace Architecture](#4-six-crate-workspace-architecture)
|
||||
5. [Complete Capability Inventory](#5-complete-capability-inventory)
|
||||
6. [Native Linux Execution Planes](#6-native-linux-execution-planes)
|
||||
7. [Closed-Loop Reconciliation & Convergence](#7-closed-loop-reconciliation--convergence)
|
||||
8. [Embedded Single Page Application (SPA) & WebSockets](#8-embedded-single-page-application-spa--websockets)
|
||||
9. [REST API & WebSocket Protocol Reference](#9-rest-api--websocket-protocol-reference)
|
||||
10. [Native CLI Command System](#10-native-cli-command-system)
|
||||
11. [Security Model & Capability Isolation](#11-security-model--capability-isolation)
|
||||
12. [Disaster Recovery, Backups & Upgrades](#12-disaster-recovery-backups--upgrades)
|
||||
13. [Release Engineering & Deployment Lifecycle](#13-release-engineering--deployment-lifecycle)
|
||||
14. [Quality Assurance & Verification Evidence](#14-quality-assurance--verification-evidence)
|
||||
15. [Ecosystem & License Summary](#15-ecosystem--license-summary)
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive Summary
|
||||
|
||||
`nx9-wg` is a sovereign, self-hosted, Linux-native VPN and network control plane built directly around the Linux kernel's in-tree WireGuard implementation (`wireguard.ko`).
|
||||
|
||||
Rather than functioning as a fragile user interface wrapper that shells out to external command-line utilities (`wg`, `ip`, `nft`, `sysctl`), `nx9-wg` establishes an integrated, single-binary architecture. It combines **authoritative SQLite desired-state persistence**, **direct kernel Netlink execution** (RTNETLINK and WireGuard Generic Netlink), **in-process Netfilter firewall/NAT compilation** (`libnftables.so.1`), **continuous closed-loop reconciliation**, a **multi-format native CLI**, and an **embedded zero-dependency Single Page Application (SPA) Web UI**.
|
||||
|
||||
---
|
||||
|
||||
## 2. The NX9 Philosophy in Implementation
|
||||
|
||||
Every design and architectural choice in `nx9-wg` directly reflects the core philosophy of the **NX9 Ecosystem** ([https://nx9.in](https://nx9.in)):
|
||||
|
||||
| NX9 Principle | Core Intent | `nx9-wg` Implementation |
|
||||
| :--- | :--- | :--- |
|
||||
| 🔑 **Operator Ownership** | Full control over binaries, configurations, databases, keys, and backups with zero dependency on cloud-hosted control planes. | 100% local operation. Private keys, configuration data, and encryption parameters never leave the operator's host. |
|
||||
| 🖥️ **Self-Hosting First** | Built to be deployed and operated effortlessly by a single administrator without requiring Kubernetes or external infrastructure. | Self-contained single executable. Bootstrapped with a single command (`nx9-wg init`), running seamlessly under standard systemd. |
|
||||
| ⚡ **Simplicity Over Complexity** | One binary, one configuration file, one SQLite database, one administrator. | No multi-daemon orchestration, no external message queues, no Node.js runtime, no Python scripts, and zero shell wrappers. |
|
||||
| 🛡️ **Privacy by Default** | Zero analytics, zero telemetry collection, zero third-party tracking, and zero assumptions of external cloud connectivity. | No phone-home telemetry. Completely air-gapped capable with all static assets, fonts, and scripts embedded in the binary. |
|
||||
| 🔒 **Security by Design** | Modern cryptography, strict invariants, and append-only audit logging embedded from day one. | Argon2id password hashing, SHA-256 token digests, X25519 key generation, single-admin database constraint (`CHECK (id=1)`), and strict secret redaction. |
|
||||
| 🔧 **Operational Excellence** | Diagnostics, backup/restore, migration tools, CLI management, and comprehensive documentation built-in. | Multi-subsystem automated health inspections, atomic online SQLite `VACUUM INTO` snapshots with SHA-256 manifests, and 100% CLI parity. |
|
||||
| ⏳ **Long-Term Stability** | Avoid dependency churn. Build software that remains understandable and maintainable years into the future. | Built in stable native Rust (Edition 2024), standard SQLite 3 storage, standard TOML configuration, and POSIX-compliant systemd integration. |
|
||||
| 🌐 **Open Source First** | 100% free and open-source software under permissive licensing. | Dual-licensed under `MIT OR Apache-2.0`. Complete freedom to inspect, audit, build, and extend. |
|
||||
| ♾️ **Zero Vendor Lock-In** | Standard data formats, open protocols, and zero proprietary cloud lock-in. | Standard SQLite database, standard WireGuard `.conf` files, standard RFC-compliant JSON/YAML/CSV output formats, and open Netlink sockets. |
|
||||
|
||||
---
|
||||
|
||||
## 3. The Architectural Paradigm Shift
|
||||
|
||||
### Typical WireGuard Management Wrappers
|
||||
```
|
||||
┌──────────┐ ┌──────────────────────┐ ┌────────────────────────────┐ ┌──────────────┐
|
||||
│ Web UI │ ──► │ Text Config Files │ ──► │ wg / ip / nft / sysctl │ ──► │ Linux Kernel │
|
||||
│ (Node/Py)│ │(/etc/wireguard/*.conf│ │ (Subprocess Spawning) │ │ (wireguard.ko│
|
||||
└──────────┘ └──────────────────────┘ └────────────────────────────┘ └──────────────┘
|
||||
```
|
||||
*Disadvantages: Fragile process spawning, race conditions, lack of atomic state, broken host routing, vulnerability to configuration drift, and heavy runtime dependency footprints.*
|
||||
|
||||
### Whereas `nx9-wg` is a Native Control & Execution Plane:
|
||||
```
|
||||
┌───────────────────────────────────┐
|
||||
│ nx9-wg │
|
||||
└─────────────────┬─────────────────┘
|
||||
│
|
||||
┌────────────────────────────────┴────────────────────────────────┐
|
||||
│ │
|
||||
┌─────────▼─────────┐ ┌─────────▼─────────┐
|
||||
│ Desired State │ │ Live State │
|
||||
│ (Authoritative) │ │ (Kernel Cache) │
|
||||
└─────────┬─────────┘ └─────────▲─────────┘
|
||||
│ │
|
||||
┌─────────▼─────────┐ ┌─────────┴─────────┐
|
||||
│ SQLite 3 (WAL) │ │ Linux Kernel │
|
||||
└─────────┬─────────┘ └─────────▲─────────┘
|
||||
│ │
|
||||
│ Live Netlink Telemetry
|
||||
▼ │
|
||||
┌───────────────────┐ │
|
||||
│ Reconciliation │ ◄─────────────────────────────────────────────────────┘
|
||||
│ Engine │
|
||||
└─────────┬─────────┘
|
||||
│
|
||||
├── 🔐 WireGuard Generic Netlink (family "wireguard")
|
||||
├── 🌐 RTNETLINK (Links, IPv4/IPv6 Addresses, Routes)
|
||||
├── 🔥 Netfilter / libnftables FFI (table inet nx9_wg)
|
||||
└── ↔️ Direct Procfs IP Forwarding (/proc/sys/net)
|
||||
│
|
||||
▼
|
||||
┌───────────────────┐
|
||||
│ Linux Networking │
|
||||
└───────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Six-Crate Workspace Architecture
|
||||
|
||||
`nx9-wg` is architected as a modular six-crate Cargo workspace ensuring clean separation of concerns, zero circular dependencies, and isolated testability:
|
||||
|
||||
```
|
||||
nx9-wg (Root Executable & Unified Entrypoint)
|
||||
├── 📦 crates/nx9-wg-core — Domain models, RFC validation, cryptography, configuration
|
||||
├── 📦 crates/nx9-wg-db — SQLite storage engine, migration framework, repositories
|
||||
├── 📦 crates/nx9-wireguard — WireGuard Generic Netlink, RTNETLINK link engine, config & QR
|
||||
├── 📦 crates/nx9-wg-network — RTNETLINK routes/addresses, libnftables Netfilter, procfs
|
||||
├── 📦 crates/nx9-wg-api — Axum REST router, WebSockets, auth, reconciliation, IP allocator
|
||||
└── 📦 crates/nx9-wg-ui — Design tokens, CSS compiler, view models, SPA integration
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Complete Capability Inventory
|
||||
|
||||
`nx9-wg` delivers a comprehensive suite of 20 core infrastructure capabilities:
|
||||
|
||||
| Icon | Capability | Architectural Description |
|
||||
| :---: | :--- | :--- |
|
||||
| 🔐 | **WireGuard Interface Lifecycle** | In-process RTNETLINK link creation (`RTM_NEWLINK`), state toggling (`IFF_UP`/`IFF_DOWN`), and WireGuard Generic Netlink cryptokey configuration (`WG_CMD_SET_DEVICE`). |
|
||||
| 👥 | **Cryptographic Peer Enrollment** | Dynamic Curve25519 public key management, preshared keys, CIDR allowed IPs, persistent keepalive intervals, and atomic `ReplacePeers` synchronization. |
|
||||
| 🌐 | **IPv4 & IPv6 Address Management** | Native Netlink address assignment (`RTM_NEWADDR` / `RTM_DELADDR`) across WireGuard interfaces without invoking `ip addr`. |
|
||||
| 🛣️ | **Kernel Route Management** | Routing table synchronization (`RTM_NEWROUTE` / `RTM_DELROUTE`) with strict default gateway protection and multi-tenant non-interference invariants. |
|
||||
| 🔥 | **nftables Netfilter Firewall** | In-process rule compilation and transactional application via `libnftables.so.1` confined strictly to `table inet nx9_wg`. |
|
||||
| 🛡️ | **Scoped NAT & Masquerading** | Outbound NAT masquerade dynamically calculated and applied strictly to managed WireGuard client subnets, preventing host network disruption. |
|
||||
| ↔️ | **Direct Procfs IP Forwarding** | Direct atomic mutation of `/proc/sys/net/ipv4/ip_forward` and `/proc/sys/net/ipv6/conf/all/forwarding` without invoking `sysctl`. |
|
||||
| 📡 | **Live Kernel Telemetry** | Real-time extraction of handshake timestamps, rx/tx byte counters, and roaming remote socket endpoints directly from kernel sockets. |
|
||||
| 🔄 | **Desired-State Reconciliation** | Continuous closed-loop control cycle bringing the Linux kernel execution plane into alignment with authoritative SQLite storage. |
|
||||
| 🧭 | **5-Subsystem Drift Detection** | Deterministic, read-only calculation of state divergence across interfaces, peers, routes, firewall rules, and IP forwarding. |
|
||||
| ♻️ | **Cold-Boot Restart Recovery** | Autonomous reconstruction of kernel network topology, routes, and firewall rules upon daemon startup or host reboot. |
|
||||
| 🧪 | **Cross-Platform Simulation Engine** | In-memory simulated execution planes enabling full UI and CLI development on macOS and Windows without requiring Linux Netlink. |
|
||||
| 🖥️ | **Multi-Format Native CLI** | 100% native CLI coverage across all 17 subcommands supporting `table`, `json`, `yaml`, and `csv` output with script-friendly exit codes. |
|
||||
| 🌐 | **Axum REST API & WebSockets** | High-performance asynchronous HTTP server with session/bearer authentication and a real-time WebSocket event broadcaster (`/api/v1/ws`). |
|
||||
| 💻 | **Embedded Zero-Dependency SPA** | Modern HTML5/CSS3/Vanilla ES6+ Single Page Application compiled into the binary with Light/Dark theme support and 15 interactive views. |
|
||||
| 📊 | **Automated Health Diagnostics** | Built-in inspection across 9 subsystems with structured status reporting, root-cause diagnostics, and actionable remediation hints. |
|
||||
| 💾 | **Atomic Disaster Recovery Backups** | Online non-blocking SQLite `VACUUM INTO` snapshots with SHA-256 manifest verification and automatic pre-restore safety checkpoints. |
|
||||
| 🔑 | **Single-Administrator Security** | Database-enforced single identity (`CHECK (id=1)`), Argon2id password hashing, SHA-256 API token storage, and brute-force login rate limiting. |
|
||||
| 📱 | **Client Profiles & QR Engine** | Provider/device MTU profiles (1280 vs 1360 vs 1420), collision-resistant IP allocation, and pure Rust vector SVG, PNG, and ASCII QR generation. |
|
||||
| 📦 | **Production Release Packaging** | Standalone distribution archive generator (`scripts/package-release.sh`), automated installer/uninstaller, and hardened systemd service unit. |
|
||||
|
||||
---
|
||||
|
||||
## 6. Native Linux Execution Planes
|
||||
|
||||
`nx9-wg` enforces a **Zero Subprocess Guarantee** across the entire production codebase:
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Rust Production Binary │
|
||||
├────────────────────────────┬────────────────────────────┬──────────────────────────────┤
|
||||
│ NativeLinuxWireGuardEngine │ NativeLinuxNetworkEngine │ NativeLinuxNftablesEngine │
|
||||
├────────────────────────────┼────────────────────────────┼──────────────────────────────┤
|
||||
│ • AF_NETLINK │ • NETLINK_ROUTE │ • In-process libnftables FFI │
|
||||
│ • NETLINK_GENERIC (wg) │ • RTM_NEWLINK / DELLINK │ • nft_ctx_new() │
|
||||
│ • WG_CMD_SET_DEVICE │ • RTM_NEWADDR / DELADDR │ • Atomic Netfilter batch │
|
||||
│ • WG_CMD_GET_DEVICE │ • RTM_NEWROUTE / DELROUTE │ • Scoped: table inet nx9_wg │
|
||||
│ • WGDEVICE_F_REPLACE_PEERS │ • Direct /proc/sys writes │ • Zero host table flushes │
|
||||
└─────────────┬──────────────┴─────────────┬──────────────┴──────────────┬───────────────┘
|
||||
▼ ▼ ▼
|
||||
┌────────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Linux Kernel │
|
||||
│ (wireguard.ko • RTNETLINK • Netfilter • /proc/sys/net) │
|
||||
└────────────────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Closed-Loop Reconciliation & Convergence
|
||||
|
||||
Reconciliation is the foundational control loop that bridges authoritative SQLite state with the Linux kernel:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ 1. SQLite Desired State (Authoritative Persistent Source of Truth│
|
||||
└────────────────────────────────┬─────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ 2. Live Kernel Query (WireGuard Genl, RTNL routes, table nx9_wg) │
|
||||
└────────────────────────────────┬─────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ 3. Drift Detection (Read-Only Deterministic Multi-Subsystem Diff)│
|
||||
└────────────────────────────────┬─────────────────────────────────┘
|
||||
│
|
||||
┌───────────────┴───────────────┐
|
||||
│ has_drift == false? │
|
||||
├───────────────────────┬───────┤
|
||||
│ YES │ NO │
|
||||
▼ ▼ │
|
||||
┌─────────────┐ ┌──────────────▼────────────────┐
|
||||
│ Converged │ │ 4. Acquire Async Mutex Lock │
|
||||
│ (In Sync) │ └──────────────┬────────────────┘
|
||||
└─────────────┘ │
|
||||
▼
|
||||
┌───────────────────────────────┐
|
||||
│ 5. Execute Native Mutations │
|
||||
│ (Genl SET_DEVICE, RTNL) │
|
||||
└──────────────┬────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────────┐
|
||||
│ 6. Post-Apply Verification │
|
||||
└──────────────┬────────────────┘
|
||||
│
|
||||
┌───────────────┴───────────────┐
|
||||
▼ ▼
|
||||
┌─────────────┐ ┌─────────────┐
|
||||
│ Converged │ │ Partial │
|
||||
│ (100% Sync)│ │ Failure │
|
||||
└─────────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
### The Six Reconciliation Lifecycle States
|
||||
1. **`Plan`**: Read-only calculation of drift between SQLite and kernel.
|
||||
2. **`Applying`**: In-progress dispatch of native mutations across execution planes.
|
||||
3. **`Verifying`**: Querying live kernel state to confirm applied changes took effect.
|
||||
4. **`Converged`**: 100% synchronization achieved with zero remaining drift.
|
||||
5. **`PartialFailure`**: One or more execution planes failed during apply (e.g. permission error).
|
||||
6. **`DriftRemains`**: Apply completed without fatal error, but post-verification detected unapplied state.
|
||||
|
||||
---
|
||||
|
||||
## 8. Embedded Single Page Application (SPA) & WebSockets
|
||||
|
||||
The `nx9-wg` frontend is a zero-dependency HTML5/CSS/JavaScript SPA embedded directly into the Rust binary:
|
||||
|
||||
- **Zero External Toolchains**: No Node.js, npm, Webpack, Vite, React, or external CDN dependencies.
|
||||
- **Embedded In-Memory Delivery**: Bundled at compile-time via `include_str!()` and served from memory.
|
||||
- **Design Tokens**: Custom CSS token system ([`crates/nx9-wg-ui/src/css.rs`](file:///home/sunil/Programs/nx9-wg/crates/nx9-wg-ui/src/css.rs)) supporting Light and Dark modes.
|
||||
- **Real-Time WebSocket Stream**: Subscribes to `ws://<host>/api/v1/ws` for live handshakes and drift alerts without polling.
|
||||
- **15 Interactive Views**:
|
||||
1. `#dashboard` — System overview, uptime, interface/peer counts, health cards.
|
||||
2. `#interfaces` — WireGuard interface CRUD, listen port, MTU, state toggles.
|
||||
3. `#peers` — Enrolled peer table, real-time handshakes, profile resolution, `.conf` export, SVG QR modal.
|
||||
4. `#networks` — Subnet network ranges, CIDR masks, available IP inspector.
|
||||
5. `#routes` — Kernel route definitions, gateway assignments, interface scoping.
|
||||
6. `#firewall` — nftables packet filtering rules in `table inet nx9_wg`, priority sorting.
|
||||
7. `#nat` — Managed subnet NAT masquerade status and instant toggle.
|
||||
8. `#forwarding` — Kernel IPv4/IPv6 packet forwarding status and toggle.
|
||||
9. `#reconciliation` — Real-time kernel drift overview, action plan table, interactive Apply button.
|
||||
10. `#diagnostics` — Automated multi-subsystem health checks with remediation hints.
|
||||
11. `#live-state` — Raw Linux Netlink telemetry, active kernel interfaces, live routing table.
|
||||
12. `#settings` — Key-value appliance parameters and danger zone reset controls.
|
||||
13. `#backups` — Online SQLite backup snapshots list, instant backup creation, `.db` download.
|
||||
14. `#audit` — Append-only security and administrative audit trail.
|
||||
15. `#administrator` — Admin account verification, password rotation, and one-time API token generation.
|
||||
|
||||
---
|
||||
|
||||
## 9. REST API & WebSocket Protocol Reference
|
||||
|
||||
### Authentication Mechanisms
|
||||
- **Session Cookie**: `nx9_session=<UUID>` returned via `POST /api/v1/auth/login`.
|
||||
- **Bearer Token**: `Authorization: Bearer nx9_<UUID>_<SECRET>` passed in HTTP headers.
|
||||
|
||||
### Complete REST Route Inventory
|
||||
```http
|
||||
POST /api/v1/auth/login - Authenticate administrator & create session
|
||||
POST /api/v1/auth/logout - Invalidate active session
|
||||
GET /api/v1/auth/session - Query authenticated session info
|
||||
POST /api/v1/auth/password - Rotate admin password (invalidates all sessions)
|
||||
GET /api/v1/auth/tokens - List active API token metadata
|
||||
POST /api/v1/auth/tokens - Generate new API token (one-time raw secret return)
|
||||
DELETE /api/v1/auth/tokens/{id} - Revoke an API token
|
||||
|
||||
GET /api/v1/system - System operational overview and object counts
|
||||
GET /api/v1/system/health - Public health check endpoint
|
||||
GET /api/v1/system/version - Version, build edition, and architecture
|
||||
GET /api/v1/system/settings - List all appliance key-value settings
|
||||
PUT /api/v1/system/settings - Upsert appliance setting
|
||||
|
||||
GET /api/v1/interfaces - List all WireGuard interfaces
|
||||
POST /api/v1/interfaces - Create WireGuard interface
|
||||
GET /api/v1/interfaces/{id} - Get interface details
|
||||
PUT /api/v1/interfaces/{id} - Update interface configuration
|
||||
DELETE /api/v1/interfaces/{id} - Delete interface (cascades to peers)
|
||||
POST /api/v1/interfaces/{id}/enable - Set interface IFF_UP
|
||||
POST /api/v1/interfaces/{id}/disable - Set interface IFF_DOWN
|
||||
GET /api/v1/interfaces/{id}/status - Query live kernel netlink telemetry
|
||||
GET /api/v1/interfaces/{id}/peers - List peers attached to interface
|
||||
POST /api/v1/interfaces/{id}/peers - Enroll new peer on interface
|
||||
|
||||
GET /api/v1/peers/{id} - Get peer details
|
||||
PUT /api/v1/peers/{id} - Update peer parameters
|
||||
DELETE /api/v1/peers/{id} - Delete peer
|
||||
POST /api/v1/peers/{id}/enable - Enable peer
|
||||
POST /api/v1/peers/{id}/disable - Disable peer
|
||||
GET /api/v1/peers/{id}/config - Download client .conf file
|
||||
GET /api/v1/peers/{id}/qr - Render QR code (SVG / PNG / ASCII)
|
||||
|
||||
GET /api/v1/networks - List subnet networks
|
||||
POST /api/v1/networks - Create subnet network
|
||||
GET /api/v1/networks/{id} - Get network details
|
||||
DELETE /api/v1/networks/{id} - Delete network
|
||||
GET /api/v1/networks/{id}/available - List available unallocated IP addresses
|
||||
|
||||
GET /api/v1/routes - List kernel routing entries
|
||||
POST /api/v1/routes - Create routing entry
|
||||
DELETE /api/v1/routes/{id} - Delete routing entry
|
||||
|
||||
GET /api/v1/firewall/rules - List nftables firewall rules
|
||||
POST /api/v1/firewall/rules - Create firewall rule
|
||||
DELETE /api/v1/firewall/rules/{id} - Delete firewall rule
|
||||
POST /api/v1/firewall/rules/{id}/enable - Enable firewall rule
|
||||
POST /api/v1/firewall/rules/{id}/disable - Disable firewall rule
|
||||
|
||||
GET /api/v1/reconcile/plan - Read-only drift calculation plan
|
||||
POST /api/v1/reconcile/apply - Serialized kernel apply and convergence check
|
||||
|
||||
GET /api/v1/diagnostics/all - Run full diagnostics across all 9 subsystems
|
||||
GET /api/v1/diagnostics/{subsystem} - Run diagnostics for single subsystem
|
||||
|
||||
GET /api/v1/backups - List backup records
|
||||
POST /api/v1/backups/create - Trigger atomic online VACUUM INTO snapshot
|
||||
GET /api/v1/backups/{id}/download - Download raw SQLite database snapshot
|
||||
POST /api/v1/backups/{id}/restore - Restore database with automatic safety backup
|
||||
DELETE /api/v1/backups/{id} - Delete backup snapshot file and metadata
|
||||
|
||||
GET /api/v1/audit - Query append-only audit trail
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Native CLI Command System
|
||||
|
||||
`nx9-wg` provides 100% native CLI coverage across all 17 subcommands:
|
||||
|
||||
```bash
|
||||
# Global output formats: --format table | json | yaml | csv
|
||||
nx9-wg version
|
||||
nx9-wg serve --bind 127.0.0.1:8080
|
||||
nx9-wg init --generate-password --write-password-file /var/lib/nx9-wg/admin-password
|
||||
|
||||
# Administration & Authentication
|
||||
nx9-wg admin info
|
||||
nx9-wg admin password
|
||||
nx9-wg admin token create "ci-pipeline" --expires-in-days 90 --write-token-file /tmp/token
|
||||
nx9-wg admin token list
|
||||
nx9-wg admin token revoke <TOKEN_UUID>
|
||||
|
||||
# Interface & Peer Management
|
||||
nx9-wg interface list
|
||||
nx9-wg interface create wg0 --address-v4 10.100.0.1/24 --port 51820 --mtu 1420
|
||||
nx9-wg peer create --interface wg0 --name alice --profile full_tunnel --mtu 1280
|
||||
nx9-wg peer qr <PEER_UUID>
|
||||
nx9-wg peer config <PEER_UUID>
|
||||
|
||||
# Networking, Firewall & NAT
|
||||
nx9-wg route add --destination 192.168.50.0/24 --gateway 10.100.0.2 --interface-name wg0
|
||||
nx9-wg firewall add --name "allow-dns" --protocol udp --port 53 --action accept --priority 10
|
||||
nx9-wg nat enable
|
||||
nx9-wg forwarding enable
|
||||
|
||||
# State Reconciliation & Diagnostics
|
||||
nx9-wg reconcile plan
|
||||
nx9-wg reconcile apply
|
||||
nx9-wg diagnostics all
|
||||
|
||||
# Disaster Recovery
|
||||
nx9-wg backup create --description "Pre-upgrade snapshot"
|
||||
nx9-wg backup verify /var/lib/nx9-wg/backups/nx9-backup-...db
|
||||
nx9-wg backup restore <BACKUP_UUID>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Security Model & Capability Isolation
|
||||
|
||||
### A. Single Administrator Identity
|
||||
- Database-level integrity constraint: `CHECK (id = 1)` in `admins` table.
|
||||
- Eliminates multi-tenant privilege escalation and role-confusion attack surfaces.
|
||||
|
||||
### B. Credential Protection
|
||||
- **Passwords**: Hashed with Argon2id using unique cryptographic salts.
|
||||
- **API Tokens**: Stored exclusively as SHA-256 digests in SQLite; raw tokens are displayed once upon creation.
|
||||
- **Secret Redaction**: Private keys, preshared keys, and password hashes implement custom `std::fmt::Debug` implementations returning `[REDACTED]`.
|
||||
|
||||
### C. Hardened systemd Sandbox (`nx9-wg.service`)
|
||||
```ini
|
||||
[Unit]
|
||||
Description=NX9 WireGuard Native VPN Platform
|
||||
After=network.target network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/local/bin/nx9-wg serve
|
||||
Restart=always
|
||||
RestartSec=5s
|
||||
|
||||
# Minimal Linux Capabilities
|
||||
CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
|
||||
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
|
||||
|
||||
# Sandboxing Directives
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
ProtectControlGroups=true
|
||||
RestrictSUIDSGID=true
|
||||
LockPersonality=true
|
||||
NoNewPrivileges=true
|
||||
|
||||
# Allowed Network Address Families
|
||||
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK
|
||||
|
||||
# Explicit Read-Write Paths (for state and direct procfs forwarding)
|
||||
ReadWritePaths=/var/lib/nx9-wg /etc/nx9-wg /var/log/nx9-wg /proc/sys/net
|
||||
ProtectKernelTunables=false
|
||||
|
||||
# Systemd Directory Management
|
||||
StateDirectory=nx9-wg
|
||||
ConfigurationDirectory=nx9-wg
|
||||
LogsDirectory=nx9-wg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. Disaster Recovery, Backups & Upgrades
|
||||
|
||||
### Atomic Online Backups
|
||||
- Uses SQLite `VACUUM INTO` to produce non-blocking, consistent binary database snapshots while the daemon is actively serving traffic.
|
||||
- Generates SHA-256 manifest records for each snapshot.
|
||||
|
||||
### Safe Rollback Workflow
|
||||
```
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ 1. Operator Initiates Restore (nx9-wg backup restore) │
|
||||
└───────────────────────────┬────────────────────────────┘
|
||||
│
|
||||
┌───────────────────────────▼────────────────────────────┐
|
||||
│ 2. Verify Backup File Size, Magic Header & SHA-256 │
|
||||
└───────────────────────────┬────────────────────────────┘
|
||||
│
|
||||
┌───────────────────────────▼────────────────────────────┐
|
||||
│ 3. Create Pre-Restore Safety Snapshot (Automatic Fallback│
|
||||
└───────────────────────────┬────────────────────────────┘
|
||||
│
|
||||
┌───────────────────────────▼────────────────────────────┐
|
||||
│ 4. Close Connection Pools, Replace .db, Clean WAL/SHM │
|
||||
└───────────────────────────┬────────────────────────────┘
|
||||
│
|
||||
┌───────────────────────────▼────────────────────────────┐
|
||||
│ 5. Reopen Database & Execute Reconciliation Engine │
|
||||
│ (Kernel state converged to restored desired state) │
|
||||
└────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. Release Engineering & Deployment Lifecycle
|
||||
|
||||
### Automated Packaging Pipeline ([`scripts/package-release.sh`](file:///home/sunil/Programs/nx9-wg/scripts/package-release.sh))
|
||||
Generates self-contained, reproducible distribution archives in `target/dist/`:
|
||||
- `nx9-wg-v1.0.0-linux-x86_64.tar.gz` (7.8 MB)
|
||||
- `nx9-wg-v1.0.0-linux-x86_64.tar.xz` (5.0 MB)
|
||||
- `nx9-wg-v1.0.0-linux-x86_64.sha256` (Cryptographic checksum manifest)
|
||||
|
||||
### Production Filesystem Layout & Permissions
|
||||
```
|
||||
/usr/local/bin/nx9-wg 0755 root:root - Native Executable Binary
|
||||
/etc/nx9-wg/ 0750 root:root - Configuration Directory
|
||||
└── config.toml 0640 root:root - Production Configuration
|
||||
/var/lib/nx9-wg/ 0700 root:root - State & SQLite Directory
|
||||
├── nx9-wg.db 0600 root:root - Authoritative SQLite Database
|
||||
├── nx9-wg.db-wal 0600 root:root - WAL Journal
|
||||
├── admin-password 0600 root:root - Initial Bootstrap Password
|
||||
└── backups/ 0700 root:root - Backup Snapshots Directory
|
||||
/var/log/nx9-wg/ 0750 root:root - Operational Logs
|
||||
/etc/systemd/system/nx9-wg.service 0644 root:root - Hardened Service Unit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 14. Quality Assurance & Verification Evidence
|
||||
|
||||
All quality gates have been executed and verified clean:
|
||||
|
||||
| Quality Gate | Verification Command | Result |
|
||||
| :--- | :--- | :---: |
|
||||
| **Code Formatting** | `cargo fmt --all -- --check` | **PASS** (Zero diffs) |
|
||||
| **Workspace Compilation** | `cargo check --workspace` | **PASS** (Zero errors) |
|
||||
| **Workspace Unit Tests** | `cargo test --workspace` | **PASS** (**162 / 162 passed**, 100%) |
|
||||
| **Clippy Linter Check** | `cargo clippy --workspace --all-targets --all-features -- -D warnings` | **PASS** (Zero warnings) |
|
||||
| **Comprehensive CLI Suite** | `LIVE=0 bash scripts/test-cli-comprehensive.sh` | **PASS** (**203 passed** / 7 skipped) |
|
||||
| **Native Integration Suite** | `LIVE=0 bash scripts/test-native-integration.sh` | **PASS** (**19 passed** / 1 skipped) |
|
||||
| **Dedicated Live Kernel Suite** | `LIVE=0 bash scripts/test-live-kernel.sh` | **PASS** (**23 passed** / 1 skipped) |
|
||||
| **Subprocess Safety Audit** | Automated source scan for `Command::new` | **PASS** (Zero subprocesses) |
|
||||
| **Secret Leakage Audit** | Automated audit for plaintext credentials | **PASS** (Zero secrets leaked) |
|
||||
| **Standalone Package Verification**| Fresh directory extraction & independent run | **PASS** (Standalone execution) |
|
||||
| **Git Diff Whitespace Audit** | `git diff --check` | **PASS** (Zero whitespace issues) |
|
||||
|
||||
---
|
||||
|
||||
## 15. Ecosystem & License Summary
|
||||
|
||||
`nx9-wg` is part of the **NX9 Ecosystem** ([https://nx9.in](https://nx9.in)) created by **Sunil Thakare**.
|
||||
|
||||
Dual-licensed under either:
|
||||
- **MIT License** ([`LICENSE-MIT`](file:///home/sunil/Programs/nx9-wg/LICENSE-MIT))
|
||||
- **Apache License, Version 2.0** ([`LICENSE-APACHE`](file:///home/sunil/Programs/nx9-wg/LICENSE-APACHE))
|
||||
|
||||
at your option.
|
||||
|
||||
---
|
||||
|
||||
> **NX9 WireGuard** — Sovereign, Self-Hosted, Linux-Native Network Infrastructure.
|
||||
@@ -326,7 +326,7 @@ All release quality gates have been executed and verified on Debian Linux:
|
||||
| **Formatting** | `cargo fmt --all -- --check` | **PASS** (0 errors) |
|
||||
| **Compilation** | `cargo check --workspace --all-targets` | **PASS** (0 errors) |
|
||||
| **Clippy Linting** | `cargo clippy --workspace --all-targets --all-features -- -D warnings` | **PASS** (0 warnings) |
|
||||
| **Workspace Test Suite** | `cargo test --workspace --all-targets` | **PASS** (All 88 tests passing) |
|
||||
| **Workspace Test Suite** | `cargo test --workspace --all-targets` | **PASS** (All 162 tests passing) |
|
||||
| **CLI Test Suite** | `cargo test --test test_cli_commands` | **PASS** (11 tests passing) |
|
||||
| **Release Compilation** | `cargo build --release --workspace` | **PASS** (Optimized release binary) |
|
||||
| **Production Server Acceptance** | Physical Android WireGuard client connection | **VERIFIED** (Live handshake and RX/TX telemetry confirmed) |
|
||||
@@ -374,21 +374,88 @@ at your option.
|
||||
## 16. Documentation Master Index
|
||||
|
||||
For detailed subsystem documentation, see the [**Documentation Index**](docs/README.md):
|
||||
- [NX9 Design Principles](docs/design-principles.md)
|
||||
- [System Architecture](docs/architecture.md)
|
||||
- [Installation Guide](docs/installation.md)
|
||||
- [Native WireGuard Engine](docs/native-wireguard.md)
|
||||
- [Native Network Engine](docs/native-network.md)
|
||||
- [Native nftables Engine](docs/nftables.md)
|
||||
- [Firewall & NAT Model](docs/firewall_nat.md)
|
||||
- [Reconciliation & Convergence](docs/reconciliation.md)
|
||||
- [Web User Interface Reference](docs/ui.md)
|
||||
- [REST API & WebSocket Reference](docs/api.md)
|
||||
- [CLI Command Reference](docs/cli.md)
|
||||
- [Configuration Reference](docs/configuration.md)
|
||||
- [Security & Privilege Architecture](docs/security.md)
|
||||
- [Backup & Disaster Recovery](docs/backup_restore.md)
|
||||
- [Release Engineering](docs/release.md)
|
||||
- [NX9 Design Principles](docs/DESIGN-PRINCIPLES.md)
|
||||
- [System Architecture](docs/ARCHITECTURE.md)
|
||||
- [Installation Guide](docs/INSTALLATION.md)
|
||||
- [Native WireGuard Engine](docs/NATIVE-WIREGUARD.md)
|
||||
- [Native Network Engine](docs/NATIVE-NETWORK.md)
|
||||
- [Native nftables Engine](docs/NFTABLES.md)
|
||||
- [Firewall & NAT Model](docs/FIREWALL_NAT.md)
|
||||
- [Reconciliation & Convergence](docs/RECONCILIATION.md)
|
||||
- [Web User Interface Reference](docs/UI.md)
|
||||
- [REST API & WebSocket Reference](docs/API.md)
|
||||
- [CLI Command Reference](docs/CLI.md)
|
||||
- [Configuration Reference](docs/CONFIGURATION.md)
|
||||
- [Security & Privilege Architecture](docs/SECURITY.md)
|
||||
- [Backup & Disaster Recovery](docs/BACKUP_RESTORE.md)
|
||||
- [Release Engineering](docs/RELEASE.md)
|
||||
- [Testing Specification](docs/TESTING.md)
|
||||
- [Development Guide](docs/development.md)
|
||||
- [Docker Deployment](docs/docker.md)
|
||||
- [Development Guide](docs/DEVELOPMENT.md)
|
||||
- [Docker Deployment](docs/DOCKER.md)
|
||||
|
||||
## 17. Web UI Screenshots
|
||||
|
||||
The following screenshots provide visual evidence of the production Web UI and its native
|
||||
WireGuard/network administration workflow. Sensitive endpoint and cryptographic values in
|
||||
the captured evidence have been redacted where applicable.
|
||||
|
||||
### Dashboard
|
||||
|
||||

|
||||
|
||||
### Interfaces
|
||||
|
||||

|
||||
|
||||
### Peer Management
|
||||
|
||||

|
||||
|
||||
### New Peer Enrollment
|
||||
|
||||

|
||||
|
||||
### Client Configuration Export
|
||||
|
||||

|
||||
|
||||
### QR Code Export
|
||||
|
||||

|
||||
|
||||
### Networks
|
||||
|
||||

|
||||
|
||||
### IP Forwarding
|
||||
|
||||

|
||||
|
||||
### NAT & Masquerade
|
||||
|
||||

|
||||
|
||||
### Reconciliation
|
||||
|
||||

|
||||
|
||||
### Diagnostics — System, Network & WAN
|
||||
|
||||

|
||||
|
||||
### Diagnostics — Firewall, NAT, MTU & Reconciliation
|
||||
|
||||

|
||||
|
||||
### Settings
|
||||
|
||||

|
||||
|
||||
### Backups
|
||||
|
||||

|
||||
|
||||
> **Additional evidence:** `screenshots/admin-instance.png` contains the captured administrative
|
||||
> instance documentation and is retained in the repository alongside the UI screenshots.
|
||||
- [Development Guide](docs/DEVELOPMENT.md)
|
||||
- [Docker Deployment](docs/DOCKER.md)
|
||||
@@ -23,7 +23,7 @@ reconciliation_interval_secs = 60
|
||||
dir = "/var/lib/nx9-wg/backups"
|
||||
|
||||
# Maximum number of automated backup snapshots to retain
|
||||
max_count = 10
|
||||
max_count = 5
|
||||
|
||||
# Optional cron schedule for automated database backups (e.g. "0 2 * * *" for 02:00 UTC)
|
||||
# schedule = "0 2 * * *"
|
||||
|
||||
@@ -20,6 +20,7 @@ pub use error::{ApiError, ApiResult, ErrorBody, ErrorResponse};
|
||||
pub use profile_resolver::ClientProfileResolver;
|
||||
pub use reconciliation::{
|
||||
ReconciliationAction, ReconciliationEngine, ReconciliationPlan, ReconciliationReport,
|
||||
collect_managed_wg_subnets,
|
||||
};
|
||||
pub use routes::build_api_router;
|
||||
pub use state::{AppState, SystemEvent};
|
||||
@@ -6,6 +6,7 @@ use chrono::Utc;
|
||||
use ipnet::IpNet;
|
||||
use nx9_wg_core::types::audit::AuditEventType;
|
||||
use nx9_wg_core::types::wireguard::PeerState;
|
||||
use nx9_wg_db::Store;
|
||||
use nx9_wg_network::NetworkEngine;
|
||||
use nx9_wireguard::WireGuardEngine;
|
||||
use serde::{Deserialize, Serialize};
|
||||
@@ -42,6 +43,34 @@ fn matches_allowed_ips(live_allowed_ips: &[String], desired_str: &str) -> bool {
|
||||
desired_nets == live_nets
|
||||
}
|
||||
|
||||
/// Collect Interface CIDRs plus enabled Subnet Network CIDRs for NAT/forwarding.
|
||||
///
|
||||
/// Interface addresses remain the WireGuard transport identity. Enabled Network
|
||||
/// CIDRs are the peer allocation domains and must be masqueraded so selected-
|
||||
/// Network peers receive the same full-tunnel Internet path as Interface-CIDR
|
||||
/// peers. `network_id = null` peers still match the Interface CIDR.
|
||||
pub async fn collect_managed_wg_subnets(store: &Store) -> ApiResult<Vec<IpNet>> {
|
||||
let mut subnets = Vec::new();
|
||||
|
||||
for iface in store.list_interfaces().await? {
|
||||
if !iface.enabled {
|
||||
continue;
|
||||
}
|
||||
subnets.push(iface.address_v4);
|
||||
if let Some(v6) = iface.address_v6 {
|
||||
subnets.push(v6);
|
||||
}
|
||||
}
|
||||
|
||||
for net in store.list_networks().await? {
|
||||
if net.enabled {
|
||||
subnets.push(net.cidr);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(subnets)
|
||||
}
|
||||
|
||||
/// Individual action proposed or taken by the reconciler.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ReconciliationAction {
|
||||
@@ -355,7 +384,7 @@ impl ReconciliationEngine {
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Routes
|
||||
// 2. Routes (SQLite Routes table only; peer-allocation Networks are not routes)
|
||||
let desired_routes = self.state.store.list_routes().await?;
|
||||
let enabled_routes: Vec<_> = desired_routes.iter().filter(|r| r.enabled).collect();
|
||||
let has_route_drift = self
|
||||
@@ -401,15 +430,7 @@ impl ReconciliationEngine {
|
||||
.map(|s| s.value == "true" || s.value == "1")
|
||||
.unwrap_or(true);
|
||||
|
||||
let mut wg_subnets = Vec::new();
|
||||
for iface in &desired_interfaces {
|
||||
if iface.enabled {
|
||||
wg_subnets.push(iface.address_v4);
|
||||
if let Some(v6) = iface.address_v6 {
|
||||
wg_subnets.push(v6);
|
||||
}
|
||||
}
|
||||
}
|
||||
let wg_subnets = collect_managed_wg_subnets(&self.state.store).await?;
|
||||
|
||||
let expected_ruleset = nx9_wg_network::NftablesRulesetBuilder::build(
|
||||
&resolved_fw_rules,
|
||||
@@ -484,7 +505,6 @@ impl ReconciliationEngine {
|
||||
let mut details = Vec::new();
|
||||
|
||||
// 1. Sync all active WireGuard interfaces and their peers
|
||||
let mut wg_subnets = Vec::new();
|
||||
for iface in &desired_interfaces {
|
||||
if iface.enabled {
|
||||
let peers = self.state.store.list_peers_for_interface(iface.id).await?;
|
||||
@@ -497,10 +517,6 @@ impl ReconciliationEngine {
|
||||
iface.name
|
||||
))
|
||||
})?;
|
||||
wg_subnets.push(iface.address_v4);
|
||||
if let Some(v6) = iface.address_v6 {
|
||||
wg_subnets.push(v6);
|
||||
}
|
||||
details.push(format!(
|
||||
"Synchronized interface '{}' with {} peers",
|
||||
iface.name,
|
||||
@@ -515,7 +531,9 @@ impl ReconciliationEngine {
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Sync Routes
|
||||
let wg_subnets = collect_managed_wg_subnets(&self.state.store).await?;
|
||||
|
||||
// 2. Sync Routes (SQLite Routes table only; peer-allocation Networks are not routes)
|
||||
let routes = self.state.store.list_routes().await?;
|
||||
self.net_engine
|
||||
.sync_routes(&routes)
|
||||
|
||||
@@ -630,7 +630,7 @@
|
||||
<label class="form-label">Network</label>
|
||||
<select id="peer-network" class="form-select">
|
||||
<option value="">Auto-allocate next IP</option>
|
||||
${networksData.map(n => `<option value="${n.name}">${escapeHtml(n.name)} (${n.cidr})</option>`).join('')}
|
||||
${networksData.map(n => n && n.id ? `<option value="${n.id}">${escapeHtml(n.name)} (${n.cidr})</option>` : '').join('')}
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
@@ -668,13 +668,17 @@
|
||||
}
|
||||
};
|
||||
|
||||
function isNetworkUuid(value) {
|
||||
return /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(String(value || ''));
|
||||
}
|
||||
|
||||
window.submitCreatePeer = async function() {
|
||||
const errBox = document.getElementById('peer-modal-error');
|
||||
if (errBox) errBox.style.display = 'none';
|
||||
|
||||
const name = document.getElementById('peer-name')?.value?.trim();
|
||||
const ifaceId = document.getElementById('peer-iface')?.value;
|
||||
const network = document.getElementById('peer-network')?.value;
|
||||
const rawNetworkValue = (document.getElementById('peer-network')?.value || '').trim();
|
||||
const mtu = parseInt(document.getElementById('rec-mtu-val')?.textContent || '1420', 10);
|
||||
|
||||
if (!name || !ifaceId) {
|
||||
@@ -685,6 +689,22 @@
|
||||
return;
|
||||
}
|
||||
|
||||
// Resolve the selector back to the Network API object and send only its UUID.
|
||||
// Display text is name + CIDR; the request field must never be the name or CIDR.
|
||||
let networkId = null;
|
||||
if (rawNetworkValue) {
|
||||
const selectedNetwork = networksData.find(n => n && String(n.id) === rawNetworkValue);
|
||||
const resolvedId = selectedNetwork ? String(selectedNetwork.id) : rawNetworkValue;
|
||||
if (!isNetworkUuid(resolvedId)) {
|
||||
if (errBox) {
|
||||
errBox.style.display = 'block';
|
||||
errBox.textContent = '❌ Selected Network is missing a valid UUID. Refresh the page and try again.';
|
||||
}
|
||||
return;
|
||||
}
|
||||
networkId = resolvedId;
|
||||
}
|
||||
|
||||
const payload = {
|
||||
name,
|
||||
peer_type: 'road_warrior',
|
||||
@@ -693,7 +713,7 @@
|
||||
persistent_keepalive: 25,
|
||||
dns: '1.1.1.1, 1.0.0.1',
|
||||
allowed_ips: '0.0.0.0/0, ::/0',
|
||||
network: network || null
|
||||
network_id: networkId
|
||||
};
|
||||
|
||||
const res = await api(`/interfaces/${ifaceId}/peers`, {
|
||||
@@ -1547,15 +1567,17 @@
|
||||
|
||||
// ── NAT & Masquerade ────────────────────────────────────────────────────────
|
||||
async function renderNatPage(container) {
|
||||
const [settings, ifaces] = await Promise.all([
|
||||
const [settings, ifaces, networks] = await Promise.all([
|
||||
api('/system/settings'),
|
||||
api('/interfaces')
|
||||
api('/interfaces'),
|
||||
api('/networks')
|
||||
]);
|
||||
|
||||
const settingList = Array.isArray(settings) ? settings : [];
|
||||
const natSetting = settingList.find(s => s.key === 'enable_nat');
|
||||
const isNatEnabled = natSetting ? (natSetting.value === 'true' || natSetting.value === '1') : true;
|
||||
const ifaceList = Array.isArray(ifaces) ? ifaces : [];
|
||||
const networkList = Array.isArray(networks) ? networks.filter(n => n && n.enabled !== false) : [];
|
||||
|
||||
container.innerHTML = `
|
||||
<div class="page-header">
|
||||
@@ -1587,7 +1609,8 @@
|
||||
<div style="font-size: 13px; color: var(--text-secondary); margin-bottom: 12px;">
|
||||
The following subnets are dynamically deduplicated and translated to the host WAN IP:
|
||||
</div>
|
||||
${ifaceList.map(i => `<div style="font-size: 13px; padding: 4px 0;"><span class="key-code">${i.address_v4}</span> (${i.name})</div>`).join('')}
|
||||
${ifaceList.map(i => `<div style="font-size: 13px; padding: 4px 0;"><span class="key-code">${i.address_v4}</span> (${escapeHtml(i.name)})</div>`).join('')}
|
||||
${networkList.map(n => `<div style="font-size: 13px; padding: 4px 0;"><span class="key-code">${n.cidr}</span> (${escapeHtml(n.name)})</div>`).join('')}
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
@@ -16,15 +16,47 @@ use nx9_wg_core::types::wireguard::{
|
||||
WireGuardPublicKey,
|
||||
};
|
||||
use nx9_wg_core::validation::{validate_cidr, validate_mtu, validate_peer_name};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde::{Deserialize, Deserializer, Serialize};
|
||||
use std::str::FromStr;
|
||||
use uuid::Uuid;
|
||||
|
||||
/// Deserialize `network_id` from JSON null/empty as None, and from a UUID string as Some.
|
||||
/// Rejects non-UUID values instead of silently falling back to the Interface CIDR.
|
||||
fn deserialize_optional_network_id<'de, D>(deserializer: D) -> Result<Option<Uuid>, D::Error>
|
||||
where
|
||||
D: Deserializer<'de>,
|
||||
{
|
||||
let value = Option::<serde_json::Value>::deserialize(deserializer)?;
|
||||
match value {
|
||||
None | Some(serde_json::Value::Null) => Ok(None),
|
||||
Some(serde_json::Value::String(s)) => {
|
||||
let trimmed = s.trim();
|
||||
if trimmed.is_empty() {
|
||||
Ok(None)
|
||||
} else {
|
||||
Uuid::parse_str(trimmed).map(Some).map_err(|e| {
|
||||
serde::de::Error::custom(format!("network_id must be a Network UUID: {e}"))
|
||||
})
|
||||
}
|
||||
}
|
||||
Some(other) => Err(serde::de::Error::custom(format!(
|
||||
"network_id must be a UUID string, got {other}"
|
||||
))),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct CreatePeerRequest {
|
||||
pub name: String,
|
||||
pub peer_type: Option<PeerType>,
|
||||
pub profile: Option<PeerProfile>,
|
||||
/// Subnet Network UUID for IP allocation. Also accepts the historical
|
||||
/// enrollment field name `network` when that value is a UUID.
|
||||
#[serde(
|
||||
default,
|
||||
alias = "network",
|
||||
deserialize_with = "deserialize_optional_network_id"
|
||||
)]
|
||||
pub network_id: Option<Uuid>,
|
||||
pub public_key: Option<String>,
|
||||
pub private_key: Option<String>,
|
||||
@@ -264,6 +296,47 @@ async fn validate_no_server_allowed_ips_conflict(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Allocate a peer IPv4 address.
|
||||
///
|
||||
/// When `network_id` is present, allocation MUST use that Network's CIDR and
|
||||
/// MUST NOT fall back to the WireGuard Interface address space.
|
||||
/// When `network_id` is absent, preserve the existing Interface CIDR fallback.
|
||||
async fn allocate_address_v4_for_peer(
|
||||
store: &nx9_wg_db::Store,
|
||||
interface: &nx9_wg_core::types::wireguard::Interface,
|
||||
network_id: Option<Uuid>,
|
||||
) -> ApiResult<IpNet> {
|
||||
match network_id {
|
||||
Some(net_id) => {
|
||||
let network = store
|
||||
.get_network(net_id)
|
||||
.await?
|
||||
.ok_or_else(|| ApiError::NotFound(format!("Network '{net_id}' not found")))?;
|
||||
let allocated =
|
||||
IpAllocator::allocate_next_ip(store, &network, Some(interface), None).await?;
|
||||
if !network.cidr.contains(&allocated.addr()) {
|
||||
return Err(ApiError::Internal(format!(
|
||||
"allocated address {allocated} is outside selected network '{}' ({})",
|
||||
network.name, network.cidr
|
||||
)));
|
||||
}
|
||||
Ok(allocated)
|
||||
}
|
||||
None => {
|
||||
let fallback = Network {
|
||||
id: Uuid::nil(),
|
||||
name: format!("{}-subnet", interface.name),
|
||||
cidr: interface.address_v4,
|
||||
enabled: true,
|
||||
description: None,
|
||||
created_at: Utc::now().naive_utc(),
|
||||
updated_at: Utc::now().naive_utc(),
|
||||
};
|
||||
IpAllocator::allocate_next_ip(store, &fallback, Some(interface), None).await
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// POST /api/v1/interfaces/{id}/peers
|
||||
pub async fn create_peer_handler(
|
||||
State(state): State<AppState>,
|
||||
@@ -289,28 +362,12 @@ pub async fn create_peer_handler(
|
||||
_ => None,
|
||||
};
|
||||
|
||||
// If address_v4 was not explicitly provided, automatically allocate it
|
||||
// If address_v4 was not explicitly provided, automatically allocate it.
|
||||
// A present network_id selects the Subnet Network CIDR; None keeps the
|
||||
// Interface Network CIDR fallback. These paths are intentionally separate.
|
||||
if address_v4.is_none() {
|
||||
let net = match payload.network_id {
|
||||
Some(net_id) => state
|
||||
.store
|
||||
.get_network(net_id)
|
||||
.await?
|
||||
.ok_or_else(|| ApiError::NotFound(format!("Network '{net_id}' not found")))?,
|
||||
None => Network {
|
||||
id: Uuid::nil(),
|
||||
name: format!("{}-subnet", interface.name),
|
||||
cidr: interface.address_v4,
|
||||
enabled: true,
|
||||
description: None,
|
||||
created_at: Utc::now().naive_utc(),
|
||||
updated_at: Utc::now().naive_utc(),
|
||||
},
|
||||
};
|
||||
|
||||
let allocated =
|
||||
IpAllocator::allocate_next_ip(&state.store, &net, Some(&interface), None).await?;
|
||||
address_v4 = Some(allocated);
|
||||
address_v4 =
|
||||
Some(allocate_address_v4_for_peer(&state.store, &interface, payload.network_id).await?);
|
||||
}
|
||||
|
||||
let allowed_ips = match payload.allowed_ips {
|
||||
@@ -794,3 +851,55 @@ pub async fn get_peer_qr_handler(
|
||||
data_url,
|
||||
}))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod create_peer_request_tests {
|
||||
use super::CreatePeerRequest;
|
||||
use uuid::Uuid;
|
||||
|
||||
const NETWORK_UUID: &str = "c2aa62c7-3b9d-43fb-95e7-aa8ab1c71265";
|
||||
|
||||
#[test]
|
||||
fn ui_payload_deserializes_network_id_uuid() {
|
||||
let json = serde_json::json!({
|
||||
"name": "sunil-moto-mobile-network-01",
|
||||
"peer_type": "road_warrior",
|
||||
"profile": "full_tunnel",
|
||||
"mtu": 1280,
|
||||
"persistent_keepalive": 25,
|
||||
"dns": "1.1.1.1, 1.0.0.1",
|
||||
"allowed_ips": "0.0.0.0/0, ::/0",
|
||||
"network_id": NETWORK_UUID
|
||||
});
|
||||
let req: CreatePeerRequest = serde_json::from_value(json).expect("deserialize UI payload");
|
||||
assert_eq!(req.network_id, Some(Uuid::parse_str(NETWORK_UUID).unwrap()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn historical_network_field_uuid_maps_to_network_id() {
|
||||
let json = serde_json::json!({
|
||||
"name": "sunil-moto-mobile-network-01",
|
||||
"network": NETWORK_UUID
|
||||
});
|
||||
let req: CreatePeerRequest =
|
||||
serde_json::from_value(json).expect("deserialize historical network field");
|
||||
assert_eq!(req.network_id, Some(Uuid::parse_str(NETWORK_UUID).unwrap()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn null_network_id_deserializes_as_none() {
|
||||
let json = serde_json::json!({
|
||||
"name": "bob-fallback",
|
||||
"network_id": null
|
||||
});
|
||||
let req: CreatePeerRequest = serde_json::from_value(json).expect("deserialize null");
|
||||
assert_eq!(req.network_id, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_network_id_deserializes_as_none() {
|
||||
let json = serde_json::json!({ "name": "bob-fallback" });
|
||||
let req: CreatePeerRequest = serde_json::from_value(json).expect("deserialize missing");
|
||||
assert_eq!(req.network_id, None);
|
||||
}
|
||||
}
|
||||
@@ -409,3 +409,134 @@ async fn test_list_all_peers_collection_endpoint() {
|
||||
assert_eq!(iface1_peers.len(), 1, "wg1 must return exactly 1 peer");
|
||||
assert_eq!(iface1_peers[0]["name"], "peer-charlie");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_peer_creation_allocates_from_selected_network() {
|
||||
let (app, cookie) = setup_test_app().await;
|
||||
|
||||
// Interface Network (WireGuard transport address space)
|
||||
let create_iface_req = Request::builder()
|
||||
.method("POST")
|
||||
.uri("/api/v1/interfaces")
|
||||
.header(header::COOKIE, &cookie)
|
||||
.header(header::CONTENT_TYPE, "application/json")
|
||||
.body(Body::from(
|
||||
json!({
|
||||
"name": "wg0",
|
||||
"listen_port": 51820,
|
||||
"address_v4": "10.100.0.1/24"
|
||||
})
|
||||
.to_string(),
|
||||
))
|
||||
.unwrap();
|
||||
let resp = app.clone().oneshot(create_iface_req).await.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK);
|
||||
let iface_val: Value =
|
||||
serde_json::from_slice(&to_bytes(resp.into_body(), usize::MAX).await.unwrap()).unwrap();
|
||||
let iface_id = iface_val["id"].as_str().unwrap().to_string();
|
||||
assert_eq!(iface_val["address_v4"], "10.100.0.1/24");
|
||||
|
||||
// Subnet Network (peer allocation domain)
|
||||
let create_net_req = Request::builder()
|
||||
.method("POST")
|
||||
.uri("/api/v1/networks")
|
||||
.header(header::COOKIE, &cookie)
|
||||
.header(header::CONTENT_TYPE, "application/json")
|
||||
.body(Body::from(
|
||||
json!({
|
||||
"name": "mobile-clients",
|
||||
"cidr": "10.100.2.0/24"
|
||||
})
|
||||
.to_string(),
|
||||
))
|
||||
.unwrap();
|
||||
let resp = app.clone().oneshot(create_net_req).await.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK);
|
||||
let net_val: Value =
|
||||
serde_json::from_slice(&to_bytes(resp.into_body(), usize::MAX).await.unwrap()).unwrap();
|
||||
let network_id = net_val["id"].as_str().unwrap();
|
||||
assert_eq!(net_val["cidr"], "10.100.2.0/24");
|
||||
|
||||
// Exact production enrollment payload: selected Subnet Network UUID as network_id.
|
||||
let selected_peer_req = Request::builder()
|
||||
.method("POST")
|
||||
.uri(format!("/api/v1/interfaces/{iface_id}/peers"))
|
||||
.header(header::COOKIE, &cookie)
|
||||
.header(header::CONTENT_TYPE, "application/json")
|
||||
.body(Body::from(
|
||||
json!({
|
||||
"name": "sunil-moto-mobile-network-01",
|
||||
"peer_type": "road_warrior",
|
||||
"profile": "full_tunnel",
|
||||
"mtu": 1280,
|
||||
"persistent_keepalive": 25,
|
||||
"dns": "1.1.1.1, 1.0.0.1",
|
||||
"allowed_ips": "0.0.0.0/0, ::/0",
|
||||
"network_id": network_id
|
||||
})
|
||||
.to_string(),
|
||||
))
|
||||
.unwrap();
|
||||
let resp = app.clone().oneshot(selected_peer_req).await.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK);
|
||||
let selected_peer: Value =
|
||||
serde_json::from_slice(&to_bytes(resp.into_body(), usize::MAX).await.unwrap()).unwrap();
|
||||
let selected_addr = selected_peer["address_v4"].as_str().unwrap();
|
||||
assert_eq!(
|
||||
selected_addr, "10.100.2.1/32",
|
||||
"selected Network must allocate the first host of 10.100.2.0/24, got {selected_addr}"
|
||||
);
|
||||
assert!(
|
||||
selected_addr.starts_with("10.100.2."),
|
||||
"selected Network must allocate from 10.100.2.0/24, got {selected_addr}"
|
||||
);
|
||||
assert!(
|
||||
!selected_addr.starts_with("10.100.0."),
|
||||
"must not allocate from Interface Network 10.100.0.0/24 when a Subnet Network is selected, got {selected_addr}"
|
||||
);
|
||||
assert!(selected_addr.ends_with("/32"));
|
||||
|
||||
// network_id = null preserves existing fallback (Interface Network CIDR)
|
||||
let fallback_peer_req = Request::builder()
|
||||
.method("POST")
|
||||
.uri(format!("/api/v1/interfaces/{iface_id}/peers"))
|
||||
.header(header::COOKIE, &cookie)
|
||||
.header(header::CONTENT_TYPE, "application/json")
|
||||
.body(Body::from(
|
||||
json!({
|
||||
"name": "bob-fallback",
|
||||
"peer_type": "road_warrior",
|
||||
"profile": "full_tunnel",
|
||||
"network_id": null
|
||||
})
|
||||
.to_string(),
|
||||
))
|
||||
.unwrap();
|
||||
let resp = app.clone().oneshot(fallback_peer_req).await.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK);
|
||||
let fallback_peer: Value =
|
||||
serde_json::from_slice(&to_bytes(resp.into_body(), usize::MAX).await.unwrap()).unwrap();
|
||||
let fallback_addr = fallback_peer["address_v4"].as_str().unwrap();
|
||||
assert!(
|
||||
fallback_addr.starts_with("10.100.0."),
|
||||
"network_id=null must preserve fallback allocation from Interface Network 10.100.0.0/24, got {fallback_addr}"
|
||||
);
|
||||
assert!(
|
||||
!fallback_addr.starts_with("10.100.2."),
|
||||
"network_id=null must not allocate from a Subnet Network, got {fallback_addr}"
|
||||
);
|
||||
assert!(fallback_addr.ends_with("/32"));
|
||||
|
||||
// WireGuard interface address space is unchanged
|
||||
let get_iface_req = Request::builder()
|
||||
.uri(format!("/api/v1/interfaces/{iface_id}"))
|
||||
.header(header::COOKIE, &cookie)
|
||||
.body(Body::empty())
|
||||
.unwrap();
|
||||
let resp = app.oneshot(get_iface_req).await.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK);
|
||||
let iface_after: Value =
|
||||
serde_json::from_slice(&to_bytes(resp.into_body(), usize::MAX).await.unwrap()).unwrap();
|
||||
assert_eq!(iface_after["name"], "wg0");
|
||||
assert_eq!(iface_after["address_v4"], "10.100.0.1/24");
|
||||
}
|
||||
@@ -5,10 +5,12 @@ use axum::body::Body;
|
||||
use axum::http::{Request, StatusCode};
|
||||
use chrono::Utc;
|
||||
use ipnet::IpNet;
|
||||
use nx9_wg_api::collect_managed_wg_subnets;
|
||||
use nx9_wg_api::reconciliation::ReconciliationEngine;
|
||||
use nx9_wg_api::routes::build_api_router;
|
||||
use nx9_wg_api::state::AppState;
|
||||
use nx9_wg_core::crypto::generate_keypair;
|
||||
use nx9_wg_core::types::network::Network;
|
||||
use nx9_wg_core::types::wireguard::{Interface, Peer, PeerProfile, PeerState, PeerType};
|
||||
use nx9_wg_db::Store;
|
||||
use nx9_wg_network::{NetworkEngine, SimulatedNetworkEngine};
|
||||
@@ -344,6 +346,149 @@ async fn test_forwarding_and_nat_reconciliation_invariants() {
|
||||
assert_eq!(plan.interface_changes, 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_selected_network_dataplane_nat_and_routes() {
|
||||
let (state, iface, _peer, _session_id) = setup_test_context().await;
|
||||
let now = Utc::now().naive_utc();
|
||||
|
||||
let network = Network {
|
||||
id: Uuid::new_v4(),
|
||||
name: "mobile-clients".to_string(),
|
||||
cidr: IpNet::from_str("10.100.2.0/24").unwrap(),
|
||||
enabled: true,
|
||||
description: None,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
};
|
||||
state.store.create_network(&network).await.unwrap();
|
||||
|
||||
let (peer_priv, peer_pub) = generate_keypair();
|
||||
let selected_peer = Peer {
|
||||
id: Uuid::new_v4(),
|
||||
interface_id: iface.id,
|
||||
name: "test-mobile".to_string(),
|
||||
peer_type: PeerType::RoadWarrior,
|
||||
state: PeerState::Active,
|
||||
public_key: peer_pub,
|
||||
private_key: Some(peer_priv),
|
||||
preshared_key: None,
|
||||
endpoint: None,
|
||||
allowed_ips: "0.0.0.0/0, ::/0".to_string(),
|
||||
server_allowed_ips: None,
|
||||
address_v4: Some(IpNet::from_str("10.100.2.1/32").unwrap()),
|
||||
address_v6: None,
|
||||
dns: Some("1.1.1.1, 1.0.0.1".to_string()),
|
||||
mtu: Some(1280),
|
||||
persistent_keepalive: Some(25),
|
||||
profile: PeerProfile::FullTunnel,
|
||||
expires_at: None,
|
||||
last_handshake_at: None,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
};
|
||||
state.store.create_peer(&selected_peer).await.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
selected_peer.server_wireguard_allowed_ips(),
|
||||
"10.100.2.1/32",
|
||||
"server-side AllowedIPs must remain the assigned selected-Network address"
|
||||
);
|
||||
assert_eq!(selected_peer.allowed_ips, "0.0.0.0/0, ::/0");
|
||||
|
||||
let subnets = collect_managed_wg_subnets(&state.store).await.unwrap();
|
||||
assert!(
|
||||
subnets
|
||||
.iter()
|
||||
.any(|s| s.trunc().to_string() == "10.100.0.0/24"),
|
||||
"Interface CIDR must remain in managed NAT subnets"
|
||||
);
|
||||
assert!(
|
||||
subnets
|
||||
.iter()
|
||||
.any(|s| s.trunc().to_string() == "10.100.2.0/24"),
|
||||
"selected Network CIDR must participate in managed NAT subnets"
|
||||
);
|
||||
|
||||
let wg_engine = Arc::new(SimulatedWireGuardEngine::new());
|
||||
let net_engine = Arc::new(SimulatedNetworkEngine::new());
|
||||
let reconciler =
|
||||
ReconciliationEngine::new(state.clone(), wg_engine.clone(), net_engine.clone());
|
||||
|
||||
let report = reconciler.apply().await.unwrap();
|
||||
assert!(report.success);
|
||||
|
||||
let persisted_iface = state.store.get_interface(iface.id).await.unwrap().unwrap();
|
||||
assert_eq!(persisted_iface.address_v4.to_string(), "10.100.0.1/24");
|
||||
assert_eq!(persisted_iface.name, "wg0");
|
||||
let stored_routes = state.store.list_routes().await.unwrap();
|
||||
assert!(
|
||||
!stored_routes
|
||||
.iter()
|
||||
.any(|r| r.destination.trunc().to_string() == "10.100.2.0/24"),
|
||||
"peer-allocation Network CIDR must not be persisted as a static route"
|
||||
);
|
||||
|
||||
let ruleset = net_engine.get_active_nftables_ruleset().await.unwrap();
|
||||
assert!(
|
||||
ruleset.contains("ip saddr 10.100.0.0/24 oifname != \"wg*\" masquerade"),
|
||||
"Interface-CIDR peers must keep existing NAT: {ruleset}"
|
||||
);
|
||||
assert!(
|
||||
ruleset.contains("ip saddr 10.100.2.0/24 oifname != \"wg*\" masquerade"),
|
||||
"selected Network CIDR must be masqueraded for full-tunnel Internet: {ruleset}"
|
||||
);
|
||||
|
||||
let live_stats = wg_engine.get_interface_stats("wg0").await.unwrap().unwrap();
|
||||
assert!(
|
||||
live_stats
|
||||
.peers
|
||||
.iter()
|
||||
.any(|p| p.allowed_ips.iter().any(|a| a == "10.100.2.1/32")),
|
||||
"kernel peer AllowedIPs must include the selected-Network assignment"
|
||||
);
|
||||
|
||||
let (fallback_priv, fallback_pub) = generate_keypair();
|
||||
let fallback_peer = Peer {
|
||||
id: Uuid::new_v4(),
|
||||
interface_id: iface.id,
|
||||
name: "fallback-null-network".to_string(),
|
||||
peer_type: PeerType::RoadWarrior,
|
||||
state: PeerState::Active,
|
||||
public_key: fallback_pub,
|
||||
private_key: Some(fallback_priv),
|
||||
preshared_key: None,
|
||||
endpoint: None,
|
||||
allowed_ips: "0.0.0.0/0, ::/0".to_string(),
|
||||
server_allowed_ips: None,
|
||||
address_v4: Some(IpNet::from_str("10.100.0.2/32").unwrap()),
|
||||
address_v6: None,
|
||||
dns: None,
|
||||
mtu: None,
|
||||
persistent_keepalive: Some(25),
|
||||
profile: PeerProfile::FullTunnel,
|
||||
expires_at: None,
|
||||
last_handshake_at: None,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
};
|
||||
state.store.create_peer(&fallback_peer).await.unwrap();
|
||||
assert_eq!(
|
||||
fallback_peer.server_wireguard_allowed_ips(),
|
||||
"10.100.0.2/32"
|
||||
);
|
||||
|
||||
let report = reconciler.apply().await.unwrap();
|
||||
assert!(report.success);
|
||||
let ruleset = net_engine.get_active_nftables_ruleset().await.unwrap();
|
||||
assert!(ruleset.contains("ip saddr 10.100.0.0/24 oifname != \"wg*\" masquerade"));
|
||||
assert!(ruleset.contains("ip saddr 10.100.2.0/24 oifname != \"wg*\" masquerade"));
|
||||
|
||||
let plan = reconciler.plan().await.unwrap();
|
||||
assert!(!plan.has_drift);
|
||||
assert_eq!(plan.firewall_changes, 0);
|
||||
assert_eq!(plan.route_changes, 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_interface_editing_persistence_and_key_preservation() {
|
||||
let (state, iface, _peer, session_id) = setup_test_context().await;
|
||||
|
||||
@@ -123,6 +123,16 @@ async fn test_ui_spa_index_and_stylesheet_endpoints() {
|
||||
assert!(html.contains("triggerCreateBackup"));
|
||||
assert!(html.contains("openClientExportModal"));
|
||||
assert!(html.contains("openAddPeerModal"));
|
||||
|
||||
// Peer enrollment must submit the selected Network UUID as network_id,
|
||||
// never the display name or CIDR.
|
||||
assert!(html.contains(r#"value="${n.id}""#));
|
||||
assert!(html.contains("${escapeHtml(n.name)} (${n.cidr})"));
|
||||
assert!(html.contains("network_id: networkId"));
|
||||
assert!(html.contains("isNetworkUuid"));
|
||||
assert!(html.contains("selectedNetwork.id"));
|
||||
assert!(!html.contains("network: network || null"));
|
||||
assert!(!html.contains(r#"value="${n.name}""#));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -2,9 +2,10 @@
|
||||
|
||||
use crate::error::{Result, WireGuardError};
|
||||
use chrono::{NaiveDateTime, Utc};
|
||||
use ipnet::IpNet;
|
||||
use nx9_wg_core::types::wireguard::{Interface, Peer, PeerState};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::HashMap;
|
||||
use std::collections::{BTreeSet, HashMap};
|
||||
use std::sync::Arc;
|
||||
use tokio::sync::RwLock;
|
||||
|
||||
@@ -36,6 +37,36 @@ pub struct LiveInterfaceStats {
|
||||
pub is_up: bool,
|
||||
}
|
||||
|
||||
/// Peer tunnel addresses that are not on the Interface connected prefix.
|
||||
///
|
||||
/// Interface-CIDR peers (e.g. 10.100.0.x with wg0 10.100.0.1/24) are already
|
||||
/// reachable via the kernel connected route created by the interface address.
|
||||
/// Selected-Network peers (e.g. 10.100.2.1/32) are not. Those prefixes must be
|
||||
/// installed as on-link device routes on the WireGuard interface so the FIB
|
||||
/// delivers packets into wg0, where cryptokey routing (AllowedIPs) applies.
|
||||
///
|
||||
/// This is not a Routes-table LAN-behind-peer destination and has no gateway.
|
||||
pub fn onlink_peer_address_prefixes(interface: &Interface, peers: &[Peer]) -> Vec<IpNet> {
|
||||
let mut prefixes = BTreeSet::new();
|
||||
for peer in peers.iter().filter(|p| p.state == PeerState::Active) {
|
||||
if let Some(v4) = peer.address_v4
|
||||
&& v4.prefix_len() > 0
|
||||
&& !interface.address_v4.contains(&v4.addr())
|
||||
{
|
||||
prefixes.insert(v4);
|
||||
}
|
||||
if let Some(v6) = peer.address_v6
|
||||
&& v6.prefix_len() > 0
|
||||
&& !interface
|
||||
.address_v6
|
||||
.is_some_and(|iface_v6| iface_v6.contains(&v6.addr()))
|
||||
{
|
||||
prefixes.insert(v6);
|
||||
}
|
||||
}
|
||||
prefixes.into_iter().collect()
|
||||
}
|
||||
|
||||
/// Abstract WireGuard Engine interface for kernel netlink and simulated environments.
|
||||
#[async_trait::async_trait]
|
||||
pub trait WireGuardEngine: Send + Sync {
|
||||
|
||||
@@ -10,7 +10,7 @@ pub mod qr;
|
||||
pub use config_builder::ClientConfigBuilder;
|
||||
pub use engine::{
|
||||
LiveInterfaceStats, LivePeerStats, NativeLinuxWireGuardEngine, SimulatedWireGuardEngine,
|
||||
WireGuardEngine,
|
||||
WireGuardEngine, onlink_peer_address_prefixes,
|
||||
};
|
||||
pub use error::{Result, WireGuardError};
|
||||
pub use qr::{
|
||||
|
||||
@@ -8,7 +8,9 @@
|
||||
//!
|
||||
//! No external commands (wg, ip, wg-quick, nft, sysctl) are ever executed.
|
||||
|
||||
use crate::engine::{LiveInterfaceStats, LivePeerStats, WireGuardEngine};
|
||||
use crate::engine::{
|
||||
LiveInterfaceStats, LivePeerStats, WireGuardEngine, onlink_peer_address_prefixes,
|
||||
};
|
||||
use crate::error::{Result, WireGuardError};
|
||||
use base64::Engine as _;
|
||||
use chrono::NaiveDateTime;
|
||||
@@ -24,9 +26,13 @@ use netlink_packet_wireguard::{
|
||||
};
|
||||
use nx9_wg_core::types::wireguard::{Interface, Peer, PeerState};
|
||||
use rtnetlink::LinkWireguard;
|
||||
use rtnetlink::RouteMessageBuilder;
|
||||
use rtnetlink::packet_route::AddressFamily;
|
||||
use rtnetlink::packet_route::address::{AddressAttribute, AddressMessage};
|
||||
use rtnetlink::packet_route::link::{InfoKind, LinkAttribute, LinkFlags, LinkInfo};
|
||||
use std::net::{IpAddr, SocketAddr};
|
||||
use rtnetlink::packet_route::route::{RouteAddress, RouteAttribute, RouteMessage};
|
||||
use std::collections::HashSet;
|
||||
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr};
|
||||
|
||||
/// Linux Native WireGuard Engine using kernel RTNETLINK and Generic Netlink.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
@@ -852,6 +858,178 @@ fn parse_endpoint(s: &str) -> Result<SocketAddr> {
|
||||
)))
|
||||
}
|
||||
|
||||
/// Install on-link device routes for peer tunnel addresses outside the Interface prefix.
|
||||
///
|
||||
/// Interface-CIDR peers are already covered by the connected route from the
|
||||
/// interface address. Selected-Network peer addresses are not; without a FIB
|
||||
/// path into wg0, cryptokey routing never sees the packet. Routes have no
|
||||
/// gateway and are not Routes-table LAN destinations.
|
||||
async fn ensure_onlink_peer_routes(interface: &Interface, peers: &[Peer]) -> Result<()> {
|
||||
let (handle, _join) = rtnetlink_handle().await?;
|
||||
|
||||
let mut links = handle
|
||||
.link()
|
||||
.get()
|
||||
.match_name(interface.name.to_string())
|
||||
.execute();
|
||||
let Some(link) = links.try_next().await.map_err(|e| {
|
||||
WireGuardError::Netlink(format!(
|
||||
"failed to resolve interface '{}' for on-link routes: {e}",
|
||||
interface.name
|
||||
))
|
||||
})?
|
||||
else {
|
||||
return Ok(());
|
||||
};
|
||||
let link_index = link.header.index;
|
||||
|
||||
let desired: HashSet<IpNet> = onlink_peer_address_prefixes(interface, peers)
|
||||
.into_iter()
|
||||
.collect();
|
||||
|
||||
let live = list_onlink_routes_for_index(&handle, link_index).await?;
|
||||
|
||||
for prefix in &desired {
|
||||
if live.contains(prefix) {
|
||||
continue;
|
||||
}
|
||||
add_onlink_device_route(&handle, link_index, *prefix).await?;
|
||||
}
|
||||
|
||||
let iface_v4 = interface.address_v4.trunc();
|
||||
let iface_v6 = interface.address_v6.map(|n| n.trunc());
|
||||
for prefix in live {
|
||||
let is_host = matches!(prefix, IpNet::V4(n) if n.prefix_len() == 32)
|
||||
|| matches!(prefix, IpNet::V6(n) if n.prefix_len() == 128);
|
||||
if !is_host {
|
||||
continue;
|
||||
}
|
||||
if prefix.trunc() == iface_v4 || iface_v6 == Some(prefix.trunc()) {
|
||||
continue;
|
||||
}
|
||||
if desired.contains(&prefix) {
|
||||
continue;
|
||||
}
|
||||
let _ = delete_onlink_device_route(&handle, link_index, prefix).await;
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn list_onlink_routes_for_index(
|
||||
handle: &rtnetlink::Handle,
|
||||
link_index: u32,
|
||||
) -> Result<HashSet<IpNet>> {
|
||||
let mut results = HashSet::new();
|
||||
for family in [AddressFamily::Inet, AddressFamily::Inet6] {
|
||||
let mut req = RouteMessage::default();
|
||||
req.header.address_family = family;
|
||||
let mut stream = handle.route().get(req).execute();
|
||||
while let Some(msg) = stream
|
||||
.try_next()
|
||||
.await
|
||||
.map_err(|e| WireGuardError::Netlink(format!("RTNETLINK route dump failed: {e}")))?
|
||||
{
|
||||
let mut dest_ip = match family {
|
||||
AddressFamily::Inet => IpAddr::V4(Ipv4Addr::UNSPECIFIED),
|
||||
AddressFamily::Inet6 => IpAddr::V6(Ipv6Addr::UNSPECIFIED),
|
||||
_ => continue,
|
||||
};
|
||||
let mut oif = None;
|
||||
let mut has_gateway = false;
|
||||
for attr in &msg.attributes {
|
||||
match attr {
|
||||
RouteAttribute::Destination(RouteAddress::Inet(v4)) => {
|
||||
dest_ip = IpAddr::V4(*v4);
|
||||
}
|
||||
RouteAttribute::Destination(RouteAddress::Inet6(v6)) => {
|
||||
dest_ip = IpAddr::V6(*v6);
|
||||
}
|
||||
RouteAttribute::Gateway(_) => has_gateway = true,
|
||||
RouteAttribute::Oif(idx) => oif = Some(*idx),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
if has_gateway || oif != Some(link_index) {
|
||||
continue;
|
||||
}
|
||||
if let Ok(net) = IpNet::new(dest_ip, msg.header.destination_prefix_length) {
|
||||
results.insert(net);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(results)
|
||||
}
|
||||
|
||||
async fn add_onlink_device_route(
|
||||
handle: &rtnetlink::Handle,
|
||||
link_index: u32,
|
||||
prefix: IpNet,
|
||||
) -> Result<()> {
|
||||
let exec_result = match prefix {
|
||||
IpNet::V4(v4) => {
|
||||
let msg = RouteMessageBuilder::<Ipv4Addr>::new()
|
||||
.destination_prefix(v4.addr(), v4.prefix_len())
|
||||
.output_interface(link_index)
|
||||
.build();
|
||||
handle.route().add(msg).execute().await
|
||||
}
|
||||
IpNet::V6(v6) => {
|
||||
let msg = RouteMessageBuilder::<Ipv6Addr>::new()
|
||||
.destination_prefix(v6.addr(), v6.prefix_len())
|
||||
.output_interface(link_index)
|
||||
.build();
|
||||
handle.route().add(msg).execute().await
|
||||
}
|
||||
};
|
||||
if let Err(e) = exec_result {
|
||||
let err_str = e.to_string();
|
||||
if err_str.contains("File exists") || err_str.contains("17") {
|
||||
return Ok(());
|
||||
}
|
||||
if err_str.contains("permission")
|
||||
|| err_str.contains("EPERM")
|
||||
|| err_str.contains("Operation not permitted")
|
||||
{
|
||||
return Err(WireGuardError::PermissionDenied(format!(
|
||||
"insufficient privileges to add on-link route '{prefix}': {e}"
|
||||
)));
|
||||
}
|
||||
return Err(WireGuardError::Netlink(format!(
|
||||
"failed to add on-link route '{prefix}': {e}"
|
||||
)));
|
||||
}
|
||||
tracing::info!(prefix = %prefix, "On-link peer address route added via RTNETLINK");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn delete_onlink_device_route(
|
||||
handle: &rtnetlink::Handle,
|
||||
link_index: u32,
|
||||
prefix: IpNet,
|
||||
) -> Result<()> {
|
||||
let exec_result = match prefix {
|
||||
IpNet::V4(v4) => {
|
||||
let msg = RouteMessageBuilder::<Ipv4Addr>::new()
|
||||
.destination_prefix(v4.addr(), v4.prefix_len())
|
||||
.output_interface(link_index)
|
||||
.build();
|
||||
handle.route().del(msg).execute().await
|
||||
}
|
||||
IpNet::V6(v6) => {
|
||||
let msg = RouteMessageBuilder::<Ipv6Addr>::new()
|
||||
.destination_prefix(v6.addr(), v6.prefix_len())
|
||||
.output_interface(link_index)
|
||||
.build();
|
||||
handle.route().del(msg).execute().await
|
||||
}
|
||||
};
|
||||
if let Err(e) = exec_result {
|
||||
tracing::debug!(prefix = %prefix, error = %e, "On-link peer address route delete skipped");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ── WireGuardEngine Trait Implementation ─────────────────────────────────────
|
||||
|
||||
#[async_trait::async_trait]
|
||||
@@ -863,6 +1041,16 @@ impl WireGuardEngine for NativeLinuxWireGuardEngine {
|
||||
// 2. Configure the WireGuard device (private key, listen port, peers)
|
||||
configure_device(interface, peers).await?;
|
||||
|
||||
// 3. On-link device routes for peer tunnel addresses outside the
|
||||
// Interface connected prefix (cryptokey routing still uses AllowedIPs).
|
||||
if let Err(e) = ensure_onlink_peer_routes(interface, peers).await {
|
||||
tracing::warn!(
|
||||
interface = %interface.name,
|
||||
error = %e,
|
||||
"On-link peer address routes skipped"
|
||||
);
|
||||
}
|
||||
|
||||
tracing::info!(
|
||||
interface = %interface.name,
|
||||
active_peers = peers.iter().filter(|p| p.state == PeerState::Active).count(),
|
||||
|
||||
@@ -53,7 +53,7 @@ The `port_range` field supports three RFC-compliant formats:
|
||||
NAT masquerading is governed by key-value appliance settings in SQLite:
|
||||
|
||||
- **`enable_nat`**: Boolean string (`"true"` / `"false"`). When enabled, all active managed WireGuard subnets are masqueraded outbound to the host WAN interface.
|
||||
- **Dynamic Subnet Calculation**: The reconciliation engine queries all enabled interfaces (`Interface.address_v4`) and generates dedicated masquerade rules for each unique subnet.
|
||||
- **Dynamic Subnet Calculation**: The reconciliation engine queries enabled Interface address CIDRs and enabled Subnet Network CIDRs, then generates dedicated masquerade rules for each unique subnet. Interface addresses remain the WireGuard transport identity; Network CIDRs are the peer allocation domains.
|
||||
|
||||
---
|
||||
|
||||
@@ -66,8 +66,9 @@ table inet nx9_wg {
|
||||
|
||||
Outbound NAT masquerading is dynamically scoped exclusively to managed WireGuard client subnets:
|
||||
1. **Subnet Deduplication**: Overlapping subnets are merged to prevent redundant rules.
|
||||
2. **Interface Exclusion**: Traffic routing back into the WireGuard interface (`oifname != "wg0"`) is not masqueraded to preserve true source IPs for site-to-site tunnels.
|
||||
2. **Interface Exclusion**: Traffic routing back into the WireGuard interface (`oifname != "wg*"`) is not masqueraded to preserve true source IPs for site-to-site tunnels.
|
||||
3. **No Catch-All Masquerade**: `nx9-wg` never creates a catch-all `masquerade` rule that affects non-WireGuard traffic on the host.
|
||||
4. **Interface and Subnet Network CIDRs**: Masquerade sources include each enabled Interface address CIDR and each enabled Subnet Network CIDR. A peer allocated from a selected Network (for example outside the WireGuard interface `/24`) is masqueraded from that Network CIDR; the Interface address itself is unchanged.
|
||||
|
||||
---
|
||||
|
||||
@@ -5,38 +5,38 @@ Welcome to the official documentation for the **NX9 WireGuard (`nx9-wg`)** appli
|
||||
---
|
||||
|
||||
## 1. Getting Started & Philosophy
|
||||
- [**NX9 Design Principles**](design-principles.md) — Architectural philosophy, self-hosted sovereignty, zero scripting runtime, and SQLite authority.
|
||||
- [**Installation & Deployment Guide**](installation.md) — Production installation, systemd service, admin bootstrap, first interface, and peer setup.
|
||||
- [**Linux Platform & Kernel Requirements**](linux_requirements.md) — Kernel 5.6+, in-tree WireGuard module, Netlink sockets, `libnftables.so.1`, and capabilities.
|
||||
- [**NX9 Design Principles**](DESIGN-PRINCIPLES.md) — Architectural philosophy, self-hosted sovereignty, zero scripting runtime, and SQLite authority.
|
||||
- [**Installation & Deployment Guide**](INSTALLATION.md) — Production installation, systemd service, admin bootstrap, first interface, and peer setup.
|
||||
- [**Linux Platform & Kernel Requirements**](LINUX_REQUIREMENTS.md) — Kernel 5.6+, in-tree WireGuard module, Netlink sockets, `libnftables.so.1`, and capabilities.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture & Native Linux Execution
|
||||
- [**System Architecture & Workspace Structure**](architecture.md) — Six-crate workspace breakdown, layer boundaries, and end-to-end data flows.
|
||||
- [**Native WireGuard Netlink Engine**](native-wireguard.md) — Direct RTNETLINK and Generic Netlink (`wireguard`) protocol implementation.
|
||||
- [**Native Network & Routing Engine**](native-network.md) — RTNETLINK link/address/route lifecycle and direct procfs IP packet forwarding.
|
||||
- [**Native nftables Engine**](nftables.md) — In-process `libnftables.so.1` FFI transactions and dedicated `table inet nx9_wg` scoping.
|
||||
- [**Firewall & NAT Domain Model**](firewall_nat.md) — Typed rules, protocol groups, port ranges, priorities, and outbound NAT masquerading.
|
||||
- [**System Architecture & Workspace Structure**](ARCHITECTURE.md) — Six-crate workspace breakdown, layer boundaries, and end-to-end data flows.
|
||||
- [**Native WireGuard Netlink Engine**](NATIVE-WIREGUARD.md) — Direct RTNETLINK and Generic Netlink (`wireguard`) protocol implementation.
|
||||
- [**Native Network & Routing Engine**](NATIVE-NETWORK.md) — RTNETLINK link/address/route lifecycle and direct procfs IP packet forwarding.
|
||||
- [**Native nftables Engine**](NFTABLES.md) — In-process `libnftables.so.1` FFI transactions and dedicated `table inet nx9_wg` scoping.
|
||||
- [**Firewall & NAT Domain Model**](FIREWALL_NAT.md) — Typed rules, protocol groups, port ranges, priorities, and outbound NAT masquerading.
|
||||
|
||||
---
|
||||
|
||||
## 3. Control Plane, UI & Telemetry
|
||||
- [**Reconciliation Engine & Convergence**](reconciliation.md) — Closed-loop drift detection, read-only planning, serialized apply, and convergence lifecycle states.
|
||||
- [**Web User Interface (SPA)**](ui.md) — Zero-dependency embedded HTML5/CSS/JS frontend, theme engine, and all 15 application routes.
|
||||
- [**Axum REST API & WebSocket Protocol**](api.md) — Complete endpoint reference, JSON schemas, error handling, and real-time event broadcaster.
|
||||
- [**Native CLI Command Reference**](cli.md) — Full reference for all 17 CLI subcommands, multi-format output (`table`/`json`/`yaml`/`csv`), and secret files.
|
||||
- [**Reconciliation Engine & Convergence**](RECONCILIATION.md) — Closed-loop drift detection, read-only planning, serialized apply, and convergence lifecycle states.
|
||||
- [**Web User Interface (SPA)**](UI.md) — Zero-dependency embedded HTML5/CSS/JS frontend, theme engine, and all 15 application routes.
|
||||
- [**Axum REST API & WebSocket Protocol**](API.md) — Complete endpoint reference, JSON schemas, error handling, and real-time event broadcaster.
|
||||
- [**Native CLI Command Reference**](CLI.md) — Full reference for all 18 CLI subcommands, multi-format output (`table`/`json`/`yaml`/`csv`), and secret files.
|
||||
|
||||
---
|
||||
|
||||
## 4. Security & Disaster Recovery
|
||||
- [**Security Model & Privilege Architecture**](security.md) — Single admin model (`CHECK (id=1)`), Argon2id hashing, SHA-256 tokens, permissions matrix, and brute-force protection.
|
||||
- [**Backup & Disaster Recovery Guide**](backup_restore.md) — Atomic online SQLite backups (`VACUUM INTO`), SHA-256 manifests, and pre-restore safety snapshots.
|
||||
- [**Security Model & Privilege Architecture**](SECURITY.md) — Single admin model (`CHECK (id=1)`), Argon2id hashing, SHA-256 tokens, permissions matrix, and brute-force protection.
|
||||
- [**Backup & Disaster Recovery Guide**](BACKUP_RESTORE.md) — Atomic online SQLite backups (`VACUUM INTO`), SHA-256 manifests, and pre-restore safety snapshots.
|
||||
|
||||
---
|
||||
|
||||
## 5. Operations, Development & Release
|
||||
- [**Release Engineering & Packaging**](release.md) — Standalone distribution packages (`.tar.gz`/`.tar.xz`), systemd sandboxing, installer, and rollback strategy.
|
||||
- [**Release Engineering & Packaging**](RELEASE.md) — Standalone distribution packages (`.tar.gz`/`.tar.xz`), systemd sandboxing, installer, and rollback strategy.
|
||||
- [**Comprehensive Testing Specification**](TESTING.md) — Multi-tiered test suites, SAFE mode (`LIVE=0`) vs real-kernel mode (`LIVE=1`), and automated security audits.
|
||||
- [**Developer & Contributing Guide**](development.md) — Building, testing, linting, and workspace contribution standards.
|
||||
- [**Configuration Reference**](configuration.md) — TOML configuration format and `NX9_WG_*` environment variable precedence.
|
||||
- [**Docker & Container Deployment**](docker.md) — Containerized deployment with Linux capability isolation and volume persistence.
|
||||
- [**Developer & Contributing Guide**](DEVELOPMENT.md) — Building, testing, linting, and workspace contribution standards.
|
||||
- [**Configuration Reference**](CONFIGURATION.md) — TOML configuration format and `NX9_WG_*` environment variable precedence.
|
||||
- [**Docker & Container Deployment**](DOCKER.md) — Containerized deployment with Linux capability isolation and volume persistence.
|
||||
@@ -6,7 +6,7 @@ This document describes the release packaging, artifact verification, filesystem
|
||||
|
||||
## 1. Release Packaging Pipeline
|
||||
|
||||
Release archives are generated using [`scripts/package-release.sh`](file:///home/sunil/Programs/nx9-wg/scripts/package-release.sh):
|
||||
Release archives are generated using [`scripts/package-release.sh`](../scripts/package-release.sh):
|
||||
|
||||
```bash
|
||||
bash scripts/package-release.sh
|
||||
@@ -0,0 +1,595 @@
|
||||
# NX9-WG Stress Test --- Remote LAN Multi-Path Integration
|
||||
|
||||
**Project:** NX9-WG\
|
||||
**Test Type:** Integration / Stress Test\
|
||||
**Status:** PASS\
|
||||
**Date:** 2026-08-19\
|
||||
**Purpose:** Validate robust remote access to a protected LAN through
|
||||
NX9-WG across substantially different underlying network paths.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
## 1. Executive Summary
|
||||
|
||||
This test validates that `nx9-wg` can provide authenticated, routed
|
||||
access to a remote `192.168.1.0/24` LAN while the WireGuard client uses
|
||||
different and independent Internet/access-network paths.
|
||||
|
||||
The test deliberately exercised NX9-WG through:
|
||||
|
||||
1. Cellular 5G → mobile WAN AP → laptop.
|
||||
2. Airtel Wi-Fi → Pixel Wi-Fi Station + Access Point concurrency →
|
||||
laptop.
|
||||
3. A separate Wi-Fi network → laptop.
|
||||
|
||||
In all tested paths, the same NX9-WG peer successfully established the
|
||||
VPN and reached hosts and services on the protected LAN.
|
||||
|
||||
The strongest validation was successful interactive SSH access to
|
||||
`192.168.1.200`, in addition to ICMP, HTTP, network filesystem access,
|
||||
and LAN host discovery.
|
||||
|
||||
**Overall result: PASS.**
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
## 2. Test Objective
|
||||
|
||||
Validate that NX9-WG:
|
||||
|
||||
- establishes a WireGuard tunnel independently of the client's
|
||||
underlying access network;
|
||||
- correctly routes traffic from a remote peer to the protected LAN;
|
||||
- handles different upstream NAT/network topologies;
|
||||
- provides access to multiple LAN hosts rather than only a single
|
||||
endpoint;
|
||||
- supports real application traffic over the tunnel;
|
||||
- preserves connectivity when the client changes access-network
|
||||
topology;
|
||||
- does not require changes to the protected LAN or ISP router
|
||||
configuration.
|
||||
|
||||
This test is intended as an **integration and robustness validation**,
|
||||
not as a throughput benchmark.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
## 3. Network Topology
|
||||
|
||||
### Protected LAN
|
||||
|
||||
``` text
|
||||
192.168.1.0/24
|
||||
```
|
||||
|
||||
Representative hosts:
|
||||
|
||||
``` text
|
||||
192.168.1.1 Airtel AirFiber / Nokia AAP321NK
|
||||
192.168.1.200 Debian server / Thakares IoT Hub
|
||||
```
|
||||
|
||||
### NX9-WG peer
|
||||
|
||||
``` text
|
||||
WireGuard interface: Office-Laptop
|
||||
WireGuard address: 10.100.0.6/32
|
||||
MTU: 1280
|
||||
```
|
||||
|
||||
The important architectural property is that the client-side underlay
|
||||
network can change while the WireGuard overlay identity and protected
|
||||
LAN remain unchanged.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 4. Test Scenario A --- Cellular 5G
|
||||
|
||||
## Path
|
||||
|
||||
``` text
|
||||
5G Internet
|
||||
│
|
||||
▼
|
||||
Mobile phone
|
||||
│
|
||||
│ WAN AP / hotspot
|
||||
▼
|
||||
Laptop
|
||||
│
|
||||
│ NX9-WG
|
||||
▼
|
||||
NX9-WG server
|
||||
│
|
||||
│ routed LAN access
|
||||
▼
|
||||
192.168.1.0/24
|
||||
```
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Mobile phone connected to cellular 5G.
|
||||
2. Mobile phone provided WAN/AP connectivity.
|
||||
3. Laptop connected to the mobile AP.
|
||||
4. NX9-WG VPN was enabled.
|
||||
5. Laptop received an Internet address on the mobile network.
|
||||
6. Remote LAN routes became reachable through NX9-WG.
|
||||
|
||||
## Observed client addressing
|
||||
|
||||
``` text
|
||||
wlan0: 10.137.106.48/24
|
||||
WireGuard: 10.100.0.6/32
|
||||
```
|
||||
|
||||
## Validation
|
||||
|
||||
### Internet
|
||||
|
||||
``` text
|
||||
ping google.com
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
0% packet loss
|
||||
```
|
||||
|
||||
### LAN host
|
||||
|
||||
``` text
|
||||
ping 192.168.1.200
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
0% packet loss
|
||||
```
|
||||
|
||||
### LAN discovery
|
||||
|
||||
An initial ordinary `nmap -sP 192.168.1.0/24` did not discover hosts
|
||||
while the LAN was only reachable through the routed WireGuard path.
|
||||
Individual routed hosts were nevertheless reachable.
|
||||
|
||||
This was subsequently validated more comprehensively in Scenario C using
|
||||
`nmap` with the active routed path.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 5. Test Scenario B --- Wi-Fi STA + AP Concurrency
|
||||
|
||||
This was the more interesting underlay test.
|
||||
|
||||
The Pixel device supports simultaneous Wi-Fi client (STA) and Access
|
||||
Point operation.
|
||||
|
||||
## Path
|
||||
|
||||
``` text
|
||||
Airtel Wi-Fi
|
||||
│
|
||||
▼
|
||||
Pixel Wi-Fi STA
|
||||
│
|
||||
Pixel Wi-Fi AP
|
||||
│
|
||||
▼
|
||||
Laptop
|
||||
│
|
||||
NX9-WG
|
||||
│
|
||||
▼
|
||||
NX9-WG server
|
||||
│
|
||||
▼
|
||||
192.168.1.0/24
|
||||
```
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Pixel connected to Airtel Wi-Fi.
|
||||
2. Pixel simultaneously enabled its Access Point.
|
||||
3. Laptop connected to the Pixel AP.
|
||||
4. Laptop enabled NX9-WG.
|
||||
5. No cellular Internet was used.
|
||||
6. Remote LAN access worked immediately.
|
||||
|
||||
## Result
|
||||
|
||||
**PASS**
|
||||
|
||||
This demonstrates that NX9-WG remains functional when the client reaches
|
||||
the Internet through a Wi-Fi STA/AP-concurrent intermediate device.
|
||||
|
||||
No cellular fallback was required.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 6. Test Scenario C --- Remote LAN Application and Host Validation
|
||||
|
||||
A further test was performed from an external Wi-Fi network.
|
||||
|
||||
## Client addressing
|
||||
|
||||
``` text
|
||||
wlan0:
|
||||
10.24.3.48/24
|
||||
|
||||
WireGuard:
|
||||
10.100.0.6/32
|
||||
```
|
||||
|
||||
The underlying Wi-Fi network therefore differed from the protected LAN.
|
||||
|
||||
## Internet validation
|
||||
|
||||
``` bash
|
||||
ping nx9.in
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
``` text
|
||||
5 packets transmitted
|
||||
5 received
|
||||
0% packet loss
|
||||
|
||||
min/avg/max/mdev:
|
||||
82.653 / 93.637 / 101.691 / 7.049 ms
|
||||
```
|
||||
|
||||
## Public Internet validation
|
||||
|
||||
``` bash
|
||||
ping google.com
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
``` text
|
||||
5 packets transmitted
|
||||
5 received
|
||||
0% packet loss
|
||||
|
||||
min/avg/max/mdev:
|
||||
87.633 / 118.502 / 165.108 / 30.034 ms
|
||||
```
|
||||
|
||||
## Airtel gateway validation
|
||||
|
||||
``` bash
|
||||
ping 192.168.1.1
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
``` text
|
||||
2 packets transmitted
|
||||
2 received
|
||||
0% packet loss
|
||||
|
||||
min/avg/max/mdev:
|
||||
98.975 / 99.509 / 100.043 / 0.534 ms
|
||||
```
|
||||
|
||||
## LAN server validation
|
||||
|
||||
``` bash
|
||||
ping 192.168.1.200
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
``` text
|
||||
3 packets transmitted
|
||||
3 received
|
||||
0% packet loss
|
||||
|
||||
min/avg/max/mdev:
|
||||
88.959 / 95.690 / 105.145 / 6.882 ms
|
||||
```
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 7. LAN Host Discovery
|
||||
|
||||
The routed LAN was scanned using:
|
||||
|
||||
``` bash
|
||||
nmap -sP 192.168.1.0/24
|
||||
```
|
||||
|
||||
The following 17 hosts were discovered:
|
||||
|
||||
``` text
|
||||
192.168.1.1
|
||||
192.168.1.3
|
||||
192.168.1.4
|
||||
192.168.1.5
|
||||
192.168.1.6
|
||||
192.168.1.7
|
||||
192.168.1.8
|
||||
192.168.1.9
|
||||
192.168.1.10
|
||||
192.168.1.13
|
||||
192.168.1.18
|
||||
192.168.1.20
|
||||
192.168.1.60
|
||||
192.168.1.64
|
||||
192.168.1.100
|
||||
192.168.1.152
|
||||
192.168.1.200
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
17 hosts up
|
||||
```
|
||||
|
||||
This is significant because it validates routed access to the LAN as a
|
||||
network segment rather than connectivity to only one predefined host.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 8. Application-Level Validation
|
||||
|
||||
## HTTP
|
||||
|
||||
The NX9-WG client successfully accessed:
|
||||
|
||||
``` text
|
||||
http://192.168.1.200:8008
|
||||
```
|
||||
|
||||
The Thakares IoT Hub web application loaded successfully.
|
||||
|
||||
## Network filesystem
|
||||
|
||||
The Linux desktop successfully accessed the remote Debian server:
|
||||
|
||||
``` text
|
||||
192.168.1.200
|
||||
/home/sunil
|
||||
```
|
||||
|
||||
## SSH
|
||||
|
||||
The strongest application-level validation was:
|
||||
|
||||
``` bash
|
||||
ssh 192.168.1.200
|
||||
```
|
||||
|
||||
which produced a normal interactive login:
|
||||
|
||||
``` text
|
||||
Linux thakares 6.12.101+deb13-rt-amd64
|
||||
Debian GNU/Linux
|
||||
x86_64
|
||||
```
|
||||
|
||||
The remote shell was successfully entered and exited normally.
|
||||
|
||||
This confirms:
|
||||
|
||||
- TCP connectivity;
|
||||
- routing;
|
||||
- return-path routing;
|
||||
- firewall/forwarding compatibility;
|
||||
- SSH service accessibility;
|
||||
- stable encrypted transport;
|
||||
- real bidirectional application traffic.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 9. Test Results
|
||||
|
||||
Validation Result
|
||||
------------------------------------ --------
|
||||
WireGuard tunnel establishment PASS
|
||||
External Internet through tunnel PASS
|
||||
Cellular 5G underlay PASS
|
||||
Wi-Fi underlay PASS
|
||||
Wi-Fi STA + AP concurrent underlay PASS
|
||||
Protected LAN reachability PASS
|
||||
Airtel gateway `192.168.1.1` PASS
|
||||
LAN server `192.168.1.200` PASS
|
||||
ICMP PASS
|
||||
LAN host discovery PASS
|
||||
HTTP application PASS
|
||||
Network filesystem access PASS
|
||||
SSH PASS
|
||||
Interactive remote shell PASS
|
||||
Multiple LAN hosts PASS
|
||||
No cellular dependency PASS
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 10. Key Finding
|
||||
|
||||
The same NX9-WG peer:
|
||||
|
||||
``` text
|
||||
10.100.0.6/32
|
||||
```
|
||||
|
||||
successfully accessed:
|
||||
|
||||
``` text
|
||||
192.168.1.0/24
|
||||
```
|
||||
|
||||
while its underlying client network changed.
|
||||
|
||||
Examples included:
|
||||
|
||||
``` text
|
||||
10.137.106.0/24
|
||||
10.24.3.0/24
|
||||
```
|
||||
|
||||
This demonstrates a clean separation between:
|
||||
|
||||
- **underlay:** whatever network currently provides Internet
|
||||
connectivity;
|
||||
- **overlay:** the authenticated NX9-WG tunnel;
|
||||
- **protected network:** the routed `192.168.1.0/24` LAN.
|
||||
|
||||
The protected LAN required no modification for these tests.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 11. Why This Is a Meaningful Stress Test
|
||||
|
||||
This test does not attempt to measure maximum WireGuard throughput.
|
||||
|
||||
Instead, it stresses the **network topology and routing assumptions** of
|
||||
NX9-WG.
|
||||
|
||||
The tested paths introduce:
|
||||
|
||||
- different client networks;
|
||||
- different NAT environments;
|
||||
- mobile AP routing;
|
||||
- Wi-Fi STA/AP concurrency;
|
||||
- additional network hops;
|
||||
- remote access to an entire private subnet;
|
||||
- multiple simultaneous LAN destinations;
|
||||
- multiple application protocols.
|
||||
|
||||
The successful results indicate that NX9-WG is not dependent on a
|
||||
particular client access network.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 12. Airtel LAN Collision Use Case
|
||||
|
||||
The test originated from a practical problem involving an Airtel
|
||||
AirFiber Nokia AAP321NK using:
|
||||
|
||||
``` text
|
||||
192.168.1.1
|
||||
```
|
||||
|
||||
which overlaps with the existing home LAN addressing.
|
||||
|
||||
Instead of modifying the ISP-controlled router configuration, NX9-WG
|
||||
provided an independent routed management path:
|
||||
|
||||
``` text
|
||||
External network
|
||||
│
|
||||
▼
|
||||
NX9-WG
|
||||
│
|
||||
▼
|
||||
192.168.1.0/24
|
||||
│
|
||||
├── 192.168.1.1
|
||||
├── 192.168.1.200
|
||||
└── other LAN hosts
|
||||
```
|
||||
|
||||
This demonstrates a practical operational benefit of the NX9-WG
|
||||
architecture: remote authenticated access to infrastructure can remain
|
||||
available without requiring ISP router reconfiguration.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 13. What This Test Does Not Establish
|
||||
|
||||
This test should **not** be interpreted as a performance benchmark.
|
||||
|
||||
The following remain outside the scope of this test:
|
||||
|
||||
- maximum throughput;
|
||||
- sustained high-volume transfer;
|
||||
- CPU utilisation;
|
||||
- simultaneous high-volume traffic from many peers;
|
||||
- large-scale concurrent peer testing;
|
||||
- packet-loss recovery under severe loss;
|
||||
- roaming during an active session;
|
||||
- MTU/fragmentation testing under load;
|
||||
- IPv6 routed-LAN testing;
|
||||
- server restart/recovery testing;
|
||||
- client sleep/resume testing;
|
||||
- long-duration soak testing.
|
||||
|
||||
These should be covered by separate tests if required.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 14. Recommended Future Stress Tests
|
||||
|
||||
Future NX9-WG validation can extend this matrix with:
|
||||
|
||||
### Performance
|
||||
|
||||
``` text
|
||||
iperf3
|
||||
```
|
||||
|
||||
- TCP throughput;
|
||||
- UDP throughput;
|
||||
- bidirectional traffic;
|
||||
- sustained transfers.
|
||||
|
||||
### Reliability
|
||||
|
||||
- 1-hour soak test;
|
||||
- 24-hour soak test;
|
||||
- repeated tunnel reconnects;
|
||||
- server restart;
|
||||
- client suspend/resume.
|
||||
|
||||
### Mobility
|
||||
|
||||
- Wi-Fi → 5G;
|
||||
- 5G → Wi-Fi;
|
||||
- AP changes while tunnel is active;
|
||||
- changing NAT environments.
|
||||
|
||||
### Scale
|
||||
|
||||
- multiple simultaneous peers;
|
||||
- multiple LAN destinations;
|
||||
- concurrent HTTP/SSH/file traffic.
|
||||
|
||||
### Network edge cases
|
||||
|
||||
- MTU stress;
|
||||
- fragmentation;
|
||||
- packet loss;
|
||||
- high latency;
|
||||
- jitter;
|
||||
- IPv6.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
# 15. Final Assessment
|
||||
|
||||
**NX9-WG Remote LAN Multi-Path Integration Test: PASS**
|
||||
|
||||
The implementation successfully provided authenticated routed access
|
||||
from externally connected clients to the protected `192.168.1.0/24` LAN
|
||||
across multiple substantially different network paths.
|
||||
|
||||
The successful SSH session to `192.168.1.200`, access to the IoT Hub,
|
||||
network filesystem access, and discovery of 17 LAN hosts provide strong
|
||||
practical evidence that the VPN is functioning as a complete remote-LAN
|
||||
connectivity solution rather than merely establishing a WireGuard
|
||||
handshake.
|
||||
|
||||
------------------------------------------------------------------------
|
||||
|
||||
**Test conclusion:**
|
||||
|
||||
> NX9-WG successfully maintained functional remote access to the
|
||||
> protected LAN independently of the underlying client access network,
|
||||
> including cellular, Wi-Fi, and Wi-Fi STA/AP concurrent paths.
|
||||
|
||||
**Status: PASS**
|
||||
@@ -1,46 +0,0 @@
|
||||
# Quality Assurance & Testing Strategy
|
||||
|
||||
`nx9-wg` enforces a comprehensive, multi-tiered verification strategy designed to guarantee code correctness, memory safety, failure semantics, and secret protection.
|
||||
|
||||
---
|
||||
|
||||
## 1. Test Suite Summary & Quality Gates
|
||||
|
||||
| Tier | Test Suite / Check | Scope & Execution Target | Current Status |
|
||||
| :--- | :--- | :--- | :---: |
|
||||
| **Tier 1** | Code Formatting | `cargo fmt --all -- --check` | **PASS** (Zero diffs) |
|
||||
| **Tier 2** | Type & Borrow Check | `cargo check --workspace` | **PASS** (Zero errors) |
|
||||
| **Tier 3** | Workspace Unit Tests | `cargo test --workspace` | **PASS** (**91 / 91 passed**) |
|
||||
| **Tier 4** | Clippy Linter Check | `cargo clippy --workspace --all-targets --all-features -- -D warnings` | **PASS** (Zero warnings) |
|
||||
| **Tier 5** | Release Compilation | `cargo build --release` | **PASS** (Clean build) |
|
||||
| **Tier 6** | Comprehensive CLI Suite | `LIVE=0 bash scripts/test-cli-comprehensive.sh` | **PASS** (**203 passed** / 7 skipped) |
|
||||
| **Tier 7** | Native Integration Suite | `LIVE=0 bash scripts/test-native-integration.sh` | **PASS** (**19 passed** / 1 skipped) |
|
||||
| **Tier 8** | Dedicated Live Kernel Suite | `LIVE=0 bash scripts/test-live-kernel.sh` | **PASS** (**23 passed** / 1 skipped) |
|
||||
| **Tier 9** | Subprocess Safety Audit | Automated source scan for `Command::new` | **PASS** (Zero subprocesses) |
|
||||
| **Tier 10** | Secret Leakage Audit | Automated scan for plaintext credentials | **PASS** (Zero secrets leaked) |
|
||||
| **Tier 11** | Release Package Check | Standalone archive extraction & verification | **PASS** (Independent execution) |
|
||||
|
||||
---
|
||||
|
||||
## 2. SAFE Mode (`LIVE=0`) vs Real-Kernel Mode (`LIVE=1`)
|
||||
|
||||
To guarantee safety when developing on unprivileged developer workstations:
|
||||
|
||||
### SAFE Mode (`LIVE=0` — Default)
|
||||
- Uses real in-memory SQLite stores and dry-run Netlink message builders.
|
||||
- Validates CLI parsers, JSON/YAML/CSV output formatters, route equality rules, and read-only reconciliation planning.
|
||||
- Automatically skips live kernel mutation steps that require root or `CAP_NET_ADMIN`.
|
||||
|
||||
### Real-Kernel Mode (`LIVE=1` — Dedicated Host Only)
|
||||
- Requires `root` or `CAP_NET_ADMIN` in a dedicated, disposable Linux VM.
|
||||
- Creates real kernel WireGuard interfaces (e.g. `nx9t...`), attaches IPv4/IPv6 addresses, installs routes in the kernel routing table, configures `table inet nx9_wg` in Netfilter, and validates live handshake telemetry.
|
||||
|
||||
---
|
||||
|
||||
## 3. Automated Subprocess & Secret Audits
|
||||
|
||||
Every verification run executes strict source-level security audits:
|
||||
|
||||
1. **Subprocess Audit**: Confirms zero instances of `std::process::Command`, `tokio::process::Command`, `Command::new`, or shell scripts in production Rust crates.
|
||||
2. **Secret Redaction Audit**: Confirms that password hashes, private keys, preshared keys, and token hashes are never printed in human-readable status outputs or logs.
|
||||
3. **Environment Audit**: Confirms that all recognized environment variables strictly observe the `NX9_WG_*` namespace.
|
||||
|
After Width: | Height: | Size: 11 MiB |
|
After Width: | Height: | Size: 372 KiB |
|
After Width: | Height: | Size: 452 KiB |
|
After Width: | Height: | Size: 521 KiB |
|
After Width: | Height: | Size: 495 KiB |
|
After Width: | Height: | Size: 331 KiB |
|
After Width: | Height: | Size: 333 KiB |
|
After Width: | Height: | Size: 396 KiB |
|
After Width: | Height: | Size: 332 KiB |
|
After Width: | Height: | Size: 275 KiB |
|
After Width: | Height: | Size: 321 KiB |
|
After Width: | Height: | Size: 498 KiB |
|
After Width: | Height: | Size: 299 KiB |
|
After Width: | Height: | Size: 403 KiB |
|
After Width: | Height: | Size: 491 KiB |
@@ -2614,8 +2614,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
}
|
||||
FirewallSubcommands::Sync => {
|
||||
let rules = store.list_firewall_rules().await?;
|
||||
let ifaces = store.list_interfaces().await?;
|
||||
let subnets: Vec<_> = ifaces.into_iter().map(|i| i.address_v4).collect();
|
||||
let subnets = nx9_wg_api::collect_managed_wg_subnets(&store).await?;
|
||||
let net = NativeLinuxNetworkEngine::new();
|
||||
net.sync_firewall(&rules, true, &subnets).await?;
|
||||
println!("Firewall ruleset synchronized successfully.");
|
||||
@@ -2639,10 +2638,10 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
|
||||
match args.subcommand {
|
||||
NatSubcommands::Status => {
|
||||
let ifaces = store.list_interfaces().await?;
|
||||
let subnets: Vec<String> = ifaces
|
||||
let subnets: Vec<String> = nx9_wg_api::collect_managed_wg_subnets(&store)
|
||||
.await?
|
||||
.into_iter()
|
||||
.map(|i| i.address_v4.to_string())
|
||||
.map(|s| s.to_string())
|
||||
.collect();
|
||||
let status = serde_json::json!({
|
||||
"nat_masquerade_enabled": true,
|
||||
@@ -2660,17 +2659,16 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
println!("NAT masquerade disabled in settings.");
|
||||
}
|
||||
NatSubcommands::List => {
|
||||
let ifaces = store.list_interfaces().await?;
|
||||
let subnets: Vec<String> = ifaces
|
||||
let subnets: Vec<String> = nx9_wg_api::collect_managed_wg_subnets(&store)
|
||||
.await?
|
||||
.into_iter()
|
||||
.map(|i| i.address_v4.to_string())
|
||||
.map(|s| s.to_string())
|
||||
.collect();
|
||||
print_output(&subnets, format)?;
|
||||
}
|
||||
NatSubcommands::Sync => {
|
||||
let rules = store.list_firewall_rules().await?;
|
||||
let ifaces = store.list_interfaces().await?;
|
||||
let subnets: Vec<_> = ifaces.into_iter().map(|i| i.address_v4).collect();
|
||||
let subnets = nx9_wg_api::collect_managed_wg_subnets(&store).await?;
|
||||
let net = NativeLinuxNetworkEngine::new();
|
||||
net.sync_firewall(&rules, true, &subnets).await?;
|
||||
println!("NAT masquerade rules synchronized with nftables.");
|
||||
|
||||