Minimize at source
Remove tokens, keys and unnecessary sensitive data before sealing; do not edit sealed content to hide source mistakes.
Preserve closed log sets with source-event links and missing/duplicate checks before deposit.
Preserve closed log windows with checks for missing, duplicated or misattributed source records.
| Prepare | Receive |
|---|---|
| Minimized transaction, API-gateway or business-system logs. | An immutable batch with time range, inventory, count and evidence reference. |
Participants: SOC, source-system administrators and auditors.
Remove tokens, keys and unnecessary sensitive data before sealing; do not edit sealed content to hide source mistakes.
Retain start/end times, time zone, source, sequence range and row count; distinguish late arrivals.
Check sequence gaps, duplicate batches and file digests. The design uses Kafka for events/references, not implicitly for entire logs.
Link source batch ID to archiveId and evidence. Corrective batches use new IDs linked to predecessors.
| Current | Guard | Actor | Next | Artifact / recovery |
|---|---|---|---|---|
| Open window [model] | Freeze source, sequence and time | SOC / source system | Closed batch [model] | Retain sequence range/count; source time is not inherently trusted. |
| Closed batch [model] | Complete inventory and matching digests | Batch intake | Accepted / quarantined [model] | Sequence gaps differ from duplicates; retain distinct reasons. |
| Preserved [model] | Late data or source correction | Log owner | Supplemental batch [model] | New batch links the old; sealed bytes are never overwritten. |
| Situation | Action | Boundary |
|---|---|---|
| Sequence gap | Quarantine for source reconciliation. | A digest does not establish completeness. |
| Late arrival | Create a linked supplemental batch. | Do not overwrite an evidenced batch. |
| Source clock skew | Retain source and intake times separately. | Do not treat source time as trusted time. |
Seal before hashing; retain time zone and source order. Kafka carries events and references, not binary archive payloads by default.
Quarantine missing or duplicate batches for source reconciliation. Issue a corrected batch with a new ID linked to the previous one.
Reconcile source sequence, time zone, window, row count and batch ID; corrected batches use new IDs linked to predecessors.
| API | Application scope | Documentation |
|---|---|---|
| POST /v1/archives | archives:write | Archive API |
| GET /v1/archives | archives:read | Archive API |
GET /v1/archives/{archiveId} | archives:read | Archive API |
GET /v1/archives/{archiveId}/status | archives:read | Archive API |
| Element | Definition |
|---|---|
| Input | Closed log batch, time range, source, record count, manifest and sensitive-data minimization policy. |
| Output | Evidence for an immutable batch with source-event to archiveId traceability; an open log stream is not a finalized batch. |
| Element | Action |
|---|---|
| Synthetic ID | log-window-test-001 |
| Task | Close window/source sequence range; record timezone, checkpoint and batch members. |
| Acceptance case | Detect sequence gaps/duplicate batches even when remaining files hash correctly. |
These are business test criteria; synthetic packages alone do not establish backend execution or valid ERS.