Skip to content

OUTCROP — Build Brief v0.1 (overnight run)

Working codename: Outcrop. Rename later; nothing keys off the name.

What this is

A DBMS whose storage layer is a folder of markdown files. Record = one .md file with YAML frontmatter. Table = folder. The DBMS surface (SQL + graph + full-text) is the primary interaction mode; markdown is the durable, human-readable, git-diffable substrate. OKF-conformant bundles in and out. Git is the transaction and safety layer. Scenarios (branches) are user-facing. Canvas UI later doubles as dashboard builder and graph explorer.

Not competing with Obsidian. Riding the OKF standard, not inventing a rival file convention.

Non-negotiable design decisions (already made — do not relitigate)

  1. Writes from day one. Read-only index fails the "is it a DBMS if you can't UPDATE" test.
  2. Constraint tools, not schema enforcement. Per-folder policy: ignore | warn | block, applied at write time and at merge time. An externally edited file that violates constraints is a valid file and a flagged record — never corruption.
  3. Git = WAL. Stage = open transaction. Commit = commit. Rollback = discard working set. Crash recovery = git status. Every transaction is a readable diff.
  4. Lossless round-trip. An UPDATE touching one frontmatter field leaves every other byte identical. CST-level editing (rowan or tree-sitter based), never parse-and-reserialize.
  5. External writers are first-class. Obsidian, git pull, agents, Finder. Watcher detects, incrementally reindexes, treats conflicting external edits as merge conflicts against snapshot reads — never lockout, never corruption.
  6. Isolation honesty. Snapshot reads only. We do not pretend to serializable over a filesystem shared with humans. Document it.
  7. OKF conformance. Only required frontmatter field is type. resource field is the future data-virtualization hook (federation to warehouses/APIs). Tolerate unknown keys. Target OKF v0.2 (provenance/trust fields pass through untouched).
  8. User-facing vocabulary. Never surface git terms. Scenario (branch), Snapshot (pinned commit), Propose changes (merge request), Keep mine / Keep theirs / Keep both (resolution), History (log), Restore (revert). Conflict UI is record-scoped and field-level — never <<<<<<< markers.
  9. No plugin system. No custom connectors. Defer both. Federation will ride ADBC / Arrow Flight SQL later — tonight, only keep the TableProvider abstraction clean so it slots in.

Architecture

Rust headless core (the product)

Cargo workspace:

  • outcrop-store — file model, CST round-trip (YAML frontmatter + markdown body), atomic write path (temp → fsync → rename), file watcher, incremental index.
  • outcrop-schema — per-folder schema files (_schema.yaml inside each folder): field types, required fields, enum constraints, link-target rules. Validation engine returning structured diagnostics. Policy levels ignore/warn/block.
  • outcrop-txn — git-backed transactions (use gitoxide preferred, git2 fallback). Working-set staging, commit, rollback, scenario ops (branch via worktree), diff, merge with post-merge validation pass producing a "needs attention" list. Per-folder programmatic merge policies (last-write-wins | manual | custom-later); field-level frontmatter merge (different keys auto-merge, same key conflicts).
  • outcrop-query — Apache DataFusion. One TableProvider per folder; frontmatter fields → columns, body H2 sections → text columns, plus computed columns (_path, _links_out, _links_in, _modified). Tantivy FTS exposed as table function: SELECT * FROM search('query'). Graph ops as table functions: neighbors(record, depth), links_to(record), paths(a, b, max_depth) over the wikilink/markdown-link graph.
  • outcrop-server — gRPC (tonic) for native clients + MCP server (stdio and SSE) exposing tools: query, get_record, upsert_record, delete_record, validate, create_scenario, list_scenarios, diff_scenario, propose_merge, resolve, snapshot.
  • outcrop-cli — thin cover over the server crate for scripting and CI.

Native clients (contract-first)

Shared contract: single .proto file is the source of truth; generate Swift, Kotlin, C# clients from it.

  • Apple: SwiftUI, one codebase macOS + iOS. Tonight: macOS only.
  • Android: Kotlin + Jetpack Compose. Tonight: stub project + generated client + one smoke test. No UI.
  • Windows: WinUI 3 / Windows App SDK (first-party Microsoft). Tonight: stub project + generated client. No UI.

Canvas (thin slice tonight)

macOS SwiftUI: infinite pannable/zoomable canvas. Two card types: (1) query card — SQL box, results as a table, draggable; (2) graph card — pick a record, render its link neighborhood to depth 2, click-to-expand. Scenario picker in the toolbar re-renders all cards against the selected scenario. That is the whole demo: same canvas, two scenarios, side-by-side data states. No styling polish, no dashboard persistence beyond a JSON layout file.

Overnight execution plan

Orchestration

Top-level agent = architect/integrator only; it writes no feature code. Delegate one subagent per crate plus one for the Swift client plus one test-infrastructure subagent. Contract (.proto) and the sample corpus are frozen by the architect in the first 30 minutes; all subagents build against them in parallel.

TDD rules (hard)

  • Red-green per feature. No implementation commit without a failing test first in the same series.
  • Property tests (proptest) for round-trip: generate random frontmatter+body docs → edit one field → assert every other byte identical.
  • Crash-safety: kill the process mid-transaction in a test harness → reopen → assert clean state and git status recovery.
  • Concurrent-writer tests: external file mutation during an open transaction → assert snapshot read stability and conflict surfacing.
  • Merge tests: scenario merges producing (a) clean auto-merge on different fields, (b) field conflict, (c) constraint violation post-merge → "needs attention" list.
  • Coverage gate: core crates ≥ 80% line coverage before the integrator accepts a merge.
  • Sample corpus: generate an OKF v0.2 bundle (~200 records, 3 folders/"tables", cross-links, deliberate constraint violations) as a fixture; every integration test runs against it.

Milestone gates (in order; stop and stabilize rather than half-finish the next)

  1. M1 — outcrop-store + outcrop-txn: round-trip, atomic writes, git transactions, watcher reindex. CLI can upsert and rollback.
  2. M2 — outcrop-query: SQL over folders, search(), neighbors() working against the corpus.
  3. M3 — outcrop-server: gRPC + MCP up; MCP verified end-to-end from Claude Code itself (query + upsert + scenario tools).
  4. M4 — outcrop-schema + merge validation: policies enforced at write and merge; needs-attention list over diagnostics.
  5. M5 — macOS canvas thin slice; Android/Windows stubs compile.

Do-not-do list (tonight)

Plugins, third-party connectors, federation, auth, sync, Windows/Android UI, iOS target, canvas persistence beyond a layout JSON, performance tuning past "corpus feels instant," any renaming bikeshed.

Morning deliverables

  • Repo with green CI (GitHub Actions: fmt, clippy -D warnings, test, coverage).
  • RUNLOG.md — decisions made, deviations from this brief, open questions.
  • DEMO.md — exact commands to reproduce: seed corpus → query → edit in Obsidian → watch reindex → create scenario → diverge → propose merge → resolve in CLI → open macOS canvas → flip scenarios.

Repo bootstrap

gh repo create outcrop --private --clone
cd outcrop
# drop this file in as BRIEF.md, seed CLAUDE.md pointing at it, start the run