From pseudo-precise literary features to an auditable AI data pipeline
In June 2025, I began building a private AI writing workspace with an algorithm engineer and a content editor. I later rebuilt the exposed rights, label-truth, evaluation-isolation, and version-governance problems as this 100% synthetic, publicly auditable data pipeline.
Planning began in June 2025 and development started in July. The workspace connected ideation, source material, outlines, drafting, revision, and review. The algorithm engineer owned model calls and workflows, the editor owned content rules and final acceptance, and I owned ingestion, cleaning, databases, vector retrieval, and later the data audit and governance rebuild.
The three-person project exposed real issues in rights traceability, semantic validation, editorial bottlenecks, retry recovery, version management, and model replacement. Its code, manuscripts, logs, prompts, contracts, and operating records remain private.
I independently rebuilt the pre-training governance layer as a public demo. All public data is 100% synthetic, and every reproducible metric comes from a pinned release rather than the private project's scale or business results.
The private project explains the problem chain; the public demo supplies a verifiable implementation chain. They appear together for context while their assets remain strictly separated.
Comparing the early field specification, database tables, and analysis modules revealed three conflicting definitions of the supposed “12-dimensional literary features.”
Hooks, reversals, and emotional progression relied mainly on keywords, lexicons, and templates. These detect surface signals but cannot reliably judge irony, dry humor, or language whose literal meaning conflicts with intent.
Similar model services participated in both generation and evaluation without independent gold labels, blind review, or calibration. Some model scores were stored but never used in the final gate, which still relied mostly on surface-format rules.
Every format check passed, yet the literary signals had no meaningful variation.
“All passed” in the original report referred only to sampled format, range, and enum checks—not full semantic validation.
NoneThese aggregates come from 32 synthetic AI experiment records in a private early prototype. The source material remains private evidence and is not reproduced in the public repository; it is neither the public demo’s pinned regression data nor a claim of business scale.
A three-person team began building a private fiction-writing workbench. I owned collection, cleaning, databases, and vector retrieval; the algorithm engineer owned model calls and workflows; the editor defined content rules and acceptance criteria.
Streamlit, LangGraph, Qwen, Chroma, and SQLite connected ideation, source material, outlines, prose, and scoring. At this stage, “can generate” was still being mistaken for “training-ready data.”
Provenance and permitted use could not be proved work by work. I isolated the legacy corpus as read-only experimental data and rebuilt ledgers for author, source, license scope, withdrawal path, and version.
Editorial review found highly homogeneous samples. I traced the issue across source material, databases, and the analysis agent, confirming drifting field definitions, collapsed feature distributions, and insufficient separation between generation and evaluation.
Batch generation exposed editorial bottlenecks, rate-limit retries, checkpoint recovery, and vector-version drift. Blind comparisons reinforced that agent orchestration should remain replaceable and evaluation must be pinned and independent.
When the commercial hypothesis missed its stop criteria, we stopped expanding. I documented ownership, versions, secrets, indexes, dependencies, and handoff boundaries, preserving the reusable value in workflows, governance standards, and evaluation assets.
I independently rebuilt the pre-training governance layer as a public repository using 100% synthetic data, a pinned release, two-version CI, reproducible builds, and auditable governance documents.
Instead of adding more literary terminology to rules, I separated deterministic gates, candidate retrieval, semantic pre-screening, and final editorial judgment. The public repository implements the pre-training data-engineering layer.
Rights admission, schema, format, length, hashes, versions, and file integrity.
Similar characters, conflicts, historical material, prior outlines, and editorial experience.
Ideation, candidate generation, style rewrites, editorial suggestions, and evidence-backed semantic pre-screening.
Gold sets, blind review, final semantic judgment, and continuous calibration from editing diffs.
Text, author, work, rights fields, and version identifiers.
Invalid status, purpose, or term sends a record to the rejection path.
Unicode, whitespace, punctuation, and deterministic hash input.
Detect exact duplicates in normalized text.
Find near-duplicate candidates without treating similarity as a copyright judgment.
Group by author first, keep each work intact, then derive datasets.
All counts are pinned to v0.1.0-demo to verify rights gating, deduplication, splitting, dataset derivation, and file reconciliation. They are not business-scale metrics.
These choices keep unprovable judgments out of deterministic gates.
An author’s style, preferences, and narrative habits can recur across works. Random chapter- or sample-level splits may expose validation data to stable author patterns from training. The pipeline freezes author groups, keeps each work intact, and only then derives SFT, preference, and evaluation tasks.
SHA-256 is deterministic, fast, and traceable for identical normalized text. MinHash/LSH surfaces candidates with light rewrites or local changes. The former cannot detect semantic similarity; the latter cannot prove copyright safety. They serve different responsibilities.
When similar models generate and review the same material, they can share blind spots. Model scores are useful for pre-screening, explanations, and evidence—not as a replacement for independent gold labels, real editorial blind review, and calibrated release thresholds.
Rules handle deterministic constraints, retrieval brings back historical evidence, AI proposes candidates and semantic suggestions, and editors own final content judgment. The layers make failure modes, validation methods, and accountability clearer.
PRIVATE PROJECT · REAL CONTEXT · NOT IN PUBLIC REPO These roles and workflows reflect real collaboration in the private project. Automation varied by stage, while final content always required editorial acceptance. The public repository implements only the pre-training data-governance layer.
The governance evidence below is pinned to release commit b19b3b5…; CI links point to the corresponding Actions run.
Synthetic Portfolio Build and pinned commit snapshot.
View release → CIBoth jobs passed the lightweight public release gate, deterministic rebuild, tests, CLI, compilation, and artifact diff.
View Actions → GOVERNANCEData sources, intended use, split strategy, limitations, and authenticity boundaries.
View document → VERIFICATIONCounts, tests, deduplication, isolation, and validation for the pinned synthetic build.
View report → MANIFESTDataset versions, stage artifacts, source snapshot, and release lineage.
View manifest → SPLIT CHECKAuthor, work, and text-hash overlap checks for the pinned synthetic splits.
View report → CHECKSUMSPinned checksums for governed synthetic build files.
View checksums → ARCHITECTUREPublic data-engineering scope, stage boundaries, and unimplemented areas.
View architecture → LINEAGEImplemented stages, dataset and evaluation versions, plus placeholder links for unexecuted training and models; the production withdrawal path remains target design.
View lineage → OWNERSHIPIndependent implementation, AI-assisted development, and final accountability boundaries.
View statement → PROVENANCEHow synthetic data was generated and the boundary excluding third-party project assets.
View statement → PUBLIC SAFETYSensitive assets prohibited from the public repository and the manual-review boundary.
View policy →python scripts/run_demo.py --output-dir artifacts
python -m unittest discover -s tests -v
python scripts/check_public_repo.pyThis section describes only the pinned public implementation. It neither dismisses the real private-project experience nor presents private assets as publicly reproducible evidence.
This project does not predict “hits” and did not train a fiction model. It demonstrates how I identify falsely precise labels, establish data contracts, and turn rights, deduplication, isolation, versioning, and release validation into an executable pipeline. Literary judgment remains the responsibility of real editorial evaluation and human review.