>_ pipeline teardown
A Polish-language AI news site that writes, checks, labels and publishes itself - with a human holding a sixty-minute veto.
This is the honest version of the architecture, written from the running system rather than from the diagram I would have drawn beforehand. The interesting part is not that a model writes the articles - that is the easy half. It is everything built around the model to stop it repeating itself, to stop it publishing on a quiet news day, to keep a human able to say no, and to make the site tell the truth about which of its articles a person actually read.
1,178
articles published
10 min
scheduler cadence
60 min
human review window
0
databases
The load-bearing decision in this system is where the logic is allowed to live. Everything that can be diffed, tested and rolled back sits in files on a server. The workflow tool holds a schedule and one SSH command.
Scheduling and human-in-the-loop only
Eighteen workflows, but the news ones do exactly two things: fire a schedule trigger and wait for a human. The main workflow is four nodes - schedule trigger, SSH, a Code node to parse, an HTTP request to fan out. It executes no Python, transforms no articles, and holds no business logic. Pull n8n out and the pipeline still works from cron; you just lose the approval emails.
Every decision that matters
Bash owns orchestration and process control - locking, retries, exit-code routing, git. Python owns the parts where bash gets dangerous: parsing feeds, diffing state cursors, rewriting nested JSON, rendering an event map. Twelve Python scripts, roughly twenty shell scripts. The split is by risk, not by preference.
Writes and judges the articles
Invoked non-interactively with a JSON schema pinned to the output, WebFetch and WebSearch as the only permitted tools, and a token pool that fails over between accounts when one hits a usage limit. Two separate calls per article: one to write it, one to judge whether it duplicates something already covered.
Delivery
Content lives as JSON files in the site repo - stories.json, bodies.json, event-map.json. Publishing is a git commit and push; Vercel builds from the push. There is no CMS, no database, and no admin panel. The version history of the news site is its git log.
Seven stages from a schedule tick to a deployed page. Four of them can end the run early, and that is the point - most ticks should produce nothing.
Every ten minutes an n8n schedule trigger opens an SSH connection to the VPS and runs a single command. A run can hold Claude for two to eight minutes, so ticks overlap by design. The entry script takes an exclusive lock and, if a previous run still holds it, prints an empty array and exits clean - a skipped tick has to look like a quiet news cycle, not a failure, or n8n starts sending error mail.
run-window.sh · flock -n out/run.lock
Before spending a single token, a Python gate pulls the news APIs and RSS feeds using a stored cursor, then diffs against a seen-links state file. Three exit codes drive the run: new items found, so proceed with a priority list; sources fine but nothing new, so print an empty array and skip Claude entirely; error, so fall open to the raw headline prompt. Skipping the model on a quiet cycle is where most of the cost saving lives.
ingest-gate.py · exit 0 / 10 / 20
The prompt is assembled from a versioned prompt file, the priority news list, the feed list, and the newest 400 published titles as an explicit do-not-repeat block. Claude runs headless with a JSON schema attached, so the output is structurally valid or the call failed - there is no parsing of prose. It returns zero to two articles. Zero is a normal outcome.
generate-article.sh → run-claude.sh
A second model call, this one with web tools disabled, scores the draft against a rolling fourteen-day event map: is this a genuinely new real-world event, a follow-up with a materially new fact, or a rehash of something already covered? It shipped in shadow mode - always logging a verdict, changing nothing - until the verdicts were trustworthy enough to enforce. It fails open by design: any error yields "new" and the article proceeds.
judge-article.sh · eventmap.py
The draft is POSTed to an n8n webhook, which reads the file back over SSH, renders an approval email through Resend, and waits. The editor approves, rejects, or requests a revision. Approval writes a provenance record and publishes. Silence for sixty minutes is also a decision - see the next step.
NPapproveFlow01 · review_apply.py
An article nobody touched within sixty minutes plus a fifteen-minute grace period publishes itself, labeled as unreviewed rather than editor-approved. The grace period exists so n8n's own timeout branch acts first and the cron sweep stays a backstop rather than a competitor. Anything still unpublished after forty-eight hours is stale news and gets parked, not published.
auto-publish-sweep.sh · 60 + 15 min, 48h cap
Rebase onto origin, rotate the article into the content JSON, attach a licensed stock photo if the generator produced none, fold the event into the event map so the judge keeps cross-day memory, then commit and push with one retry on conflict. Vercel builds from the push. If any step fails, the article stays in the queue and the next run picks it up.
publish-to-site.sh · rotate_content.py
Getting a model to write a news article is a weekend. Getting it to run unattended for months without publishing the same story twice, without stalling silently, and without lying about who reviewed it - that is the actual work.
One in the prompt (400 recent titles injected as a negative constraint), one deterministic after generation (exact slug match plus a title-prefix probe against live content), one semantic (the judge, against a fourteen-day event map). They catch different failure modes: the prompt layer is cheap and leaky, the string probe is exact and blind to rewording, the judge understands that two headlines describe the same event.
The judge, the stock-photo picker, and the event-map writer all return success on error. A broken guardrail must not silently stop a news site from publishing. The inverse - a bad article slipping through - is recoverable in one commit. A pipeline that quietly stopped four days ago is not.
Non-blocking flock on the run, on the publish sweep, on the queue. Every lock is taken with -n and skips rather than queues. Overlapping ticks are the normal case at this cadence, so the design assumption is contention, not exclusion.
The model runner walks an ordered list of tokens and moves to the next on any failure - usage limit, auth, transport. It only reports failure when every account in the pool is exhausted. This is the difference between a rate limit costing one article and costing a day.
A failed publish can leave the site clone dirty, which then blocks the next rebase. The publish script detects a dirty tree and restores from HEAD before rotating. That is only safe because the clone holds no human work - the assumption is written into the script as a comment, because in two months nobody will remember why discarding local changes was correct.
Dropped duplicates become .dropped sidecars. Expired articles move to out/expired. Rejected drafts move to a rejected directory. Nothing in the pipeline unlinks a file. Recovery from a bad automated decision is a mv, and the audit trail survives.
EU AI Act article 50(4) requires disclosure of AI-generated text published to inform the public. A single site-wide “AI generated” banner would be easy and would also be misleading in both directions. So the label is per article, and it is derived from what the pipeline actually recorded rather than from what anyone intended.
reviewed
AI-assisted text a human editor approved through the review gate. The audit record names who reviewed it, when, and which individual fields they ticked as verified.
generated
Generated entirely by AI and auto-published because the review window lapsed with no decision. No human review is claimed. The record stores the timestamp and the window length in force at the time.
human
Written by a person.
archive
Published before the review process launched in August 2026. Absence of a label means archive, so the legacy entries need no backfill and no retroactive claim is made about them.
Behind each label sits an append-only provenance record: when the draft was generated, when and by whom it was reviewed, which individual fields were ticked as verified, every edit applied at approval time, and - for the lapse path - the auto-publish timestamp plus the review window length in force when it happened. The window length is stored rather than assumed specifically so that changing the default later cannot retroactively alter what a past label means.
The language split is by failure mode, not by taste. Shell is excellent at process control and quietly terrible at structured data; Python is the opposite. Neither is the site.
| Component | Runtime | Why |
|---|---|---|
| Orchestration, locking, git, retries | Bash | Process control and exit-code routing is what shell is actually good at |
| Feed ingest, state cursor, dedup diff | Python | HTTP plus JSON plus date math is where shell starts producing silent wrong answers |
| Content rotation into stories/bodies JSON | Python | Nested JSON rewrites with jq are write-only code |
| Event map render and upsert | Python | Needs a data structure, not a text stream |
| Article generation and judging | Claude Code (headless) | Schema-pinned output means the boundary is typed |
| Scheduling and approval email | n8n | Wait-for-human with a timeout branch is genuinely tedious to hand-roll |
| The website | Next.js 16 + React 19 + TypeScript | Static output, content read from JSON at build time |
n8n is genuinely good at waiting for a human and branching on a timeout. It is a bad place to keep logic you will need to test, diff, or roll back. Everything reviewable lives in files on disk; n8n holds a schedule and an SSH command. That boundary is the single decision I would keep in any rebuild.
The expensive mistake is calling the model 144 times a day and discarding most of the output. A cursor-driven ingest diff that exits early on a quiet cycle removes the majority of those calls before a token is spent. Post-hoc filtering is a quality control; pre-flight gating is the cost control.
Every model call is pinned to a JSON schema. There is no prose parsing anywhere in the pipeline, no regex over model output, no repair step. Either the call returned a valid article or the call failed, and both are handled the same way - try again next tick.
Most human-in-the-loop designs stall when the human is busy. This one publishes on lapse and labels the result honestly as unreviewed. The alternative - a queue that quietly grows while nobody looks at it - is worse for a news site and worse for transparency, because the label would still claim review that never happened.
No CMS, no Postgres, no admin panel. Content is JSON in a repo, publishing is a commit, deployment is a push, rollback is a revert, and the edit history is free. It stops scaling somewhere well above a thousand articles - but the ceiling is much further out than the instinct to reach for a database suggests.
A teardown that only lists the good decisions is marketing. These are the open problems, current as of August 2026.