$ cd ../projects

~/projects/ncct-reports/README.md

LIVEINTERNAL# AI Automation# Broadcast

NCCT Compliance Reports Generator

A free-to-air TV network has to file monthly reports with the National Council for Children's Television (NCCT): a Daily Programming Guide and a daily children's-content compliance report. Both were retyped by hand from weekly program grids every month. This system takes the grids through a simple upload form and emails back finished Excel and PDF reports, with compliance percentages computed and data problems flagged.

Pythonopenpyxln8nGotenbergDockerSMTP
automated tests
74
titles in show catalog
222
2026 months regenerated
9/9
replaces manual retyping
1 form

cat architecture.flow

System Architecture

architecture.flow

Request

n8n form (month + grids)
Basic Auth
Base64 encode
POST /run (token)

Engine

Grid parser
Schedule builder
Catalog lookup
DPG + MCR fillers
Compliance %

Delivery

Gotenberg → PDF
n8n rebuild files
Email xlsx + PDF
Warnings + unknown titles

cat problem.md

Business Problem

Every month, operators rebuilt two regulator reports by hand from the network's weekly program grids. The Daily Programming Guide (DPG) lists every show by day. The daily compliance report (MCR) computes how much airtime went to qualifying children's programming.

The source grids are made by people, so they contain typos, mistyped durations, inconsistent AM/PM markers, shows that run past midnight, and different layouts from week to week. Retyping them meant errors carried straight into a regulatory filing.

cat solution.md

Solution

I built a Python engine that parses the weekly grids, reconstructs the full month's schedule, looks up each show in a catalog for format, genre, rating and children's-content flags, and fills the official DPG and MCR Excel templates exactly as the regulator expects.

Operators use an n8n form: pick the month, enter a recipient, and upload the grid workbooks. The engine runs as an internal HTTP service. Gotenberg converts the workbooks to PDF, and n8n emails back the xlsx and PDF files with the compliance percentage per report, every data warning, and any titles missing from the catalog.

The show catalog was seeded from 313 historical report tabs, so the engine starts from the network's real history instead of an empty list.

ls ./components

Architecture Decisions

Internal HTTP service

The engine runs in its own container on the shared Docker network with no published ports, a token header and a memory limit. Only n8n can reach it.

Modular engine

Separate modules for grid parsing, schedule building, layout, the DPG and MCR fillers, catalog lookup and time parsing. Each is small enough to test on its own.

Template-faithful output

Reports are written into the regulator's own Excel templates, not regenerated look-alikes. Styles are re-applied per row so the output matches the official format.

Catalog as data

Show metadata lives in a mounted CSV, so operators can add or correct titles without a code change or redeploy.

Gotenberg for PDF

LibreOffice-based conversion through the Gotenberg service that is already shared by other apps on the homelab.

Error alerting

The workflow is wired to the shared error-alert workflow, so a failed run notifies me instead of failing silently.

./run --trace

Monthly Report Run

  1. 01

    Upload

    The operator opens the form and enters the month, a recipient email, and the weekly grid workbooks.

  2. 02

    Select & parse

    The service picks the visible tabs that cover the month and parses each show block, normalising times and durations.

  3. 03

    Group days

    Days with identical schedules are grouped into report tabs, so a month produces as few tabs as the data allows.

  4. 04

    Resolve & compute

    Titles are resolved against the catalog, and children's-content compliance is computed per tab.

  5. 05

    Render

    The DPG and MCR templates are filled and converted to PDF with Gotenberg.

  6. 06

    Deliver

    n8n emails all xlsx and PDF files with the compliance percentages, warnings, and any unknown titles.

tail -f incidents.log

Engineering Challenges

The engine wouldn't fit through SSH

Symptom
Passing a grid workbook to the engine over SSH failed for real files.
Root cause
n8n's SSH credentials are forced-command keys, and Linux caps a single command-line argument at 128 KB. A ~100 KB workbook is ~136 KB in base64.
Fix
Moved the engine behind a small internal HTTP service on the Docker network with a token header, and kept the forced-command keys locked down.

Dirty source data

Symptom
A generated compliance figure came out at 16.90% where the operator's own file said 14.49%.
Root cause
A 60-minute show had been mistyped in the source grid.
Fix
The engine trusts row position over typed AM/PM, trims overlaps, ignores post-midnight spill-over, and reports every correction as a warning. Its formula reproduced the operator's correct days exactly, which proved the discrepancy was in the source data.

A template the library rejected

Symptom
openpyxl refused to open the official MCR template.
Root cause
The template's styles contain an out-of-range font-family value.
Fix
Clamp the value in memory at load time without modifying the official template, and re-apply show styles per row because the original styles were tied to the original layout.

Layouts that changed under real data

Symptom
Weeks with more than three day-blocks, and workbooks with leftover hidden tabs, broke the run.
Fix
Added a fourth layout block, split such weeks across multiple tabs, skipped hidden tabs and listed unexpected tabs as warnings. All nine 2026 months were then regenerated successfully as a stress test.

cat package.json | jq .stack

Technology Stack

Engine

Python 3openpyxlCSV catalogAutomated tests

Orchestration

n8n Form triggern8n Code nodesSMTPError-alert workflow

Platform

Docker ComposeGotenbergShared Docker networkClaude Code + n8n MCP

echo $EXIT_STATUS

Project Outcome

outcome.log

A monthly regulatory filing that used to be retyped by hand is now one form submission, and the result is in the operator's inbox as finished Excel and PDF reports.

Just as important, the system catches mistakes in the source grids. Every correction and unknown title is reported back, so the filing is more accurate than the manual process it replaced.