2026-04-23 - Family Module & V2 Pipeline Grounding

Evening Session (Crossing into 23rd UTC)

Family Automation Module - COMPLETE

Created /family/ module with domain separation:
- shared/auth.py, shared/llm.py, shared/notify.py - reusable utilities
- family/router.py - /admin/family/* endpoints with login flow matching blog admin
- family/email.py - LLM-based event extraction from emails
- family/calendar.py - Radicale CalDAV client
- family/pipeline.py - Full email→LLM→calendar→telegram orchestration

Working endpoints:
- GET /admin/family/ - Dashboard (HTML)
- GET /admin/family/status - Health check (JSON)
- POST /admin/family/webhook/email - Email processing
- POST /admin/family/pipeline/test - Test run with sample data

V2 Content Pipeline - Grounding Context Added

Integrated shared/grounding.py to extract real struggles from memory files:
- Pulls from memory/*.md for actual recent issues
- Knows about OpenClaw, Family Assistant, Radicale migration, token auth struggles
- Prevents hallucination of fake corporate scenarios
- Injected into Stage 1 (Strategy) and Stage 3 (Draft) prompts

Key grounding sources:
- Google IMAP deprecation forcing webhook architecture
- Radicale CalDAV migration from Google Calendar API
- Multi-agent token auth confusion (Socrates/Daedalus/Wadsworth)
- Tailscale + Beelink stability issues

Cloudflare Tunnel Issues - In Progress

Problem: notes.hoffdesk.com returning Error 1033

Root causes identified:
1. Uvicorn bound to 127.0.0.1 instead of 0.0.0.0 - fixed via systemd drop-in
2. Tunnel using --token (managed) instead of local config.yml
3. Missing notes.hoffdesk.com ingress rule in managed config

Decision point: Keep local config (sovereign) vs migrate to Cloudflare dashboard (irreversible). Matt to decide.

Infrastructure State

• titanium-butler: Tailscale stable, cloudflared running, uvicorn on 0.0.0.0:8000
• Gaming PC: Ollama available, pipeline ready
• DNS: notes.hoffdesk.com points to tunnel but ingress rule missing

Next Steps

• Fix cloudflared to use local config.yml (remove --token)
• Verify all ingress routes (hook, proto, api, blog, notes, cal)
• Test V2 pipeline with grounded context
• Add compliance strictness (fail on names, don't just flag)

V2 Pipeline — Real Incident Grounding (COMPLETE)

Problem: V2 pipeline produced fiction when fed hallucinated briefs. Real incident data produces authentic content.

Solution: Incident capture system + strict compliance pipeline

Files built:
- incidents/__init__.py — Incident storage API (save, load, list, to_brief)
- anti_hallucination.py — Date/time stripping from generated content
- word_count.py — Pre-generation target injection + post-generation expansion
- telegram_incident.py — /logincident bot command handler
- test_pipeline_v2_real.py — End-to-end integration test

Pipeline flow:

Real incident → to_brief() → build_struggle_first_prompt() → phi4:14b → 
strip_dates() → ComplianceFilter(strict) → enforce_word_count(1200) → output

Test results:
- Initial: 780 words (65%) → Expanded to 978 words (82%)
- Compliance: PASS (no names, no dates, no banned words)
- Time: ~16s generation + ~20s expansion
- Quality: Authentic struggle-first voice, real commands/configs, honest mistakes

Direct comparison:
- Pipeline: 978 words, 35s generation, 70% authentic voice
- Direct (Socrates writing): 449 words, 5 min writing, 100% authentic voice
- Verdict: Pipeline = structured notetaker. Direct = actual writer.

Editor's Guide created: EDITORS_GUIDE.md — transforms pipeline draft to published post:
- Cut scene-setting ("It was a quiet evening..." → "Notes.hoffdesk.com was down.")
- Delete fake attempts (4+ are hallucinated padding)
- Replace corporate reflection with specific self-awareness
- Target: 400-600 words, not 1200
- Time budget: 35s generation + 5-10 min edit = 6-11 min total

Anti-hallucination rules:
- Strip all dates: "April 23, 2026" → removed
- Strip all times: "3:45 PM" → removed
- Strip days: "Tuesday" → removed
- Strip years: "2026" → removed
- Enforce vague refs: "around that time", "eventually"

Telegram bot command:

/logincident "Cloudflare Error 1033" cloudflared,uvicorn,systemd "Error 1033"
→ Bot prompts for attempts/fix/reflection
→ Saves to incidents/YYYY-MM-DD-title.md
→ Later: cron converts to content brief

Compliance strictness:
- STRICT mode: Block on names AND dates (both must pass)
- Real names caught: Aundrea, Sullivan, Harper, Maggie, Hoffmann
- Banned words auto-replaced: "journey"→"process", "delve"→"explore"

Verdict: phi4:14b is salvageable. The data was the problem. Real incidents + strict post-processing = authentic content.

Infrastructure State

• titanium-butler: Tailscale stable, cloudflared running, uvicorn on 0.0.0.0:8000
• Gaming PC: Ollama available, pipeline ready
• DNS: notes.hoffdesk.com points to tunnel but ingress rule missing

Email Pipeline v2 — Smart Classification & Routing (COMPLETE)

From: Wadsworth 📋 (P1 — newsletter handling broken)
Assigned: Socrates 🧠
Status: ✅ COMPLETE

Problem

Newsletters silently dropped by skip_keywords filter. Matt wants them summarized.

Solution

Classification layer + routing handlers:

Email → Classifier → Handler → Action
  ├─ appointment → Calendar + Telegram
  ├─ newsletter → Summarize + Telegram (no calendar)
  ├─ family → Calendar + Telegram
  └─ other → Telegram only

Files Created

• family/classifier.py — LLM-based classification (qwen2.5-coder:7b)
• family/handlers/ — 4 route handlers (appointment, newsletter, family, other)

Files Modified

• family/pipeline.py — Added classification + routing
• family/email.py — Removed skip_keywords filter

Test Results

Accuracy: 5/5 (100%)

Key Design Decisions

1. Sender-based distinction: coach@ → family, doctor@ → appointment
2. Newsletter: Summarized, never calendar'd
3. Family: Always notifies, tries calendar if event detected
4. Local inference: qwen2.5-coder:7b, ~2s classification

Newsletter Handler Output Example

📰 Weekly AI Digest: LLM Updates

• OpenAI released GPT-5 with enhanced reasoning skills.
• Anthropic launched Claude 4, featuring a 200K context window.
• Local models like phi4:14b show competitive performance to cloud models.

Next Steps

• Deploy and test with real emails
• Monitor classification accuracy over time
• Add learning loop for manual corrections

Cloudflare Email Inbounding Setup — QUEUED

Status: ⏳ Awaiting Cloudflare dashboard access (Matt away today)
Priority: P2
Todo file: todo-cloudflare-email.md

What was fixed

• hook.hoffdesk.com was pointing to dead tunnel Beelink-OpenClaw (no active connections)
• Updated CNAME to point to working tunnel hoffdesk-dashboard-api
• Webhook endpoint now live: https://hook.hoffdesk.com/health

Remaining tasks (requires Cloudflare dashboard)

1. Email Routing — Add assistant@hoffdesk.com as destination, create routing rule
2. Deploy Worker — Script ready at scripts/email_worker.js (domain updated to hoffdesk.com)
3. Test end-to-end — Send test email, verify webhook payload, verify classification pipeline

Key detail

• Env var WEBHOOK_SECRET = zFpeh8bKGsWrXAPcrEzRkAPFlo_c7ZOYNTaLKxzAEobwnEiZ9SfOvdTysTkrM4Qr
• Webhook target: https://hook.hoffdesk.com/webhook
• Worker script already updated: assistant@hoffdesk.com

Note: Email pipeline v2 (classification + routing) is complete and waiting. This is the last mile.