diff --git a/README.md b/README.md index fc781ed..9241c6a 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,70 @@ -# chirp-pwa +# Chirp — Fediverse Microblogging PWA -Chirp — Fediverse microblogging PWA (chirp.myfacefeed.com) \ No newline at end of file +A custom Progressive Web App frontend for the Fediverse, powered by GoToSocial. + +**Instance:** chirp.myfacefeed.com + +## Architecture + +``` +Browser / Mobile + ↓ HTTPS +Chirp Go Web Server (this repo) + ↓ Mastodon-compatible API +GoToSocial (ActivityPub backend) + ↓ PostgreSQL + MinIO +``` + +## Stack + +| Layer | Technology | +|-------|-----------| +| Backend | Go 1.22, GoToSocial v0.21.2 | +| Auth | Zitadel (OIDC) | +| Storage | PostgreSQL 16, Redis, MinIO | +| Search | Meilisearch | +| Frontend | Vanilla JS PWA (no framework) | +| Deploy | Nginx + systemd on 38.242.152.38 | + +## Project Layout + +``` +cmd/server/ — main entry point +internal/ + config/ — env-based config struct + gts/ — GoToSocial API client + proxy/ — API proxy (/api/v1/* passthrough) + oauth/ — Zitadel OIDC integration + db/ — chirp schema CRUD (bookmarks, lists, etc.) + ws/ — WebSocket hub (GTS stream bridge) + scheduler/ — post scheduler (Redis sorted set) + search/ — Meilisearch client +web/ + templates/ — Go HTML templates + static/ — CSS, JS, icons + sw/ — service worker +deploy/ + k8s/ — Kubernetes manifests (future) + nginx/ — nginx config +tests/ + integration/ — integration tests + e2e/ — end-to-end tests +``` + +## Design Spec + +Full technical specification: [chirp-design-spec.md](chirp-design-spec.md) + +## Infrastructure Notes + +- GoToSocial DB: PostgreSQL `gotosocial` (127.0.0.1:5432) +- Chirp schema: PostgreSQL `chirp` (8 custom tables) +- Redis: 127.0.0.1:6379 (sessions, queues, pub/sub) +- Meilisearch: 127.0.0.1:7700 (search indexes) +- MinIO: s3.myfacefeed.com / chirp-media bucket + +## Status + +Phase 1 (Infrastructure) — complete as of 2026-05-31 +Phase 2 (Go Server Core) — in progress +Phase 3+ (PWA) — assigned to Claude3 diff --git a/chirp-design-spec.md b/chirp-design-spec.md new file mode 100644 index 0000000..9f90a8f --- /dev/null +++ b/chirp-design-spec.md @@ -0,0 +1,857 @@ +# Chirp — Full Technical Design Specification +# chirp.myfacefeed.com +# Last updated: 2026-05-30 + +--- + +## 1. SYSTEM OVERVIEW — END TO END FLOWCHART + +``` +USER (browser / mobile) + │ + │ HTTPS + ▼ +┌─────────────────────────────────────────────────────┐ +│ K3s Ingress (176.9.93.253) │ +│ chirp.myfacefeed.com │ +└─────────────────────┬───────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────┐ +│ CHIRP GO WEB SERVER │ +│ (K3s pod, mff namespace) │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────┐ │ +│ │ Landing Page │ │ PWA Shell │ │ Admin │ │ +│ │ / │ │ /app/* │ │ /admin │ │ +│ └──────────────┘ └──────────────┘ └──────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────┐ │ +│ │ API PROXY LAYER │ │ +│ │ /api/v1/* → GoToSocial pass-through │ │ +│ │ /api/v2/* → Chirp custom extensions │ │ +│ └──────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────┐ │ +│ │ WEBSOCKET HUB (Go) │ │ +│ │ Real-time: timeline, notifications, DMs │ │ +│ │ Bridges GoToSocial streaming + custom events│ │ +│ └──────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────┐ │ +│ │ SCHEDULER (Go) │ │ +│ │ Post scheduling, recurring jobs │ │ +│ │ Backed by Redis job queue │ │ +│ └──────────────────────────────────────────────┘ │ +└──────────┬──────────────┬──────────────┬────────────┘ + │ │ │ + ▼ ▼ ▼ +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ GOTOSOCIAL │ │ MEILISEARCH │ │ REDIS │ +│ (Go, K3s) │ │ (Rust, K3s) │ │ (K3s) │ +│ │ │ │ │ │ +│ ActivityPub │ │ Full-text │ │ Sessions │ +│ Federation │ │ search idx │ │ Job queue │ +│ REST API │ │ posts,users │ │ Rate limit │ +│ WebSockets │ │ hashtags │ │ Cache │ +│ Web Push │ └──────────────┘ └──────────────┘ +│ OAuth2/OIDC │ +└──────┬───────┘ + │ + ├──────────────────────────┐ + │ │ + ▼ ▼ +┌──────────────┐ ┌──────────────┐ +│ POSTGRESQL │ │ MINIO │ +│ (K3s) │ │ s3.mff.com │ +│ │ │ │ +│ GTS schema │ │ chirp-media │ +│ chirp_* │ │ bucket │ +│ schema │ │ (images, │ +│ extensions │ │ video, │ +└──────────────┘ │ audio) │ + └──────────────┘ + │ + ┌─────────────────────────┤ + │ │ + ▼ ▼ +┌──────────────┐ ┌──────────────┐ +│ MEDIAMTX │ │ ZITADEL │ +│ stream.mff │ │ auth.mff │ +│ │ │ │ +│ Live video │ │ SSO / OIDC │ +│ RTMP/HLS │ │ Identity │ +│ WebRTC │ │ provider │ +└──────────────┘ └──────────────┘ + │ + ▼ +┌──────────────────────────────────────────┐ +│ FEDIVERSE (Internet) │ +│ Mastodon, Misskey, Pleroma, Pixelfed │ +│ Any ActivityPub server worldwide │ +└──────────────────────────────────────────┘ +``` + +--- + +## 2. COMPONENTS — ROLES AND RESPONSIBILITIES + +### GoToSocial (backend engine) +- ActivityPub server — handles all federation with the Fediverse +- Mastodon-compatible REST API (`/api/v1/`) +- WebSocket streaming (`wss://chirp.myfacefeed.com/api/v1/streaming`) +- Web Push notifications (v0.18+) +- OAuth2 + OIDC (integrates with Zitadel at auth.myfacefeed.com) +- Media storage via MinIO S3 (`chirp-media` bucket) +- PostgreSQL backend + +### Chirp Go Web Server (custom build) +- Serves landing page + PWA shell +- API proxy: pass `/api/v1/*` through to GoToSocial +- Custom API extensions at `/api/v2/*` for missing features +- WebSocket hub bridging GoToSocial streaming + custom events +- Post scheduler (Redis-backed) +- Meilisearch integration for full-text search +- Service worker + web manifest (makes it a PWA) +- Push notification subscription management + +### Meilisearch (search) +- Full-text search across posts, users, hashtags +- Indexes synced from GoToSocial via webhook + polling +- Go client: `github.com/meilisearch/meilisearch-go` +- RAM: practical ~200-400MB for small-medium instance + +### Redis +- Session store for Chirp Go server +- Job queue for post scheduler +- Rate limiting +- Cache for expensive GoToSocial API responses +- Pub/sub for WebSocket hub fan-out + +### PostgreSQL (shared instance, separate schemas) +- `gotosocial` schema — owned by GoToSocial, never touch directly +- `chirp` schema — owned by Chirp Go server, all custom extensions +- Same DB host, separate connection users + +### MinIO (existing at s3.myfacefeed.com) +- New bucket: `chirp-media` +- GoToSocial configured to use it directly +- CDN-fronted via cdn.myfacefeed.com for serving media + +--- + +## 3. DATABASE DESIGN + +### GoToSocial (do not modify — reference only) +GoToSocial manages its own schema via internal migrations. +Key tables it owns (read-only for Chirp): +``` +accounts — user profiles, actor info +statuses — posts +follows — follow relationships +notifications — notification records +media_attachments — uploaded media references +tokens — OAuth tokens +applications — registered OAuth apps +``` + +### Chirp Schema Extensions (chirp.*) +All custom tables live in the `chirp` PostgreSQL schema. +Chirp Go server connects as user `chirp_app` with access only to `chirp.*`. + +```sql +-- ============================================================ +-- BOOKMARKS +-- GoToSocial has /bookmark endpoint but no persistent store +-- We store the mapping here +-- ============================================================ +CREATE TABLE chirp.bookmarks ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + account_id TEXT NOT NULL, -- GoToSocial account ID + status_id TEXT NOT NULL, -- GoToSocial status ID + created_at TIMESTAMPTZ DEFAULT NOW(), + UNIQUE(account_id, status_id) +); + +-- ============================================================ +-- LISTS +-- GoToSocial has list endpoints — we extend with metadata +-- ============================================================ +CREATE TABLE chirp.lists ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + account_id TEXT NOT NULL, + title TEXT NOT NULL, + replies_policy TEXT DEFAULT 'list', -- list, followed, none + created_at TIMESTAMPTZ DEFAULT NOW() +); + +CREATE TABLE chirp.list_members ( + list_id UUID REFERENCES chirp.lists(id) ON DELETE CASCADE, + account_id TEXT NOT NULL, -- member's GoToSocial account ID + added_at TIMESTAMPTZ DEFAULT NOW(), + PRIMARY KEY (list_id, account_id) +); + +-- ============================================================ +-- SCHEDULED POSTS +-- ============================================================ +CREATE TABLE chirp.scheduled_posts ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + account_id TEXT NOT NULL, + scheduled_at TIMESTAMPTZ NOT NULL, + status_text TEXT NOT NULL, + visibility TEXT DEFAULT 'public', + content_warning TEXT, + media_ids TEXT[], -- GoToSocial media attachment IDs + poll JSONB, -- poll options if applicable + in_reply_to_id TEXT, + language TEXT DEFAULT 'en', + sensitive BOOLEAN DEFAULT FALSE, + created_at TIMESTAMPTZ DEFAULT NOW(), + dispatched_at TIMESTAMPTZ, -- set when sent to GoToSocial + failed_at TIMESTAMPTZ, + error TEXT +); + +-- ============================================================ +-- CONVERSATION MUTES +-- ============================================================ +CREATE TABLE chirp.conversation_mutes ( + account_id TEXT NOT NULL, + status_id TEXT NOT NULL, -- root status of conversation + muted_at TIMESTAMPTZ DEFAULT NOW(), + PRIMARY KEY (account_id, status_id) +); + +-- ============================================================ +-- SEARCH INDEX SYNC TRACKING +-- Tracks what has been indexed in Meilisearch +-- ============================================================ +CREATE TABLE chirp.search_sync ( + status_id TEXT PRIMARY KEY, + indexed_at TIMESTAMPTZ, + updated_at TIMESTAMPTZ, + deleted_at TIMESTAMPTZ +); + +-- ============================================================ +-- PUSH NOTIFICATION SUBSCRIPTIONS +-- Web Push subscriptions per account/device +-- ============================================================ +CREATE TABLE chirp.push_subscriptions ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + account_id TEXT NOT NULL, + endpoint TEXT NOT NULL UNIQUE, + p256dh_key TEXT NOT NULL, + auth_key TEXT NOT NULL, + alert_follow BOOLEAN DEFAULT TRUE, + alert_mention BOOLEAN DEFAULT TRUE, + alert_reblog BOOLEAN DEFAULT TRUE, + alert_poll BOOLEAN DEFAULT TRUE, + alert_status BOOLEAN DEFAULT FALSE, + created_at TIMESTAMPTZ DEFAULT NOW() +); + +-- ============================================================ +-- RATE LIMITING OVERRIDES (per account) +-- ============================================================ +CREATE TABLE chirp.rate_limit_overrides ( + account_id TEXT PRIMARY KEY, + requests_per_min INT DEFAULT 300, + updated_at TIMESTAMPTZ DEFAULT NOW() +); + +-- indexes +CREATE INDEX ON chirp.bookmarks(account_id); +CREATE INDEX ON chirp.scheduled_posts(scheduled_at) WHERE dispatched_at IS NULL; +CREATE INDEX ON chirp.conversation_mutes(account_id); +CREATE INDEX ON chirp.push_subscriptions(account_id); +``` + +--- + +## 4. MISSING GOTOSOCIAL FEATURES — HOW CHIRP HANDLES EACH + +| Missing Feature | Where Handled | Approach | +|----------------|---------------|----------| +| Full-text search | Chirp Go + Meilisearch | Index all public posts + users. Custom `/api/v2/search` endpoint | +| Post scheduling | Chirp Go + Redis + chirp.scheduled_posts | Store in DB, Redis job fires at scheduled_at, POSTs to GoToSocial | +| Conversation muting | Chirp Go + chirp.conversation_mutes | Client-side filter on timeline; Go server filters WebSocket stream | +| Federated hashtag search | Chirp Go | Query multiple known instances via their public APIs, merge results | +| Shared/imported block lists | Chirp Go admin | Fetch known block lists (e.g. Oliphant, Seirdy), push to GoToSocial admin API | +| Video streaming | MediaMTX (already running) | stream.myfacefeed.com — RTMP ingest, HLS/WebRTC playback, link in post | +| Web signup flow | Chirp Go | Custom signup page POSTs to GoToSocial admin API to create account | +| Account import/export | Chirp Go | Export: query GoToSocial for follows/blocks, serialize CSV. Import: re-follow via API | +| Follow/block list import | Chirp Go | Parse Mastodon-format CSV, batch follow/block via GoToSocial API | +| Hashtag following | Chirp Go + chirp schema | Track in DB, filter WebSocket stream to inject matching posts | +| Trending (local+federated) | Chirp Go | Aggregate hashtag counts from GoToSocial + query remote instances | + +--- + +## 5. FULL API SHEET + +### 5A. GoToSocial Pass-Through (Chirp proxies these unchanged) + +``` +ACCOUNTS + GET /api/v1/accounts/verify_credentials + PATCH /api/v1/accounts/update_credentials + GET /api/v1/accounts/:id + GET /api/v1/accounts/:id/statuses + GET /api/v1/accounts/:id/followers + GET /api/v1/accounts/:id/following + POST /api/v1/accounts/:id/follow + POST /api/v1/accounts/:id/unfollow + POST /api/v1/accounts/:id/block + POST /api/v1/accounts/:id/unblock + POST /api/v1/accounts/:id/mute + POST /api/v1/accounts/:id/unmute + GET /api/v1/accounts/relationships + GET /api/v1/accounts/search + POST /api/v1/accounts/:id/note + +STATUSES + POST /api/v1/statuses (create post) + GET /api/v1/statuses/:id + DELETE /api/v1/statuses/:id + PUT /api/v1/statuses/:id (edit post) + GET /api/v1/statuses/:id/context (thread view) + GET /api/v1/statuses/:id/reblogged_by + GET /api/v1/statuses/:id/favourited_by + POST /api/v1/statuses/:id/favourite + POST /api/v1/statuses/:id/unfavourite + POST /api/v1/statuses/:id/reblog + POST /api/v1/statuses/:id/unreblog + POST /api/v1/statuses/:id/bookmark + POST /api/v1/statuses/:id/unbookmark + POST /api/v1/statuses/:id/pin + POST /api/v1/statuses/:id/unpin + GET /api/v1/statuses/:id/history (edit history) + +TIMELINES + GET /api/v1/timelines/home + GET /api/v1/timelines/public + GET /api/v1/timelines/tag/:hashtag + GET /api/v1/timelines/list/:id + +NOTIFICATIONS + GET /api/v1/notifications + GET /api/v1/notifications/:id + POST /api/v1/notifications/clear + POST /api/v1/notifications/:id/dismiss + +MEDIA + POST /api/v2/media (upload attachment) + GET /api/v1/media/:id + PUT /api/v1/media/:id (update description) + +STREAMING (WebSocket) + GET /api/v1/streaming?stream=user + GET /api/v1/streaming?stream=public + GET /api/v1/streaming?stream=public:local + GET /api/v1/streaming?stream=hashtag&tag=name + GET /api/v1/streaming?stream=list&list=id + GET /api/v1/streaming?stream=direct + +FILTERS + GET /api/v2/filters + POST /api/v2/filters + GET /api/v2/filters/:id + PUT /api/v2/filters/:id + DELETE /api/v2/filters/:id + +LISTS + GET /api/v1/lists + POST /api/v1/lists + GET /api/v1/lists/:id + PUT /api/v1/lists/:id + DELETE /api/v1/lists/:id + GET /api/v1/lists/:id/accounts + POST /api/v1/lists/:id/accounts + DELETE /api/v1/lists/:id/accounts + +POLLS + GET /api/v1/polls/:id + POST /api/v1/polls/:id/votes + +INSTANCE + GET /api/v1/instance + GET /api/v2/instance + GET /api/v1/instance/peers + GET /api/v1/instance/rules + +OAUTH + POST /api/v1/apps + GET /oauth/authorize + POST /oauth/token + POST /oauth/revoke + +TRENDS + GET /api/v1/trends/tags + GET /api/v1/trends/statuses + GET /api/v1/trends/links + +FOLLOW REQUESTS + GET /api/v1/follow_requests + POST /api/v1/follow_requests/:id/authorize + POST /api/v1/follow_requests/:id/reject + +ADMIN (GoToSocial admin API) + GET /api/v1/admin/accounts + POST /api/v1/admin/accounts/:id/action + GET /api/v1/admin/reports + GET /api/v1/admin/domain_allows + POST /api/v1/admin/domain_allows + GET /api/v1/admin/domain_blocks + POST /api/v1/admin/domain_blocks +``` + +### 5B. Chirp Custom API Extensions (/api/v2/chirp/*) + +``` +SEARCH (Meilisearch-backed) + GET /api/v2/chirp/search + ?q=query&type=accounts|statuses|hashtags + &federated=true|false + &from=username + &before=date&after=date + &limit=20&offset=0 + +SCHEDULED POSTS + GET /api/v2/chirp/scheduled_statuses + POST /api/v2/chirp/scheduled_statuses + body: { text, scheduled_at, visibility, cw, media_ids, poll } + GET /api/v2/chirp/scheduled_statuses/:id + PUT /api/v2/chirp/scheduled_statuses/:id + DELETE /api/v2/chirp/scheduled_statuses/:id + +CONVERSATION MUTES + POST /api/v2/chirp/statuses/:id/mute_conversation + POST /api/v2/chirp/statuses/:id/unmute_conversation + GET /api/v2/chirp/muted_conversations + +ACCOUNT IMPORT / EXPORT + GET /api/v2/chirp/export/follows → CSV download + GET /api/v2/chirp/export/blocks → CSV download + GET /api/v2/chirp/export/mutes → CSV download + POST /api/v2/chirp/import/follows → CSV upload, batch follow + POST /api/v2/chirp/import/blocks → CSV upload, batch block + +BLOCK LIST SUBSCRIPTIONS (shared block lists) + GET /api/v2/chirp/admin/blocklist_subscriptions + POST /api/v2/chirp/admin/blocklist_subscriptions + body: { url, description } + DELETE /api/v2/chirp/admin/blocklist_subscriptions/:id + POST /api/v2/chirp/admin/blocklist_subscriptions/:id/sync + +FEDERATED HASHTAG SEARCH + GET /api/v2/chirp/federated/tag/:hashtag + Queries known peer instances, merges + deduplicates results + +TRENDING (enhanced) + GET /api/v2/chirp/trends/federated Merges trends from peer instances + +WEB PUSH (Chirp-managed subscriptions) + POST /api/v2/chirp/push/subscription + body: { endpoint, keys: { p256dh, auth }, alerts: {...} } + PUT /api/v2/chirp/push/subscription + GET /api/v2/chirp/push/subscription + DELETE /api/v2/chirp/push/subscription + +LIVE VIDEO (MediaMTX integration) + POST /api/v2/chirp/live/start Creates stream key, returns RTMP URL + DELETE /api/v2/chirp/live/:stream_key Ends stream + GET /api/v2/chirp/live/:stream_key HLS playback URL + viewer count + +INSTANCE ADMIN (Chirp layer) + GET /api/v2/chirp/admin/stats + GET /api/v2/chirp/admin/search_index Meilisearch index stats + POST /api/v2/chirp/admin/search_index/rebuild +``` + +--- + +## 6. SEARCH — MEILISEARCH IMPLEMENTATION + +### Index Design +``` +Index: chirp_statuses + Searchable fields: content, account_username, account_display_name, tags + Filterable: account_id, visibility, language, created_at, tags + Sortable: created_at, favourites_count, reblogs_count + +Index: chirp_accounts + Searchable fields: username, display_name, note (bio), fields + Filterable: instance, discoverable + Sortable: followers_count + +Index: chirp_tags + Searchable fields: name + Sortable: uses_week, uses_total +``` + +### Sync Strategy +``` +1. Initial index: bulk fetch from GoToSocial admin API, batch insert +2. Real-time sync: subscribe to GoToSocial WebSocket stream + - on Create → index new status + - on Delete → remove from index + - on Update → re-index status +3. Chirp Go indexer runs as goroutine, channels for queue +4. Meilisearch tasks are async — track task ID in chirp.search_sync +5. Re-index job runs nightly to catch any gaps +``` + +### Go Client +```go +import "github.com/meilisearch/meilisearch-go" + +client := meilisearch.New("http://meilisearch:7700", + meilisearch.WithAPIKey("MEILI_MASTER_KEY")) + +index := client.Index("chirp_statuses") +index.AddDocuments(docs, "id") +index.Search("golang", &meilisearch.SearchRequest{ + Limit: 20, + Filter: "visibility = public", +}) +``` + +--- + +## 7. STREAMING — WEBSOCKET ARCHITECTURE + +GoToSocial provides native WebSocket streaming at `/api/v1/streaming`. +Chirp Go server bridges it and adds custom event types. + +``` +User Browser (WebSocket) + │ + ▼ +Chirp Go WebSocket Hub + │ + ┌────┴─────────────────────────────┐ + │ │ + ▼ ▼ +GoToSocial Streaming Chirp Custom Events +/api/v1/streaming - scheduled_post_sent +(user, public, hashtag, list) - conversation_muted + - search_result_push + - live_stream_started +``` + +### Hub Design (Go) +``` +Hub struct { + clients map[*Client]bool + broadcast chan Event + register chan *Client + unregister chan *Client + rooms map[string]map[*Client]bool // room = stream type +} +``` +- One goroutine per GoToSocial WebSocket connection +- Redis pub/sub for fan-out across multiple Chirp pod replicas +- Events: `update`, `notification`, `delete`, `filters_changed` + Chirp custom types + +--- + +## 8. POST SCHEDULER — REDIS JOB QUEUE + +``` +POST /api/v2/chirp/scheduled_statuses + │ + ▼ +1. Store in chirp.scheduled_posts (PostgreSQL) +2. Push job to Redis sorted set (score = unix timestamp of scheduled_at) + │ + ▼ +Chirp Scheduler Goroutine (runs every 30 seconds) + │ +ZRANGEBYSCORE chirp:scheduled 0 now() + │ + ▼ +For each due job: + 1. POST to GoToSocial /api/v1/statuses with stored content + 2. Mark chirp.scheduled_posts.dispatched_at = now() + 3. Remove from Redis sorted set + 4. On failure: set failed_at, store error, retry up to 3x +``` + +--- + +## 9. ACTIVITYPUB — WHAT GOTOSOCIAL HANDLES + +GoToSocial manages all ActivityPub automatically. No custom AP code needed in Chirp. + +| Activity | Who sends | Who receives | +|----------|-----------|-------------| +| Create (Note) | GoToSocial | Remote servers | +| Delete | GoToSocial | Remote servers | +| Announce (boost) | GoToSocial | Remote servers | +| Like | GoToSocial | Remote servers | +| Follow/Unfollow | GoToSocial | Remote servers | +| Accept/Reject | GoToSocial | Remote servers | +| Update (profile/post) | GoToSocial | Remote servers | +| Move (migration) | GoToSocial | Remote servers | +| Flag (report) | GoToSocial | Remote servers | + +Federation endpoints GoToSocial exposes (do not intercept): +``` +GET /.well-known/webfinger +GET /.well-known/nodeinfo +GET /nodeinfo/2.0 +GET /users/:username (Actor object) +GET /users/:username/inbox (AP inbox) +POST /users/:username/inbox (receive AP activities) +GET /users/:username/outbox +GET /users/:username/followers +GET /users/:username/following +``` + +--- + +## 10. OAUTH / AUTHENTICATION FLOW + +GoToSocial supports OIDC — connect to Zitadel (already at auth.myfacefeed.com): + +``` +User clicks Login + │ + ▼ +Chirp Go → redirect to Zitadel (auth.myfacefeed.com) + │ + ▼ (user authenticates with Zitadel) + │ +Zitadel → callback to GoToSocial OIDC callback + │ +GoToSocial issues OAuth token + │ +Chirp Go stores token in Redis session + │ +User is logged in — token used for all API calls +``` + +Benefits: +- SSO across entire MFF platform (same Zitadel login as Nextcloud, Mattermost, etc.) +- No separate password for Chirp +- Chirp Go never sees raw passwords + +--- + +## 11. PWA SPECIFICS + +### Web Manifest (/manifest.json) +```json +{ + "name": "Chirp", + "short_name": "Chirp", + "start_url": "/app", + "display": "standalone", + "background_color": "#0f172a", + "theme_color": "#2563eb", + "icons": [ + { "src": "/icons/192.png", "sizes": "192x192", "type": "image/png" }, + { "src": "/icons/512.png", "sizes": "512x512", "type": "image/png" }, + { "src": "/icons/512-maskable.png", "sizes": "512x512", + "type": "image/png", "purpose": "maskable" } + ], + "shortcuts": [ + { "name": "Home", "url": "/app/home" }, + { "name": "Notifications", "url": "/app/notifications" }, + { "name": "Compose", "url": "/app/compose" } + ] +} +``` + +### Service Worker Capabilities +- Cache shell (app HTML/CSS/JS) for offline load +- Cache recent timeline posts for offline reading +- Background sync: queue posts written offline, send when back online +- Push notifications: receive Web Push, display even when app closed +- Share target: receive shared URLs from other apps + +--- + +## 12. GO CODE STRUCTURE + +``` +chirp/ +├── cmd/ +│ └── chirp/ +│ └── main.go entry point +│ +├── internal/ +│ ├── api/ +│ │ ├── proxy.go GoToSocial API pass-through +│ │ ├── search.go /api/v2/chirp/search handlers +│ │ ├── scheduler.go scheduled posts handlers +│ │ ├── mutes.go conversation mute handlers +│ │ ├── push.go Web Push handlers +│ │ ├── live.go MediaMTX live video handlers +│ │ ├── import_export.go CSV import/export +│ │ └── admin.go admin endpoints +│ │ +│ ├── ws/ +│ │ ├── hub.go WebSocket hub +│ │ ├── client.go per-connection client +│ │ └── bridge.go GoToSocial stream bridge +│ │ +│ ├── search/ +│ │ ├── meilisearch.go search client wrapper +│ │ ├── indexer.go real-time indexing goroutine +│ │ └── sync.go nightly re-index job +│ │ +│ ├── scheduler/ +│ │ ├── scheduler.go Redis job runner +│ │ └── dispatcher.go posts to GoToSocial +│ │ +│ ├── db/ +│ │ ├── db.go PostgreSQL connection pool +│ │ ├── bookmarks.go +│ │ ├── lists.go +│ │ ├── scheduled.go +│ │ ├── mutes.go +│ │ └── push.go +│ │ +│ ├── gts/ +│ │ └── client.go GoToSocial API client (typed) +│ │ +│ ├── push/ +│ │ └── webpush.go Web Push sender (VAPID) +│ │ +│ └── config/ +│ └── config.go env-based config +│ +├── web/ +│ ├── static/ +│ │ ├── sw.js service worker +│ │ ├── manifest.json +│ │ ├── app.js PWA app bundle +│ │ └── app.css +│ └── templates/ +│ ├── landing.html public landing page +│ └── app.html PWA shell (single page) +│ +├── migrations/ +│ └── *.sql chirp schema migrations +│ +├── Dockerfile (minimal — scratch or alpine) +├── go.mod +└── go.sum +``` + +--- + +## 13. K3S DEPLOYMENT + +```yaml +# 3 pods: gotosocial, chirp, meilisearch +# Shared: postgresql (existing), redis (existing), minio (existing) + +Deployments: + gotosocial 1 replica ~350MB RAM chirp-media S3 bucket + chirp 2 replicas ~100MB RAM (Go is lean) + meilisearch 1 replica ~300MB RAM PVC for index data + +Services: + gotosocial ClusterIP :8080 + chirp ClusterIP :3000 + meilisearch ClusterIP :7700 (internal only — never exposed) + +Ingress: + chirp.myfacefeed.com → chirp :3000 + +Total new RAM: ~750MB-1GB +(vs Mastodon which would need 2-4GB) +``` + +--- + +## 14. BUILD ORDER (Implementation Sequence) + +``` +Phase 1 — Backend up + 1. GoToSocial K3s deployment + MinIO bucket + PostgreSQL DB + 2. Zitadel OIDC config for GoToSocial + 3. chirp schema migrations + 4. Meilisearch deployment + initial index + +Phase 2 — Chirp Go server core + 5. API proxy (pass-through to GoToSocial) + 6. WebSocket hub + GoToSocial stream bridge + 7. Session management (Redis) + 8. OAuth flow with Zitadel + +Phase 3 — PWA frontend + 9. Landing page (chirp.myfacefeed.com public view) + 10. App shell + service worker + manifest + 11. Timeline view (home, public, local) + 12. Compose post, reply, boost, favourite + 13. Profile pages + 14. Notifications + +Phase 4 — Custom features (missing from GoToSocial) + 15. Search (Meilisearch integration) + 16. Post scheduling (Redis scheduler) + 17. Conversation muting + 18. Import/export follows/blocks + 19. Shared block lists + 20. Live video integration (MediaMTX) + +Phase 5 — Polish + 21. Push notifications (Web Push + VAPID) + 22. Offline support (service worker caching) + 23. PWA install prompt + 24. Mobile responsive polish + 25. Admin panel +``` + +--- + +## 15. ENVIRONMENT CONFIG (Chirp Go Server) + +```bash +# GoToSocial +GTS_HOST=gotosocial:8080 +GTS_ADMIN_TOKEN= + +# PostgreSQL (chirp schema) +CHIRP_DB_HOST=postgresql:5432 +CHIRP_DB_NAME=chirp +CHIRP_DB_USER=chirp_app +CHIRP_DB_PASS= + +# Redis +REDIS_HOST=redis:6379 +REDIS_PASS= + +# Meilisearch +MEILI_HOST=http://meilisearch:7700 +MEILI_KEY= + +# OAuth / Zitadel +OIDC_ISSUER=https://auth.myfacefeed.com +OIDC_CLIENT_ID=chirp +OIDC_CLIENT_SECRET= + +# Web Push (VAPID keys — generate once) +VAPID_PUBLIC_KEY= +VAPID_PRIVATE_KEY= +VAPID_SUBJECT=mailto:admin@myfacefeed.com + +# MinIO +S3_ENDPOINT=s3.myfacefeed.com +S3_BUCKET=chirp-media +S3_ACCESS_KEY= +S3_SECRET_KEY= + +# MediaMTX +MEDIAMTX_API=http://mediamtx:9997 +MEDIAMTX_RTMP_HOST=rtmp://stream.myfacefeed.com + +# App +CHIRP_HOST=https://chirp.myfacefeed.com +CHIRP_PORT=3000 +SESSION_SECRET= +``` diff --git a/cmd/server/.gitkeep b/cmd/server/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/deploy/k8s/.gitkeep b/deploy/k8s/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/deploy/nginx/.gitkeep b/deploy/nginx/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/internal/config/.gitkeep b/internal/config/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/internal/db/.gitkeep b/internal/db/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/internal/gts/.gitkeep b/internal/gts/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/internal/oauth/.gitkeep b/internal/oauth/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/internal/proxy/.gitkeep b/internal/proxy/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/internal/scheduler/.gitkeep b/internal/scheduler/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/internal/search/.gitkeep b/internal/search/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/internal/ws/.gitkeep b/internal/ws/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/tests/e2e/.gitkeep b/tests/e2e/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/tests/integration/.gitkeep b/tests/integration/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/web/static/css/.gitkeep b/web/static/css/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/web/static/icons/.gitkeep b/web/static/icons/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/web/static/js/.gitkeep b/web/static/js/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/web/sw/.gitkeep b/web/sw/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/web/templates/.gitkeep b/web/templates/.gitkeep new file mode 100644 index 0000000..e69de29