How the system fits together, what the data looks like, and what gets built next. Milestones 1 to 3 are done and M4 is under way: a daily Lambda ingests 31 sources (venue calendars on a dozen platforms, Racket's weekly roundups and Minnesota Monthly's calendar) into Neon, about 1,950 upcoming listings, and enrichment profiles the acts playing in the next eight weeks. Two pieces of later milestones are live too: the first version of the site (M6) and our own categories and genres (M7). The plan is now venue-first: venue calendars are the record, and publications like Racket point to the venues and shows we don't track yet. Requirements are in docs/requirements.md.
birdfeed-ingest./birdfeed/database_url.BirdFeed/… user agent and parsed by its adapter. Prose sources go through LLM extraction in the same run.main that touches pipeline/, config/ or infra/pipeline/ starts deploy.yml.build_lambda.sh builds the zip (x86_64, Python 3.13).terraform apply runs as IAM user birdfeed-github-deploy, since the project's SCPs block OIDC.{"action": "migrate"}, and Alembic runs against Neon's direct (unpooled) endpoint.Built so far: sources, venues and venue_aliases, events (really per-source listings), raw_pages and raw_labels, and the performer tables: performers, performer_aliases, event_performers and performer_profiles. This is the target model. Each table is labelled with the milestone that built it or will add it.
Lines are foreign keys. The links from events to festivals, event_organizers and picks aren't drawn, and neither are the links from performers to performer_profiles and memberships. Tables marked ANY ENTITY attach to events, performers, venues or sources through entity_type + entity_id.
A feed we fetch. Becomes the source of truth instead of sources.yaml.
adapterical, squarespace, racket, …settingsJSONB: url, title_time_prefix, …venue_idnull for multi-venue sources like Racketstatusproposed, active, paused, brokendiscovered_byconfig, you, or a research tasklast_fetched_at, last_error, consecutive_failuresIndex of fetched pages cached in S3. Extraction re-runs only when the hash changes.
source_id, urls3_key, content_hashfetched_at, extracted_atOne event as one source lists it. Exactly what's there today, plus a link to the canonical event.
source_id, external_idunique togetherevent_id newthe de-duplicated eventvenue_id, title, starts_at, time_known, prices, urlsperformer_names newas listed, before resolutionraw, content_hash, first_seen_at, last_seen_at, removed_at newGenre or category text exactly as a source gave it. Kept as evidence, never shown as our own classification.
listing_id, labelfieldwhere it came from: category, description, titleThe real-world happening. Several listings from different sources point to one event.
venue_id, series_id, festival_idtitle, starts_at, ends_at, time_knownstatusscheduled, cancelled, removed, sold_outprice_min, price_max, is_free, age_policyprimary_listing_idwhich listing supplies the display fieldsA recurring thing: trivia Tuesdays, a monthly open mic. Occurrences are events.
venue_id, namerruleiCal recurrence rulehost_performer_idA place. Created from config, by extraction, or by the research agent.
slug, name, address, urlkindclub, bar, theater, gallery, brewery, park, library…neighborhood, city, lat, lngcapacity, age_policy, accessibility, price_tierlinksJSONB: socials, MusicBrainz place, Wikidatastatusactive, needs_review, closedfield_provenanceJSONB: field → provenance idEvery way a source names a venue. Matching normalizes case and "the".
venue_id, aliasunique, normalizedsource_idwhere the alias was seenAny act: band, solo musician, DJ, comedian, speaker, theater company, or a person who plays in groups.
name, sort_name, kind, disambiguationorigin, active_yearsmusicbrainz_id, wikidata_idlinksJSONB: website, Bandcamp, Spotify, YouTubestatus, field_provenanceWho's on the bill, in order.
event_id, performer_idroleheadliner, support, host, dj, memberinstrumentsthat night's, e.g. a trio's listed playersposition, confidence, provenance_idSpellings, old band names and stage names, used when resolving listings.
performer_id, aliasWhat a performer sounds like, extracted from all their descriptions. Bio trivia stays out.
performer_idsound_summaryone or two sentences; embedded as the sound aspectcomparisonsJSONB: "for fans of" text + resolved performer idscontextJSONB: origin, label, active yearsevidenceJSONB: fact → source URL + quotesourcesJSONB: the pages read, by kindinputs_hash, model, provenance_idWho plays in a group, so a drummer can be followed across trios.
group_id, person_idboth performersinstruments, active_from, active_toprovenance_idPromoters, presenters and collectives. Often a better taste signal than the venue.
slug, namekindpromoter, presenter, collective, nonprofitlinks, status, field_provenanceWho's behind an event, as opposed to who's on stage.
event_id, organizer_idrolepresents, promotes, sponsorsprovenance_idA multi-day happening that groups events, possibly across venues: a music festival, the Fringe.
slug, name, urlstarts_on, ends_onorganizer_idoptionalOur taxonomy. Top level is a fixed set of categories; below are open genre tags.
slug, name, parent_idlevelcategory, genre, instrument, mood, settingstatusaccepted, proposed, mergedmerged_into_idold tag kept as an aliasHow a raw label translates to our tags, decided once and reused.
labelnormalizedtag_id, weight, provenance_idTags on events, performers and venues. Events inherit performer tags at ranking time instead of copying them.
entity_type, entity_id, tag_idweight0 to 1originsource label, llm, agent, youprovenance_idSeveral per entity. The app shows a generated summary, or an excerpt plus a link.
entity_type, entity_idkindsource, editorial, reference, generatedtext, source_url, retrieved_atbased_onfor generated ones: description ids usedprovenance_idVectors for similarity ranking. Each row records its model, so switching models means re-embedding, not a schema migration.
entity_type, entity_id, aspect, modelaspectsound first, built from the profile, never the raw biovectorpgvectorcontent_hashre-embed only on changeWhere a fact came from. Hand-entered or approved facts are never overwritten by automation.
actoradapter, llm, agent, youtool, model, source_urlretrieved_at, confidenceresearch_task_idreview_stateauto, pending, approved, rejectedThe agent's queue. Created by the gap check after each ingest.
kindenrich_performer, enrich_venue, resolve_performer, discover_sourceentity_type, entity_id, missingfields to fillpriorityfrom next event datestatusqueued, running, done, no_result, needs_review, failedattempts, retry_after, tokens_used, tool_callsThumbs on events, which also count weakly toward their performers, venue and tags.
event_id, value+1 or −1note, created_atYour written description of what you like, versioned.
text, updated_atRecommendations generated when you open the app, cached so reopening doesn't re-run the LLM.
generated_at, windowdate range coveredinputs_hashprofile, feedback, events; a change invalidatesevent_id, rank, reason, modelTags, descriptions, embeddings and research tasks use entity_type + entity_id with a check constraint, so a new entity type doesn't need new join tables. The cost is no foreign keys, so deletes clean these up in code.
Columns on venues and performers that research fills in record their provenance in a field_provenance map. A researched value never overwrites one you set or approved.
Times stay timezone-aware in UTC with the original zone. Slugs are stable and human-readable, and external IDs (MusicBrainz, Wikidata) are the preferred keys for matching.
Venue-first since 28 Oct: venue calendars are the record of what's on, and publications like Racket are discovery signals that point to venues and shows we don't track yet. Each step makes the next one testable: coverage before de-duplication, clean entities before research, something to look at before tuning taste. Performers and their profiles were pulled forward into M3.
main by GitHub Actions.Done: 27 Oct 2026.
birdfeed seed and export keep the YAML path, and birdfeed sources add adds a source of a known feed type without a deploy.venue_aliases, raw_labels and the S3 raw-page cache, so unchanged pages aren't re-extracted.needs_review.llm_calls.Done: 28 Oct 2026.
performers, performer_aliases and event_performers with roles and billing order.Remaining: lineup members as person performers with memberships (moved to M7).
page, which reads hand-built calendar pages with the LLM. Still to come as venues need them: DICE, Etix, SeeTickets and Eventbrite.Done when: a typical week is complete from venue calendars alone, and Racket mostly adds write-ups.
events into listings plus canonical events, and de-duplicate across sources, helped by the official show pages from M4. Built first: cross-source matching (same venue and day, a shared act or title) sets same_as_id to the venue calendar's listing, and the site shows each show once, crediting every source.series for recurring events. Mark removed listings and cancelled events.organizers, event_organizers and festivals.Done when: one real-world show appears once, with every source that listed it and a resolved headliner.
birdfeed-web, a read-only Lambda API behind Vercel's CDN. They share one look: a navy masthead, white pages, Libre Franklin, and maroon for what should stand out. The listings page filters by category and searches titles, venues, acts and genres.Done when: you check what ingest did in the app instead of in SQL.
tags (category, genre, instrument, mood, setting), label_mappings, taggings, descriptions and provenance. Built: tags, label_mappings and taggings for 16 categories and their genres, from config/tags.yaml, with each source label mapped once by alias or by Haiku. Instrument, mood and setting tags are still to come.person performers with memberships.research_tasks. SQS feeds a research worker that runs a Bedrock tool-use loop with MusicBrainz, Wikidata, page fetch and web search tools, within the $30/month budget.Done when: most performers with events in the next eight weeks have genres, a description and a listen link, and every fact traces to a source.
feedback thumbs that also count toward performers, venues and tags.sound vector embeds profile text, not raw bios.picks.Done when: the week view's top ten would mostly be your own picks.
picks.Done when: you ask it what to do on Friday before opening the week view.
infra/README.md).