How Observatory records become evidence

Fixed inputs

Tracked query sets declare the exact query text, category, locations, device classes and collection depth. Configuration is versioned with the methodology. A query is not retroactively edited; a changed query starts a distinct series.

Web search collection

A provider adapter returns positions, URLs, titles and result types. The archive stores normalized observations rather than provider-specific response bodies. The first private pilot uses Serper; provider identity, terms and limitations are disclosed before a dataset becomes public.

Only HTTP and HTTPS results are accepted. URL fragments and known campaign parameters are removed, hostnames are normalized, duplicate canonical URLs are discarded, and results outside the configured depth are excluded. The original observed rank is retained.

YouTube collection

Each observation makes two requests to the YouTube Data API v3 in the same run: a relevance-ordered search for the query in a region, then the public snippet and statistics of the videos it returned. Positions are the order of that search. View, like and comment counts are stored as reported; a count the channel hides is stored as null and never estimated. A video that disappears between the two requests is left out.

The API has no device dimension, so YouTube series are recorded as “unspecified” rather than claimed as desktop or mobile. Results are what the API returns for an anonymous request, which can differ from what a signed-in viewer sees.

Immutable snapshots

Each snapshot receives a deterministic identifier from its dataset, query, location, device and observation time. Files are write-once and checksummed. A repeated identical run is harmless; different content attempting to use an existing identity is rejected as a conflict.

Derived analysis

Derived numbers are computed only between snapshots of the same dataset, query, location and device, and only from published files. Each chart names what it shows.

• Volatility = (1 − RBO) × 100 between consecutive snapshots, where RBO is extrapolated rank-biased overlap with persistence p = 0.9. 0 means identical rankings, 100 means nothing in common, and changes near the top count most.
• Entered, exited, moved are counted separately. Mean movement uses only results present in both snapshots, so a missing result is never assigned an invented rank.
• Rank trajectories follow the results with the most visibility, Σ (depth + 1 − rank) / depth over all snapshots. Absent days are gaps.
• Domain share is the fraction of top-ten positions each domain held in a snapshot.
• Tenure counts the snapshots in which a result held a top-ten position, and its current unbroken streak.
• View velocity is the change in public views divided by the days between the first and last observation in a seven-day window.
• Breakout: a video in the latest snapshot whose velocity is at least three times the median velocity of the videos in that snapshot. It compares videos with each other; it is not a forecast.
• Lifecycle plots observed views against days since publication.

AI labelling and narratives

A language model (or, in the demo, fixed rules) labels each distinct result or video once: the companies, products or brands it is about, its content type (review, listicle, news, vendor, forum, tutorial, reference, opinion or other) and the searcher intent it serves (informational, commercial, transactional or navigational). The items are given to the model as data, its answer must fit that closed vocabulary, and an entity is kept only if its name can be found in the item’s own title, site or channel, so a model cannot introduce a company the result never mentions.

Labels are stored beside the archive, never inside it, and reviewed in a private console where a person can correct or reject any of them; a person’s correction is never overwritten by a later run. Different spellings of one entity are merged there, and merges apply to every past label when data is published. Each published dataset states who labelled it and what share a person reviewed.

Weekly narratives are drafted from a numbered list of facts computed from the record (volatility, entries and exits, the top result, labelled shares), never from the raw results. Every number in a draft must appear in a fact it cites, or it cannot be approved; only approved narratives are published, with the facts they cite.

Publication gate

Collection alone does not publish a dataset. A reviewed approval sets minimum history, distinct days and the largest allowed gap; the exporter re-verifies every checksum and the provenance of every snapshot before the catalog changes. Until then, the site reports that no verified dataset is available.

Demo sample

While a system has no published data, the site may show a fictional sample so its pages can be tried. Fictional sample data. Every site is invented on the reserved .example domain and every video and channel is made up, so the replay and analysis can be tried before real observations are published. It is replaced automatically once real data is published. Demo pages are excluded from search indexing and structured data, are marked DEMO wherever they appear, and pass through the same collection, publication and verification code as real data.

Changelog

1. 2026-10-06AI labelling 1.0

   Structured labels and weekly narratives, reviewed in a private console.

   • Each distinct result or video is labelled once with the entities it is about, its content type and the intent it serves, from a closed vocabulary (instructions items-1.0). An entity is kept only if its name appears in the item’s own title, site or channel.
   • Labels live beside the archive, never inside it. A person can correct or reject any label; a correction is never overwritten by a later model run. Names are merged into one entity per real-world entity when data is published.
   • Weekly narratives are drafted from facts computed from the record, never from raw results. Every number in a narrative must appear in a fact it cites, and only narratives a person approved are published.
   • Labels are published only for datasets whose approval allows it, with the share a person reviewed, and the published file is checksummed like the snapshots.

2. 2026-09-30YouTube 1.0

   YouTube collection protocol.

   • Relevance-ordered video ids for a query and region from the YouTube Data API v3, then each video’s public view, like and comment counts in the same run.
   • Hidden counts are stored as null, never estimated. A video removed between the two requests is left out.
   • No device dimension: the API does not offer one, so YouTube series are recorded as "unspecified" rather than claimed as desktop or mobile.

3. 2026-09-30Derived analysis 1.0

   Derived numbers published beside replay.

   • Volatility = (1 − rank-biased overlap) × 100 between consecutive snapshots of one series, with persistence p = 0.9.
   • Rank trajectories, domain share of the top ten, tenure, view velocity, cohort breakouts (≥ 3× the median velocity over 7 days) and view lifecycles.
   • An absent result has no rank: charts leave a gap rather than inventing one.

4. 2026-09-30Publication 1.0

   Demo sample policy.

   • A fictional sample on reserved .example domains may be shown for a system with no published data. It is labelled DEMO on every surface, excluded from search indexing and structured data, and withdrawn when real data is published.

5. 2026-09-04Web search 1.0

   Initial web-search protocol.

   • Fixed query sets with declared locations, devices and depth; one immutable, checksummed snapshot per query, location, device and time.
   • URL normalization: HTTP(S) only, fragments and campaign parameters removed, duplicates discarded, observed rank kept.
   • Reviewed publication with minimum history, maximum gap and provenance checks.