How SoonViral scores topics
This is the exact method behind every score on the site, and the public methodology page renders this file as it is. The code is lib/analytics/* (pure functions, unit-tested with fixtures in tests/unit/analytics-*.test.ts) and every weight and threshold lives in config/scoring.ts. Values written as `name` = value below are read from that file by a test (tests/unit/scoring-doc.test.ts), so this document cannot drift from the code without the build failing.
We say a topic is rising and show the evidence. We never claim it will go viral.
1. Data points
Every number comes from an official, documented API. Each stored data point is one topic, one source, one metric, one day: SignalSnapshot (topic, source, metric, date, value, fetched_at, licence). Pages and issues list only the sources their numbers actually used, with each source's attribution and licence.
| Source | Metric | Licence or terms |
|---|---|---|
| Wikipedia (Wikimedia Analytics API) | pageviews: human pageviews per day | CC0-1.0 |
| GDELT Project | articles: news articles mentioning the topic per day | GDELT-citation |
| arXiv | papers: new papers per day | CC0-1.0 (metadata) |
| Hacker News (official API) | stories: stories mentioning the topic per day | HN-API |
| Niche publications (RSS) | headlines: headlines per day | headline-link-only |
| npm | downloads_7d_mean: trailing 7-day mean of daily downloads | npm-open-source-terms |
| Open Food / Beauty Facts | new_products: new products mentioning the topic per day | ODbL-1.0 |
Sources that need a key or written permission (PubMed, Podcast Index, GitHub, Hugging Face, PyPI, Product Hunt, Reddit, Google Trends) stay off until configured; see docs/DATA_SOURCES.md. A source that fails a run is left out of that day's score (section 9).
2. Topics and the entity filter
Each topic is mapped to its Wikidata item (QID, from the Wikipedia article) and the item's "instance of" and "subclass of" values decide its class: programming language, AI model, AI app or agent, database, framework, library, developer tool, protocol, hardware, company, software, technique or concept, or a class we never show. The lists are in config/entity-classes.ts; when a value is not listed we walk up "subclass of" at most 3 steps.
- Never shown in any niche: dates, disambiguation pages, list articles and people.
- Developer Tools and AI Tools show only the classes that niche covers, and only topics with a Wikidata item. A seed that resolves to the wrong article is caught this way (the "Gemini 3" article is a 1965 spaceflight).
- Manual decisions on top:
config/entity-overrides.ts(allowlist) andconfig/blocked-topics.ts(blocklist), both reviewed in git. - A topic that is set aside keeps its history but is not measured, scored or listed. If Wikidata cannot be reached, topics keep their previous status.
3. Growth per source
For each topic and source we compare the recent level with the topic's own usual level, using robust statistics so one spike day does not distort the baseline.
- Recent level: the mean of the last
windows.recentDays= 3 days. - Usual level (4-week median): the median of days
windows.baselineStart= 4 towindows.baselineEnd= 31 back, floored atspike.baselineFloor= 1. Without at least 7 baseline days a source reports no change. - Spike ratio = recent level ÷ usual level.
- Robust z = (recent level − median) ÷ max(1.4826 × MAD, 0.1 × median, 1).
- Spiking: spike ratio ≥
spike.minRatio= 1.5 and robust z ≥spike.minRobustZ= 2. - Acceleration = ln((sum of the last 7 days + 1) ÷ (sum of the 7 days before + 1)), between −3 and 3. 0 is flat; 0.35 is about +42% week over week.
- Elevated days: the current run of days (single-day dips allowed, within the last 31) above the topic's quiet level (its 20th percentile) ×
saturation.elevatedFactor= 1.3. - Centred growth maps the ratio (recent + 1) ÷ (usual + 1) to 0..1: equal is 0.5,
growth.fullRatio= 4 times is 1, a quarter is 0, logarithmic in between.
Package downloads are stored as 7-day trailing means (downloads drop about 40% at weekends) and count for growth but not for audience size.
4. Trend score (0–100)
trend = 0.35 × attention + 0.15 × coverage + 0.10 × breadth + 0.15 × acceleration + 0.25 × audience − 0.10 × saturation, clamped to 0..1 and shown ×100.
| Component | Weight | What it is |
|---|---|---|
| Attention growth | weights.attentionGrowth = 0.35 | Mean centred growth of Wikipedia, Google Trends, npm and PyPI (sources that saw the topic) |
| Coverage growth | weights.coverageGrowth = 0.15 | Mean centred growth of the strongest 3 coverage sources (news, papers, Hacker News, feeds, launches) |
| Cross-source breadth | weights.crossSourceBreadth = 0.1 | Share of sources above their usual level (spike ratio ≥ growth.risingRatio = 1.1), a spiking source counting double |
| Acceleration | weights.acceleration = 0.15 | 0.5 + mean acceleration ÷ (2 ln 2): flat is 0.5, doubling is 1 |
| Audience | weights.audience = 0.25 | Percentile of the topic's 7-day volume within its niche |
| Saturation | weights.saturation = 0.1 | Subtracted; see below |
Volume for the audience percentile is the 7-day sum of attention sources (without package downloads) plus 50 × the 7-day sum of coverage counts. When a component has no healthy source its weight is spread over the others, so the score stays on the same scale. A topic at its usual level scores about 35–50; publish.minTrendForRadar = 50 or more reads "above usual". The trend score alone never puts a topic in Rising now (section 6).
Saturation = saturation.volumeWeight = 0.5 × audience percentile + saturation.durationWeight = 0.3 × min(1, elevated days ÷ saturation.elevatedDaysFull = 28) + saturation.mainstreamWeight = 0.2 × (1 if the article is in Wikipedia's daily top articles).
5. Stages
Rules in order, first match wins (lib/analytics/lifecycle.ts). With under confidence.limitedHistoryDays = 14 days of history a topic is Emerging if a source is at 1.5× or more and accelerating, otherwise Steady.
| Stage | Rule |
|---|---|
| Declining | More than half the sources have a 3-day moving average that fell lifecycle.declining.consecutiveFallingDays = 5 days running, and acceleration is negative |
| Rising (stored as EXPLODING) | At least lifecycle.exploding.minSourcesSpiking = 2 sources spiking, a peak spike of lifecycle.exploding.minSpike = 3 or more, acceleration ≥ lifecycle.exploding.minAcceleration = 0.35 |
| Saturated | Audience percentile ≥ lifecycle.saturated.minVolumePercentile = 0.6, elevated lifecycle.saturated.minElevatedDays = 21 days or more, acceleration ≤ lifecycle.saturated.maxAcceleration = 0.05 |
| Viral | Audience percentile ≥ lifecycle.viral.minVolumePercentile = 0.85, a spike of lifecycle.viral.minSpike = 2 or more, acceleration near zero (within lifecycle.viral.maxAbsAcceleration = 0.25) |
| Emerging | A source spiking, the rise at most lifecycle.emerging.maxAgeDays = 21 days old (since first detected or since the current elevated run began), audience percentile ≤ lifecycle.emerging.maxVolumePercentile = 0.6 |
| Steady | Anything else |
The rising rules then adjust the stage so the stage and the list always agree: a topic in Rising now is shown as Emerging, Rising or Viral (a Steady or Saturated one becomes Rising), and a topic outside it is never shown as Emerging or Rising (it becomes Steady).
6. Rising rules
Velocity is a source's spike ratio: its last-3-day mean over its 4-week median. A source only counts with a week of baseline and a last-3-day mean of at least rising.minRecentMean = 2 (sparse counts of 0–1 a day are noise). Days a source did not report (a failed backfill, a source added recently), and days it only sampled (GDELT's 15-minute files cover part of a day), are left out of its 4-week median and never count as a 2× day: a gap in history is not a zero, so it cannot pass as a jump from nothing.
A topic enters Rising now only if one of these holds:
- Rule A: velocity ≥
rising.multiSourceVelocity= 1.3 on at leastrising.minIndependentSources= 2 independent sources. Sources that measure the same activity count once: news wires and niche feeds are one source, all package registries are one, both Hacker News APIs are one (independenceGroupsinconfig/scoring.ts). - Rule B: ≥
rising.singleSourceVelocity= 2 times the 4-week median on one source on each of the lastrising.sustainedDays= 3 days. The median must be above zero: a source with no usual level cannot pass Rule B on its own.
And:
- Topics in the niche's top 10% by audience (percentile ≥
rising.bigAudiencePercentile= 0.9) also need a qualifying source at velocity ≥rising.bigAudienceVelocity= 2. Otherwise they are listed under Already big. - Stage Steady never appears in Rising now (section 5).
News spike. A day at ≥ newsSpike.minJump = 3 times the median of the 4 weeks before it, with a robust z of newsSpike.minRobustZ = 3 or more, coming from a normal day, that does not hold: the jump day and the two days after it are not all at 2× or more. A jump on one of the last two days cannot have held yet, so it stays a spike until it does. News spikes are listed in their own block, never in Rising now (unless a source has held 2× for 3 days), sorted by jump × weight, where the weight falls from 1 on the day of the jump to 0 after newsSpike.windowDays = 7 days, so spikes drop off within a week.
Lists on a niche page: Rising now (eligible, sorted by trend score), Early signals (eligible but confidence below confidence.earlySignalBelow = 0.45), News spikes (at most 5), Already big (top 10% by audience, viral or saturated, topped up with the most saturated topics to 6), Cooling down (Declining), and every other topic in a collapsed list.
7. Opportunity score (0–100)
opportunity = trend × (1 − 0.5 × saturation) × stage weight × (0.5 + 0.5 × niche relevance) × (1 − 0.5 × news-spike weight)
Saturation, relevance and a recent news spike each discount the trend by at most half. Stage weights: Emerging stageWeight.EMERGING = 1, Rising stageWeight.EXPLODING = 0.95, Steady stageWeight.STEADY = 0.8, Viral stageWeight.VIRAL = 0.75, Saturated stageWeight.SATURATED = 0.55, Declining stageWeight.DECLINING = 0.45. Sensitive topics (harm, tragedy, elections) get no opportunity score, no content ideas and no sponsors.
8. Confidence
confidence = 0.45 × min(1, sources ÷ confidence.fullSources = 4) + 0.25 × completeness + 0.30 × min(1, history days ÷ confidence.fullHistoryDays = 31)
Completeness is the share of expected source-days present over the last 31 days. Below 0.45 a topic is labelled "Early signal"; under 14 days of history, "New topic, limited history". A topic page is indexed by search engines only with at least publish.indexMinSources = 2 sources and publish.indexMinDays = 14 days of history, counting every source the page shows (the v2 inputs too, whichever version is published).
9. Freshness and failed runs
The data update runs every day at 04:30 UTC (.github/workflows/daily-data.yml), after Wikimedia publishes the previous day, and the site is rebuilt from it. Pages show "Updated N hours ago" from the time the scores were computed.
- Days already stored are never downloaded again; today and yesterday are refreshed. Each source is asked once per run (
scripts/update-niche.ts: history, live, then daily pass per niche). - The lead niches run first and in full. Paused niches measure fewer topics on the slow capped sources (
pausedNicheTopicCapsinconfig/launch.ts: GDELT 30, arXiv 30, Bluesky 60 topics a run, the strongest by trend), and a niche not started within the job's time budget waits for the next run. GitHub can start a scheduled job late. - If a core source (Wikipedia) fails for a niche, that day's scores are not published: the niche keeps showing its last good day (
Niche.lastGoodScoreDate) and its old timestamp. - Other failing sources are left out of the day's score and their weight is spread over the rest; the topic page says so.
/statusand/status.jsonshow the last successful update, each niche's scored day and the last fetch of every source.- Every published Rising now list is kept permanently (
RisingSnapshot) for the track record.
10. Limitations
- Wikipedia pageviews measure what people read, not what creators make; coverage counts are a weak proxy for how crowded a topic is.
- Baselines are the topic's own last 4 weeks, so a seasonal topic counts as rising when its season starts; there is no year-over-year adjustment yet.
- Sources only see topics by name; a name shared with something unrelated can inflate counts. Aliases and the entity filter reduce this but do not remove it.
- Scores are recomputed daily and can move a lot for small topics.
- GDELT's news counts mix two kinds of day: full-day totals from its history API (often rate-limited) and partial samples from its 15-minute files. Sampled days are left out of the rising rules' baselines, which makes news a slower signal for new topics.
- Bluesky's search counts are approximate (
hitsTotal), and topics whose names are everyday words (Rust, Python, Swift) are not searched there until they have an alias that names the tool.
11. Scoring v2 (in comparison)
Version 2 adds sources that measure what developers and AI builders actually do, and replaces the news-only saturation with content volume. From 2026-10-21 it is computed every day next to the published score (TrendScore.*Alt); the published score switches when SCORING_V2 is turned on after a week of side-by-side comparison. Until then the published (v1) score reads only the inputs it had before: Wikipedia, GDELT, arXiv, Hacker News, niche feeds, npm and Open Food Facts (v1Sources in config/scoring.ts).
| Source | Metric | Licence or terms |
|---|---|---|
| Hacker News Search (Algolia) | stories: stories whose title mentions the topic, per day | HN-Search-by-Algolia |
| Stack Overflow (Stack Exchange API) | questions: new questions in the topic's tag per day | CC-BY-SA-4.0 |
| GitHub | stars: stars gained per day by the topic's repository | GitHub-API-terms |
| Hugging Face | downloads_30d: 30-day downloads of the topic's models | HF-Hub-terms |
| PyPI (pypistats.org) | downloads_7d_mean: trailing 7-day mean of daily downloads | PyPI-CC |
| crates.io | downloads_7d_mean: trailing 7-day mean of daily downloads | crates.io-access-policy |
| Packagist | downloads_7d_mean: trailing 7-day mean of daily downloads | Packagist-API |
| RubyGems | downloads: daily downloads, from the change in the total | RubyGems-API |
| Homebrew | installs_30d: installs over the last 30 days | Homebrew-analytics |
| Bluesky | posts: posts mentioning the topic per day | Bluesky-API |
| OpenAlex | works: new research works per day | CC0-1.0 |
Repositories, Stack Overflow tags and package names come from the topic's Wikidata item (source code repository, Stack Exchange tag, PyPI, crates.io, Packagist, RubyGems and Homebrew properties), plus reviewed overrides. YouTube is not used for scoring: its API terms forbid derived metrics.
Trend v2 = 100 × ((1 − v2.audienceWeight = 0.2) × weighted velocity + 0.2 × audience percentile). Each source's centred growth (section 3: its last 3 days against its own 4-week median) is averaged within its independence group, so five package registries count as one signal, and the groups are weighted per niche:
| Group | Developer Tools | AI Tools |
|---|---|---|
| Wikipedia (reading) | v2.groupWeights.developer-tools.reading = 1 | v2.groupWeights.ai-tools.reading = 1 |
| Hacker News | v2.groupWeights.developer-tools.hacker-news = 1 | v2.groupWeights.ai-tools.hacker-news = 0.9 |
| GitHub | v2.groupWeights.developer-tools.github = 1 | v2.groupWeights.ai-tools.github = 0.8 |
| Package registries | v2.groupWeights.developer-tools.packages = 1 | v2.groupWeights.ai-tools.packages = 0.7 |
| Hugging Face | not used | v2.groupWeights.ai-tools.hugging-face = 1 |
| Stack Overflow | v2.groupWeights.developer-tools.questions = 0.8 | v2.groupWeights.ai-tools.questions = 0.4 |
| Bluesky | v2.groupWeights.developer-tools.social = 0.6 | v2.groupWeights.ai-tools.social = 0.7 |
| News | v2.groupWeights.developer-tools.news = 0.4 | v2.groupWeights.ai-tools.news = 0.5 |
| Research (arXiv, OpenAlex) | v2.groupWeights.developer-tools.research = 0.3 | v2.groupWeights.ai-tools.research = 0.6 |
Saturation v2 = the mean, over the content sources the niche has (Hacker News stories, news articles, Bluesky posts, DEV articles when enabled), of the topic's percentile of its last-v2.saturationDays = 30-day count within the niche. Topics with no coverage score 0 and are left out of the population, so a niche where most topics have no stories does not make an uncovered topic look crowded.
Opportunity v2 = Trend × (1 − Saturation) × (1 − PeakPenalty). PeakPenalty = 1 − (7-day mean of the main attention source ÷ its highest 7-day mean of the last v2.peakDays = 90 days), at most v2.peakPenaltyCap = 0.8, and only once that high is at least v2.peakMinAgeDays = 7 days old. Sensitive topics get no opportunity.
Why now is one line from the two strongest signals: the source furthest above its usual level (weighted by its group), then a lightly covered content source ("only 11 Hacker News stories in 30 days") or the next rising source.
Stages and the rising rules (sections 5 and 6) are the same in v2, evaluated over every source, so a topic can pass Rule A with, for example, GitHub stars and Stack Overflow questions. The comparison (trend, opportunity, saturation and list for every topic, and how many topics each version puts in Rising now each day) is at /status/scoring.
12. Track record
Every published Rising now list is stored permanently (RisingSnapshot) and never recomputed. /track-record scores each call for Developer Tools and AI Tools (lib/analytics/track-record.ts):
- A call is a topic's first day in Rising now with stage Emerging or Rising.
- It is a hit when, within
trackRecord.windowDays= 60 days, the topic reaches Viral, or its attention (the 7-day mean of its main source, Wikipedia pageviews where it has them) stays at or abovetrackRecord.liftRatio= 1.5 times its level on the call day fortrackRecord.sustainDays= 7 days running. - It is a miss when the window ends without a hit, and pending before that.
- Hit rate = hits ÷ (hits + misses). Lead time = days from the call to the day the hit was confirmed, averaged over hits.
- Calls made before the first stored list are a backtest: today's rules applied to the stored history, shown separately and labelled as such.