Last updated: 2026-07-23T00:12:25Z | Log entries analyzed: 173686 | Model: glm-5.2:cloud | Enrichment: Censys

Site Observatory — ai.rud.is

Report date: 2026-07-22 (reflecting the previous day’s state). Analysis window: 2026-03-10 through 2026-07-22.

What Changed

Incremental run for 2026-07-22. 534 new IPs appeared in the dataset.

New scanners: Eight newly-seen scanner IPs, led by 23.234.115.81 (81 requests in window, 7 today, python-requests/2.32.5) and 5.255.111.58 (134 requests in window, 6 today) spoofing an iPhone Safari UA. 169.150.201.35 made 6 requests today with 4 resulting in 404s, also via python-requests/2.32.5. A curl/8.0 probe from 212.135.214.5 and a Python/3.10 aiohttp/3.10.5 request from 101.43.145.79 round out the batch. Two null-UA IPs (2a01:111:f400:7e88::100, 52.112.95.134) appeared with 1-2 requests each.

New recon URIs: A single source IP fired 8 PHPUnit eval-stdin.php path variants — /wordpress/vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php, /site/vendor/..., /web/vendor/..., /symfony/vendor/..., /cms/vendor/..., /backend/vendor/..., /public/vendor/..., and /app/vendor/... — each hit 6 times. Classic CVE-2017-9841 exploitation attempt against a static site that has no PHP runtime. This is the kind of automated spray that generates noise but zero risk.

Volume trends (today vs. trailing 7-day average):

Fediverse, visitor, and scanner traffic all roughly doubled against baseline. other_crawler is the only class running below its 7-day average. The fediverse spike is notable — 3,475 requests in a single day from link-preview fetchers means something on the site got shared widely on the fediverse.

Canary triggers: /.env returned 200 to 93 distinct source IPs (189 total hits, 35,357 bytes served). /.git/config returned 200 to 56 IPs (76 hits, 912 bytes). These are decoy paths serving canned content — the scanners are being logged and measured, not exfiltrating anything real.

Classification gaps: Two bot-like UAs slipped through to the visitor class: Mozilla/5.0 (compatible; EnvScanner/1.0) (2 requests, 1 IP) and reasonix-web-fetch/1.0 (1 request, 1 IP). Both are obvious bots by UA string and should get classifier rules.

Traffic Summary

The full observation period (2026-03-10 through 2026-07-22) saw 173,686 requests from 27,980 unique IPs. The site served 3.8 GB total. Median response time is 4.52 ms with a p95 of 26.67 ms — this is a fast static site on Hetzner behind Caddy, and the numbers reflect that.

The signal-to-noise ratio is rough. Fediverse link-preview fetchers account for 49.0% of all traffic (85,156 requests from 6,018 IPs). Legitimate visitors are 23.1%. Everything else — AI crawlers (8.7%), scanners (7.0%), other crawlers (5.2%), RSS readers (3.6%), search crawlers (2.6%) — fills out the remainder. Owner traffic is 0.7%, intentionally excluded from analysis.

Resource Consumption

Fediverse fetchers consumed 2.5 GB — nearly two-thirds of all bandwidth — fetching / to generate link cards. These are automated single-request hits that pull the full homepage HTML. Bandwidth figures reflect bytes actually transferred; conditional/cached responses (304s) show 0 bytes, so logical content size is undercounted.

Temporal Patterns

The hourly distribution reveals two distinct internets operating on different clocks.

Fediverse traffic concentrates in UTC daytime hours 10-17, with a massive spike at hour 11 (13,321 requests) and hour 14 (7,813). This is the European afternoon — when Mastodon and Misskey users are active and sharing links, triggering link-preview fetches. The volume at hour 11 alone exceeds the entire daily total for many early days in the dataset.

Visitors (actual humans reading pages) peak in the late UTC evening: hours 22-23 see 2,645 and 3,768 requests respectively. Hour 0 holds strong at 4,063. This suggests a North American evening readership pattern — people reading after work on the US east coast and into the night.

Scanners are opportunistic but show clustering at hours 8 (1,718), 12 (963), 13 (1,323), 18 (1,371), 19 (1,581), 22 (1,013), and 23 (792). The hour 8 spike aligns with the start of the UTC business day — automated scanning infrastructure waking up. The evening cluster at 18-19 could be a second batch or different campaigns.

AI crawlers maintain a remarkably flat distribution — roughly 400-860 requests per hour with no dramatic peaks. They run 24/7 at a steady cadence, which is exactly what you’d expect from distributed crawling infrastructure.

By day of week, Tuesday is the scanner’s favorite day (6,040 requests, largely from the June 2 burst), while Saturday sees the most visitor traffic (8,311). Monday is the busiest day for fediverse activity (19,330 requests). Friday is the quietest day for visitors (3,318) — people have lives, apparently.

Content & Visitors

The most-read post by a wide margin is /posts/2026-06-27-running-ornith-locally-with-opencode-and-claude-code/ with 2,114 hits from 2,057 visitor IPs. The weekly bulletproof report from 2026-06-20 follows at 1,593 hits. The homepage (/) drew 2,556 hits but from only 1,306 unique IPs — many of those are fediverse link-preview fetches, not human reads.

Google is the dominant referrer (873 hits), followed by DuckDuckGo (150) and mastodon.social (74). Bing sent 62. The self-referrer rud.is (56 + 24) indicates internal navigation. Infosec.exchange (23) and phanpy.social (21) confirm the fediverse audience.

HTTP/3 adoption: Visitors made 1,676 requests over HTTP/3 — a small fraction of their 40,120 total but non-trivial. Owner traffic is overwhelmingly HTTP/3 (918 of 1,231). AI crawlers barely use it (2 requests); RSS readers have 5. The broader internet is still on HTTP/2 (41,438) or HTTP/1.1 (9,438). Caddy’s HTTP/3 support is there for those who want it, but most clients aren’t negotiating it.

Browser families: Chrome dominates (25,279 requests, 12,635 IPs), followed by Safari (5,805), Firefox (5,249), Edge (1,076), and Opera (194). The “Other” bucket (2,517) likely includes various bot UAs that didn’t match browser signatures.

RSS: 5,713 requests from 36 unique RSS reader IPs, with 331 additional RSS feed fetches from the visitor class (186 IPs). The feed audience is small but consistent — roughly 36 dedicated RSS IPs polling regularly since April 20.

Agent-Artifact Access Patterns

The site offers six agent-readable formats. Here’s what agents actually consume:

The markdown versions of posts are the clear winner — 2,019 requests from 1,098 IPs, transferring 6.9 MB. llms.txt gets moderate attention (80 requests, 66 IPs). The other four artifacts (identity.json, llms.html, ai.json, llm.txt) have been fetched exactly twice each, both times on 2026-06-12 from the same 2 IPs. These are essentially unused — either agents don’t know about them or they were one-time experiments.

Looking at who consumes agent artifacts: Bytespider leads with 142 requests across 21 artifacts. Baiduspider (91 requests, 37 artifacts) and PetalBot (88 requests, 28 artifacts) are the most artifact-hungry search crawlers. GPTBot fetched 38 requests across 32 artifacts — nearly one artifact per request, suggesting systematic retrieval. ClaudeBot similarly accessed 32 artifacts across 32 requests. Amazonbot hit 84 requests across 38 artifacts.

The top post .md file is /posts/2026-04-04-ollama-usage.md (89 requests, 84 IPs, 49 AI crawler hits). The observatory page itself (/posts/observatory.md) drew 78 requests with 770,932 bytes transferred — the largest single .md file by bandwidth, reflecting its length.

Compared to HTML page views: the top HTML post got 2,114 hits while the top .md got 89. The agent formats are being used, but at roughly 4% the volume of HTML. The designed-for-agents resources are reaching the agents they were built for — AI crawlers account for the majority of .md fetches — but the metadata artifacts (ai.json, identity.json, llms.html, llm.txt) are effectively dead. The bandwidth cost is modest: 6.9 MB for all .md files plus 307 KB for llms.txt across the entire observation period.

AI Crawler Activity

Twenty-three distinct AI crawler identifiers hit the site. The top performers:

Bytespider and ClaudeBot are the most aggressive by raw volume. ClaudeBot operates from only 92 IPs — concentrated infrastructure, likely Anthropic’s own crawling fleet. GPTBot is even tighter at 34 IPs but covers 354 unique pages, the broadest page coverage of any AI crawler. PetalBot’s 22 IPs fetching 218 pages suggests Huawei runs a small but efficient crawling operation.

For a small personal blog, 15,186 AI crawler requests across 3,596 IPs is substantial — nearly 9% of all traffic, and more than search engine crawlers combined. The site is being ingested into multiple LLM training and retrieval pipelines whether the owner likes it or not. The Other AI bucket (2,700 requests, 1,006 IPs) likely contains less-identifiable AI scrapers or new entrants that haven’t been fingerprinted yet.

Search Engine Crawlers

Google and Bing are both actively crawling — Googlebot hit 303 unique pages from 209 IPs, Bingbot covered 248 pages from 303 IPs. Baiduspider is surprisingly thorough (317 unique pages from 311 IPs), more page coverage than Google. DuckDuckBot is lighter at 71 pages. Yandex is minimal (31 pages) and hasn’t been seen since July 19. The site is being well-indexed by the major Western engines and Baidu; Yandex is an afterthought.

Fediverse Activity

The fediverse is the single largest traffic source for this site. 85,156 requests from 6,018 unique IPs, all fetching / (and occasionally other paths) to generate link-preview cards.

The 20 most active instances are a mix of Mastodon and Misskey installations:

• Misskey/2025.4.7 (evy.pet) — 75 requests, 2 IPs, 11 unique URIs
• Misskey/2025.4.6 (transfem.social) — 74 requests, 1 IP, 11 URIs
• Misskey/2025.4.7 (social.louis-vallat.dev) — 71 requests, 2 IPs, 8 URIs
• Mastodon/4.2.29 (mastodon.n41.lat) — 70 requests, 1 IP, 16 URIs
• Mastodon/4.2.17 (social.medienzentrum-hdh.de) — 69 requests, 2 IPs, 16 URIs

Most Mastodon instances fetch 16 unique URIs (the typical link-preview pattern: / plus 15 post URLs). Misskey instances tend to fetch fewer URIs (1-11). Each instance typically uses 1-2 IPs — these are different servers, not one reader. The fediverse audience for this site is real, distributed, and active.

Scanner & Recon Activity

12,200 scanner requests from 478 unique IPs over the observation period. The all-time leader is 45.148.10.95 (TLM-Audit-Scanner/1.0) with 3,816 requests and 1,704 404s across 636 unique URIs on June 2 — a single-day blitz. 185.219.151.78 followed on June 27-28 with 1,914 requests spoofing Firefox 129.

Today’s new scanner activity is lighter. The PHPUnit eval-stdin.php spray from a single IP across 8 directory prefixes is the most notable new recon pattern — a CVE-2017-9841 attempt that’s been circulating in scanner toolkits for years. It hit 6 times per path, all 404.

The top recon URIs across the full period:

The .env hunting is persistent — 67 hits from 41 distinct IPs, plus variants at /api/.env, /backend/.env, /app/.env, /public/.env, /.env.local. These are automated tools working through a checklist of common paths. The /nuclei.svg request is a Nuclei scanner template probe. /wp-admin/admin-ajax.php from a single IP (103 hits) is someone who really wants this to be a WordPress site.

Enrichment context: 100.57.165.217 (Censys: benign, ASN14618 Amazon-AES, United States) made 180 scanner requests with 176 404s, using a Linux Chrome UA. It’s an AWS host with no exposed services per Censys — likely a scanner running from an EC2 instance. 159.195.14.200 (Censys: benign, ASN214996 netcup GmbH, United States) has been seen 12 days, runs SMTP/HTTP/IMAP services, and appears to be a self-hosted mail/web server that occasionally probes. Its rDNS and service profile suggest a self-hoster, not a dedicated scanning operation.

5.255.111.58, one of today’s new scanners with an iPhone Safari spoof UA, has no enrichment line — Censys data was unavailable for this IP.

IP Persistence

Twenty IPs appear across 11 or more days. The most persistent:

43.155.125.36 (25 days, 32 requests) and the 43.155.125.* / 43.135.107.* range suggest Tencent Cloud IPs — possibly an RSS reader or monitoring service polling from a fixed cloud IP. 70.8.208.88 with 284 requests over 19 days is a more active reader or a self-hosted aggregation service. 159.195.14.200 (netcup GmbH, 12 days, 15 requests) is the self-hoster noted in the scanner section — low-frequency, benign.

IP persistence is an approximation. IPv6 privacy extensions and NAT mean IP does not equal individual identity. These are “IPs that kept coming back,” not necessarily “repeat visitors” in the human sense.

Observations

1. The fediverse is eating this site alive. 49% of all traffic and 64.6% of all bandwidth is Mastodon/Misskey instances fetching / to render link previews. Each time someone shares a link on the fediverse, an instance fetches the homepage. The site got shared a lot on July 22 (3,475 fediverse requests, 2.27× the 7-day average). This is a real audience signal — but it’s also 2.5 GB of bandwidth spent serving homepage HTML to robots that discard everything except the <meta> tags. If this trend continues, caching the link-preview-relevant headers separately or serving a lightweight <head>-only response to known fediverse UAs would cut bandwidth dramatically.

2. The canary traps are working as intended. /.env has been served as a 200 to 93 distinct IPs (189 total hits) and /.git/config to 56 IPs (76 hits). These decoy paths are doing their job — scanners are finding them, fetching them, and getting canned content. The 35 KB served to .env hunters is a small price for the intelligence on which scanner IPs are in circulation. The new PHPUnit eval-stdin.php spray is the same pattern: automated tools working a stale CVE checklist against a site that doesn’t even run PHP.

3. Two classification gaps need rules. EnvScanner/1.0 and reasonix-web-fetch/1.0 both landed in the visitor bucket because their UAs didn’t match existing patterns. EnvScanner/1.0 is self-identifying as a scanner in its name. reasonix-web-fetch/1.0 is a custom fetcher UA. Both should be added to the classifier — the first as a scanner, the second likely as an other_crawler or ai_crawler depending on what it actually does. The Other AI bucket (2,700 requests, 1,006 IPs) also suggests there are AI crawlers in the wild that haven’t been fingerprinted into named categories yet.

Generated by observatory.sh — Caddy logs → DuckDB → Censys → Ollama → Astro