Posts In Category "AI"
How To Make a Website Readable for AI
It’s now more important than ever to make sure that AI can read and understand websites. AI Overviews from Google are lowering clicks by several percentage points among other metrics. This is not to mention an insight from Seer Interactive finding a 70% drop in CTR (click-through rate) when AI Overview is present on a […]
Read More →
Cloudflare’s New Block on AI Web Crawlers
Cloudflare announced that a setting to block AI web crawlers would be enabled by default for its customers. This announcement comes almost two years after the setting was originally introduced to protect content creators and prioritize important traffic.
At first, the setting only worked for web crawlers respecting robots.txt, which many AI web crawlers are starting to ignore (see below). Some AI web crawlers also mask themselves as being from search engines. Because of these issues, Cloudflare later updated the feature to apply to any detected AI web crawler and allowed a total ban on them.
Read More →