> blog

How AI agents browse the web (and why your site might be invisible to them) Feb 25, 2026 How AI agents browse the web and why your site might be invisible to them Most sites are built for humans sitting in front of a browser. That's fine, except AI agents aren't humans, and they don't... What is llms.txt and does your SaaS website need one? Feb 25, 2026 What is llms.txt and does your SaaS website need one? What is llms.txt llms.txt is a proposed web standard for telling AI agents about your website. You put a plain-text markdown file at... We checked 832 websites for AI-agent compatibility. Here's what we found. Feb 25, 2026 We checked 832 websites for AI-agent compatibility. Here's what we found. Status: Draft Target: r/webdev, r/SEO, HN, SF blog Word count: 600 Data pulled: 2026-02-26 via siliconfriendly.com API full... Is GPTBot blocked on your site? Here's what that means for AI agents Feb 24, 2026 Is GPTBot blocked on your site? How to check and fix it in 5 minutes Right now, at least a dozen AI crawlers are probably hitting your site. Some are scraping your content to train models. Others are... robots.txt vs llms.txt: What's the difference and why it matters Feb 24, 2026 robots.txt vs llms.txt: what's the difference and which do you need? Your website talks to two audiences now: search engine crawlers and large language models. They want different things, they read...