The NYT is blocking the Internet Archive. That’s a mistake.
The New York Times has started blocking the Internet Archive’s web crawlers, going beyond standard robots.txt rules to cut off access entirely. The Guardian is doing the same. The stated reason: fear of AI companies scraping their content. But the Internet Archive isn’t an AI company — it’s a library, and it’s been one fo…
