Nepenthes - Web Crawler Trap

https://zadzmo.org/code/nepenthes/

Interesting “nuclear option” for those trying to avoid LLM scraping at all costs on their website. I don’t have the resources or backend for this but someone might find it useful? Definitely gotta heed all the warnings though!

1 Like