Separate DNS resolve from connect

temoto / robotstxt

The robots.txt exclusion protocol implementation for Go language

MIT License

269 stars 55 forks source link

Closed temoto closed 11 years ago

temoto commented 11 years ago

Separate resolve timeout
Limit concurrent resolves
Maybe cache IP addresses. Think about it, test a prototype. May have very low hit rate, so inefficient. Running local caching DNS server may be better.

temoto commented 11 years ago

whoops, wrong project