internetarchive / Zeno

State-of-the-art web crawler 🔱
GNU Affero General Public License v3.0
81 stars 10 forks source link

Remove `runtime.Gosched()` in polling #105

Closed yzqzss closed 3 months ago

yzqzss commented 3 months ago

revert #101

Do not use runtime.Gosched() in loop, which will spinning your CPUs time for endless coroutine context switching. It won't magically turn polling into blocking.

In addition, Golang's coroutines are preemptive since Go1.14. There will be no starving coroutines, we don't need to manually call runtime.Gosched() nowadays.

equals215 commented 3 months ago

Yeah revet gosched, lgtm