[...] create a library that builds gateway reputation over time (sliding window) and changes the order of gateways, prioritizing the one that responds the fastest. [...]
ps. this extra heuristic is not specific to IPLD Explorer, the ipfs-geoip used on Peers screen in webui would benefit from the same thing – currently it tries gateways in serial.
Perhaps worth extracting to standalone project and reusing in both places? (follow-up issue/pr)
I think we should create a library for this, as I've been thinking about this since I starting working with IPFS, but always in the context of a flat "gateway-racer" that queries given gateways in parallel.
It would be much more useful to the entire ecosystem if we built a library that queried various endpoints (kubo node, network via helia, gateways via trustless-fetch, IPNI routing, etc) and kept a weight-sorted array that auto-adjusted based on response timings and success frequencies. Each consumer fetching content have their own latencies from various gateways/endpoints/etc, as well as various spikes to popular services that may be overloaded at a particular time.
A library that automatically corrects to the fastest, most successful content provider would reduce load hot-spots throughout the IPFS network while improving every consumer's experience.
I think this would make sense both in the "helia-fetch" library we've discussed creating a number of times and as a plugin/addon/consumer of that library.
I think we should create a library for this, as I've been thinking about this since I starting working with IPFS, but always in the context of a flat "gateway-racer" that queries given gateways in parallel.
It would be much more useful to the entire ecosystem if we built a library that queried various endpoints (kubo node, network via helia, gateways via trustless-fetch, IPNI routing, etc) and kept a weight-sorted array that auto-adjusted based on response timings and success frequencies. Each consumer fetching content have their own latencies from various gateways/endpoints/etc, as well as various spikes to popular services that may be overloaded at a particular time.
A library that automatically corrects to the fastest, most successful content provider would reduce load hot-spots throughout the IPFS network while improving every consumer's experience.
I think this would make sense both in the "helia-fetch" library we've discussed creating a number of times and as a plugin/addon/consumer of that library.
See https://github.com/ipfs/ipld-explorer-components/issues/379#issuecomment-1634297273 for more information about the origin of this issue.
cc @lidel @whizzzkid