Note that if a request is made for all validators it still obtains data from the beacon state, as this will be faster due to SSZ encoding. As such, there is going to be a point at which fetching state would be faster than POSTing for the validators, however to make this work cleanly across relevant networks it's better to not attempt to create heuristics and instead go for this simple approach. Future changes to the beacon API may solve this problem.
Because this call can have a high impact for others I'm requesting comments and/or testing from users before merging. Pinging @moshe-blox @pk910 @y0sher @nkryuchkov .
Beacon nodes now support https://ethereum.github.io/beacon-APIs/#/Beacon/postStateValidators so this moves away from the chunk-based system to obtain all validators in a single request.
Note that if a request is made for all validators it still obtains data from the beacon state, as this will be faster due to SSZ encoding. As such, there is going to be a point at which fetching state would be faster than POSTing for the validators, however to make this work cleanly across relevant networks it's better to not attempt to create heuristics and instead go for this simple approach. Future changes to the beacon API may solve this problem.
Because this call can have a high impact for others I'm requesting comments and/or testing from users before merging. Pinging @moshe-blox @pk910 @y0sher @nkryuchkov .