lilydjwg / nvchecker

New version checker for software releases
MIT License
439 stars 70 forks source link

Avoid making multiple requests to the same URL? #221

Closed guihkx closed 1 year ago

guihkx commented 1 year ago

If one or more entries share the exact same url attribute, would it be possible for nvchecker to only make a single HTTP request to that URL and then share/reuse that response body among other entries? Here's a practical example:

This page from NVIDIA's website lists a lot of driver versions, so let's suppose I want to monitor some of these versions.

My nvchecker config file would look like this:

[__config__]
oldver = "nvidia-old.json"
newver = "nvidia-new.json"

[NVIDIA_Production]
source = "regex"
regex = "Latest Production Branch Version: <a .+?>([\\d.]+)</a>"
url = "https://www.nvidia.com/en-us/drivers/unix/"

[NVIDIA_Latest]
source = "regex"
regex = "Latest New Feature Branch Version: <a .+?>([\\d.]+)</a>"
url = "https://www.nvidia.com/en-us/drivers/unix/"

[NVIDIA_Beta]
source = "regex"
regex = "Latest Beta Version: <a .+?>([\\d.]+)</a>"
url = "https://www.nvidia.com/en-us/drivers/unix/"

As you can see, these entries share the exact same URL. However, upon running nvchecker using that config, three distinct HTTP requests will be made to the exact same page:

$ nvchecker -c nvidia.toml -l debug
[D 03-26 05:52:10.320 selector_events:54] Using selector: EpollSelector
[D 03-26 05:52:10.322 curl_httpclient:516] GET https://www.nvidia.com/en-us/drivers/unix/
[D 03-26 05:52:10.322 curl_httpclient:516] GET https://www.nvidia.com/en-us/drivers/unix/
[D 03-26 05:52:10.323 curl_httpclient:516] GET https://www.nvidia.com/en-us/drivers/unix/
...

(I'm sorry if this is already possible using current options, but I couldn't find any)

lilydjwg commented 1 year ago

Good point, implemented!

guihkx commented 1 year ago

Oh my, that was fast! Thank you so much!