zk server polls springrts.com way to often

[abma]

37.59.49.101 - - [23/Sep/2016:08:16:32 +0200] "GET /versions.gz HTTP/1.1" 304 110 "-" "-" 37.59.49.101 - - [23/Sep/2016:08:16:40 +0200] "GET /versions.gz HTTP/1.1" 304 110 "-" "-" 37.59.49.101 - - [23/Sep/2016:08:16:52 +0200] "GET /versions.gz HTTP/1.1" 304 110 "-" "-" 37.59.49.101 - - [23/Sep/2016:08:17:00 +0200] "GET /versions.gz HTTP/1.1" 304 110 "-" "-" 37.59.49.101 - - [23/Sep/2016:08:17:12 +0200] "GET /versions.gz HTTP/1.1" 304 110 "-" "-"

WTF??

(this requests are at made to all repos, means ~20 requests every 10 seconds)

fix ASAP!

(or i will add some rate limiting)

[abma]

https://github.com/ZeroK-RTS/Zero-K-Infrastructure/blob/e76c893a85c6d7594cc4e9afaa1445d6dd282017/AutoRegistrator/AutoRegistrator.cs#L75

[Licho1]

There are two instances, both test and live, what do you think is a reasonable period?

[Licho1]

Is once per 2 minutes (6x lower rate) ok ?

[Licho1]

Oh and server queries all repos, because master does not change. In the past master changed modified since and this was a signal to refresh all repos. But now it sometimes does not change therefore it refreshes all repos.

Is the master sometimes not updating a bug or feature? If its a bug i can revert it and let it check repos only if master changes.

[abma]

Is once per 2 minutes (6x lower rate) ok ?

yeah, thats ok. where is it used that so that it polls this often?

Is the master sometimes not updating a bug or feature? If its a bug i can revert it and let it check repos only if master changes.

what do you mean with "master"? version.gz is only updated when changed.

[Licho1]

By master I mean repos.gz. Does it change when any version.gz changes?

And btw it does not send full request, it sends "if-modified-since" and you can see server returning 304 - not modified. It's not actually downloading stuff, just asking the server.

[abma]

By master I mean repos.gz. Does it change when any version.gz changes?

no.

where is it used that so that it polls this often?

And btw it does not send full request, it sends "if-modified-since" and you can see server returning 304 - not modified. It's not actually downloading stuff, just asking the server.

i'm aware of that, but polling every 20s isn't needed for sure. It just fills server logfiles / makes debugging difficult / uses resources cpu/network resources.

[Licho1]

If it worked as it was originally designed to work, there would be just one request instead of 20 - going to repos.gz Oiginally we had separate file with hash, you dropped this and we switched to if-modified-since but there is no use if repos.gz isnt refreshed...

[abma]

If it worked as it was originally designed to work, there would be just one request instead of 20 - going to repos.gz Oiginally we had separate file with hash, you dropped this and we switched to if-modified-since but there is no use if repos.gz isnt refreshed...

repos.gz can't be refreshed when a repository is on a foreign server, so i'm sure thats not how it was designed to work.

[Licho1]

All server were meant to be managed through central one. At one time we had regex matchers in the master and single repo could contain multiple tags. Is it now certain that for example evo repo contains only evo: something stuff? Or can anything be anywhere? If it's certain, there can be some optimization to not refresh everything in existence.

[abma]

Is it now certain that for example evo repo contains only evo: something stuff?

basicly yes: https://github.com/spring/pr-downloader/blob/master/src/Downloader/Rapid/RapidDownloader.cpp#L240

pr-downloader refreshes all repos when the short name doesn't match for any repo.

you still didn't answer my question.

where is it used that so that it polls this often?

[Licho1]

It's autoregistrator, it fetches rapid things, extracts modinfo, extracts dependencies, stores tag<->name mapping in zk database, updates mission references and prepares steam package.

It needs to refresh rapid to detect that new ZK stable has been released, update missions for it, extract modoptions etc.