search

cloudmatrix / esky

an auto-update framework for frozen python apps
BSD 3-Clause "New" or "Revised" License
363 stars 74 forks source link

Infinite loop upon update #6

Closed vkosh closed 12 years ago

vkosh commented 12 years ago

There's a number of possibilities to achieve infinite update behaviour in DefaultVersionFinder.fetch_version_iter: request for update -> error downloading or error saving update (exception in DefaultVersionFinder._fetch_file_iter) -> retry

For example the cases may be: 1) Request to index returned file that's not available on update server for some reason. E.g. index file generated from DB and doesn't sync (even for a short time) with file system on update server. 2) Remote user machine hasn't enough space to save downloaded update.

In my case one client of thousands tries to update with such a message in webserver log (about 10 requests per second along hours): xx.xx.xx.xx - - "GET /updates/app-0.1.win32.0.2.patch HTTP/1.1" 200 36300 "-" "Python-urllib/3.2" The patch size is more than 200Kb, but every time there is 36300 bytes in log. That's really strange. May be there's a sort of race condition with urllib on windows or download timeout (but it's set to default on client). And the really frustrating is that I can't do anything with this client. Producing 403, 404 or "connection refused" for him on this patch doesn't do the thing, because open_url in _fetch_file_iter leads to exception raising (descendant of EnvironmentError) in urllib and retrying to download the patch.

So I've desided to do two things in custom implementation of version finder: 1) replace urllib with tornado http client (which's used in my app), 2) limit attempts and intervals of downloading update per update session.

But I have no ideas how to fix correctly this esky behaviour on DefaultVersionFinder level.

rfk commented 12 years ago

Ah, good catch, thanks for doing the necessary detective work on this one. I've changed things so that any failure to download a file will remove it from the version graph, which should avoid the possibility of an infinite loop. Are you able to try out the latest trunk and see if it fixes the problem?

vkosh commented 12 years ago

Yep, the latest changes fixes the problem. Great, thanks! I've tested it with different kind of possible http problems (wrong index, network errors during download, http errors) but it should fix all other cases too. I'll test it in production soon (in 1-2 weeks) and give you feedback.

rfk commented 12 years ago

I'm releasing the changes in version 0.9.7 and marking this issue closed. Thanks again for the report!