ipol-journal / ipolDevel

IPOL demo system development
GNU Affero General Public License v3.0
23 stars 3 forks source link

Failed to download sources if the HTTP server doesn't send a last-modified tag #60

Closed mcolom closed 7 years ago

mcolom commented 7 years ago

Example of URL which failed and gitHub headers: 13:07:30 ~$ curl -X HEAD -i https://github.com/npd/DCTdenoising/archive/master.zip Warning: Setting custom HTTP method to HEAD with -X/--request may not work the Warning: way you want. Consider using -I/--head instead. HTTP/1.1 302 Found Server: GitHub.com Date: Thu, 05 Jan 2017 12:07:53 GMT Content-Type: text/html; charset=utf-8 Status: 302 Found Cache-Control: no-cache Vary: X-PJAX Location: https://codeload.github.com/npd/DCTdenoising/zip/master X-UA-Compatible: IE=Edge,chrome=1 Set-Cookie: logged_in=no; domain=.github.com; path=/; expires=Mon, 05 Jan 2037 12:07:53 -0000; secure; HttpOnly Set-Cookie: _gh_sess=eyJzZXNzaW9uX2lkIjoiODZhYzg0NGY4ZGVmMzZkMTE5NzI4NzU1YTY1OGM1MjkiLCJzcHlfcmVwbyI6Im5wZC9EQ1RkZW5vaXNpbmciLCJzcHlfcmVwb19hdCI6MTQ4MzYxODA3M30%3D--8be34e5f6c9fc4d9df35c95fa16ad1350fc9d16f; path=/; secure; HttpOnly X-Request-Id: 2d07e2ddcdacdddf3984a643314fa4a7 X-Runtime: 0.015420 Content-Security-Policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src render.githubusercontent.com; connect-src 'self' uploads.github.com status.github.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com wss://live.github.com; font-src assets-cdn.github.com; form-action 'self' github.com gist.github.com; frame-ancestors 'none'; frame-src render.githubusercontent.com; img-src 'self' data: assets-cdn.github.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; media-src 'none'; script-src assets-cdn.github.com; style-src 'unsafe-inline' assets-cdn.github.com Strict-Transport-Security: max-age=31536000; includeSubdomains; preload Public-Key-Pins: max-age=5184000; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="RRM1dGqnDFsCJXBTHky16vi1obOlCgFFn/yOhI/y+ho="; pin-sha256="k2v657xBsOVe1PQRwOsHsw3bsGT2VzIqz5K+59sNQws="; pin-sha256="K87oWBWM9UZfyddvDfoxL+8lpNyoUB2ptGtn0fv6G2Q="; pin-sha256="IQBnNBEiFuhj+8x6X8XLgh01V9Ic5/V3IRQLNFFc7v4="; pin-sha256="iie1VXtL7HzAMF+/PVPR9xzT80kQxdZeJ+zduCB3uj0="; pin-sha256="LvRiGEjRqfzurezaWuj8Wie2gyHMrW5Q06LspMnox7A="; includeSubDomains X-Content-Type-Options: nosniff X-Frame-Options: deny X-XSS-Protection: 1; mode=block Vary: Accept-Encoding X-Served-By: cd77d0e314015a41e6ff7ecd541d504d X-GitHub-Request-Id: 8AE75057:2DABD:6436301:586E3719

DR Traceback: 2017-01-05 12:56:07 ERROR in ensure_compilation Traceback (most recent call last): File "/home/ipol/ipolDevel/ipol_demo/modules/demorunner/demorunner.py", line 554, in ensure_compilation make_info = self.make_new(path_for_the_compilation, build_params) File "/home/ipol/ipolDevel/ipol_demo/modules/demorunner/demorunner.py", line 384, in make_new extract_needed = build.download(url, tgz_file) File "/home/ipol/ipolDevel/ipol_demo/modules/demorunner/Tools/build.py", line 40, in download url_ctime = time.strptime(url_handle.info()['last-modified'], File "/usr/lib/python2.7/rfc822.py", line 388, in getitem return self.dict[name.lower()] KeyError: 'last-modified'

mcolom commented 7 years ago

Fixed.