search

scikit-build / github-release

Manage github releases from the command line
Apache License 2.0
56 stars 22 forks source link

github-asset fails to upload #1

Closed jnweiger closed 2 years ago

jnweiger commented 9 years ago

Affter pip install githubrelease, I am trying to upload a new file into an existing release tag. I get the following error:

github-asset owncloud/vm upload v8.1.2-16.1-201509032039 ./img/xUbuntu_14.04-owncloud-8.1.2-16.1-201509032039-DEBUG.vmx.zip
release v8.1.2-16.1-201509032039: uploading ./img/xUbuntu_14.04-owncloud-8.1.2-16.1-201509032039-DEBUG.vmx.zip
Traceback (most recent call last):
  File "/usr/bin/github-asset", line 9, in 
    load_entry_point('githubrelease==0.0.0', 'console_scripts', 'github-asset')()
  File "/usr/lib/python2.7/site-packages/github_release.py", line 183, in gh_asset
    return handle_http_error(lambda: commands[args.pop(1)](*args))
  File "/usr/lib/python2.7/site-packages/github_release.py", line 187, in handle_http_error
    func()
  File "/usr/lib/python2.7/site-packages/github_release.py", line 183, in 
    return handle_http_error(lambda: commands[args.pop(1)](*args))
  File "/usr/lib/python2.7/site-packages/github_release.py", line 126, in gh_asset_upload
    data=f.read())
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 438, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 328, in send
    raise SSLError(e)
requests.exceptions.SSLError: hostname 'uploads.github.com' doesn't match either of '*.githubusercontent.com', 'githubusercontent.com'
jnweiger commented 9 years ago

Workaround: 

--- xxx 2015-09-04 01:16:09.704558643 +0200
+++ /usr/lib/python2.7/site-packages/github_release.py  2015-09-04 01:11:46.750523838 +0200
@@ -123,7 +123,7 @@
             response = request('POST', 
                 'https://uploads.github.com/repos/{0}/releases/{1}/assets?name={2}'.format(repo_name, release['id'], basename),
                 headers={'Content-Type':'application/octet-stream'},
-                data=f.read())
+                data=f.read(), verify=False)
             response.raise_for_status()

 def gh_asset_erase(repo_name, tag_name, pattern):
j0057 commented 9 years ago

Hey Jürgen, I noticed the same thing, but I haven't yet found out what's causing it. It used to work ;-) and comparing Github's API docs to the code, the bug doesn't immediately jump out at me.

j0057 commented 9 years ago

verify=False would indicate that it's an SSL certificate problem, right?

j0057 commented 9 years ago

Qualys SSL labs gives an A+ rating: https://www.ssllabs.com/ssltest/analyze.html?d=uploads.github.com

j0057 commented 9 years ago

OK, I dove into it some more. I created a release, which was overdue:

env/bin/github-release j0057/setuptools-version-command create 2.2
env/bin/github-release j0057/setuptools-version-command debug 2.2
env/bin/github-asset j0057/setuptools-version-command upload 2.2 /var/lib/pypkg27/setuptools-version-command-2.2.tar.gz 
env/bin/github-asset j0057/setuptools-version-command upload 2.2 /var/lib/pypkg27/setuptools_version_command-2.2-py2.py3-none-any.whl 
env/bin/github-release j0057/setuptools-version-command release-notes 2.2
env/bin/github-release j0057/setuptools-version-command publish 2.2
twine upload /var/lib/pypkg27/setuptools-version-command-2.2.tar.gz 
twine upload /var/lib/pypkg27/setuptools_version_command-2.2-py2.py3-none-any.whl

And as you can see on github and on PyPI, it worked.

I guess what I'm saying is: I can't reproduce it, but I agree there is a bug, because I saw it happen too. I hate bugs that magically disappear... :suspect:

jnweiger commented 9 years ago

Hmm, I am running into more issues, maybe I just don't have the proper setup? The SSL certificate problem is handled by verify=False, but this leads to warnings. And then it fails. I admit, it's 600MB, maybe that is too much? The same upload via web-interface succeeds from one machine, but not from another.

uploading ./img/xUbuntu_14.04-owncloud-8.1.2-17.1-201509072043-DEBUG.raw.zip
/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object
 is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.  InsecurePlatformWarning
release v8.1.2-17.1-201509072043: uploading ./img/xUbuntu_14.04-owncloud8.1.2-17.1-201509072043-DEBUG.raw.zip
/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning /usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
Error sending POST to https://uploads.github.com/repos/owncloud/vm/releases/1779018/assets?name=xUbuntu_14.04-owncloud-8.1.2-17.1-201509072043-DEBUG.raw.zip
< POST /repos/owncloud/vm/releases/1779018/assets?name=xUbuntu_14.04-owncloud-8.1.2-17.1-201509072043-DEBUG.raw.zip
< Accept : */*
< Accept-Encoding : gzip, deflate
< Connection : keep-alive
< Content-Length : 660807496
< Content-Type : application/octet-stream
< User-Agent : python-requests/2.7.0 CPython/2.7.6 Linux/3.11.10-29-desktop
< 
< 'PK\x03\x04-\x00\x00\x00\x08\x00Q\xa6\'G\x8cH+"\xff\xff\xff\xff\xff\xff\xff\xff8\x000\x00xUbun' (total 660807496 bytes of data)

> 404 Not Found
> Cache-Control : no-cache
> Content-Length : 129            
> 404 Not Found
> Cache-Control : no-cache
> Content-Length : 129
> Content-Security-Policy : default-src 'none'
> Content-Type : application/json; charset=utf-8
> Date : Mon, 07 Sep 2015 21:43:00 GMT
> Strict-Transport-Security : max-age=31557600
> X-Content-Type-Options : nosniff
> X-Frame-Options : deny
> X-Github-Media-Type : github.v3
> X-Github-Request-Id : 6916f9a0-55a9-11e5-8657-a4cbd6e71a9a
> X-Xss-Protection : 1; mode=block
>
> '{"message":"Not Found","request_id"' (total 129 bytes of data)
- done
j0057 commented 9 years ago

So in the error you originally posted, you got presented a certificate for githubusercontent.com, but connected to uploads.github.com.

In the second error which you just posted, there are SSL warnings, but ultimately the connection goes through, only the URL posted to appears to be wrong -- or maybe there's a problem with the .netrc?

I tested yesterday against Python 2.7.9 and requests 2.7.0. If I read the documentation of the warning right, upgrading to Python 2.7.9 should fix at least the InsecurePlatformWarnings, is that possible for you? It sounds like it could be related to the original error as well, like if the OpenSSL library you use doesn't support SNI, you could unexpectedly get presented a different certificate.

But I've a feeling that upgrading Python won't make the 404 go away.

Can you maybe upgrade to githubrelease 1.4.3a0 and post the output of github-release owncloud/vm debug $tag -- where $tag is the release version you're creating? That should tell us if it's trying to post to the correct URL or not.

jcfr commented 7 years ago

@jnweiger Let us know if this is still a problem. In the mean time, I am closing this issue.

Flamefire commented 6 years ago

I just ran into the same problem:

Traceback (most recent call last): File "/home/travis/.local/bin/githubrelease", line 11, in sys.exit(main()) File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 764, in call return self.main(args, kwargs) File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/travis/.local/lib/python2.7/site-packages/click/core.py", line 555, in invoke return callback(args, kwargs) File "/home/travis/.local/lib/python2.7/site-packages/click/decorators.py", line 27, in new_func return f(get_current_context().obj, *args, *kwargs) File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 631, in _cli_asset_upload gh_asset_upload(args, kwargs) File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 116, in with_check_for_credentials return func(*args, *kwargs) File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 760, in gh_asset_upload repo_name, tag_name, upload_url, filename, verbose, dry_run) File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 704, in _upload_release_file data=_ProgressFileReader(f, reporter)) File "/home/travis/.local/lib/python2.7/site-packages/github_release.py", line 65, in _request response = request(args, kwargs) File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request return session.request(method=method, url=url, kwargs) File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 455, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 558, in send r = adapter.send(request, kwargs) File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 385, in send raise SSLError(e) requests.exceptions.SSLError: hostname 'uploads.github.com' doesn't match either of '*.githubusercontent.com', 'githubusercontent.com'

Can you reopen this?

jcfr commented 6 years ago

@Flamefire Thanks for reporting the issue. Could you provide more details about the context leading to the error ?

Flamefire commented 6 years ago

I'm using this on travis: https://travis-ci.com/Flamefire/release-tests/jobs/151523149

Flamefire commented 6 years ago

It seems to work for newer python (3.6): https://travis-ci.com/Flamefire/release-tests/jobs/151600878

jcfr commented 6 years ago

2.7.x (trusty default of travis

Thanks for the additional details. I am trying to reproduce now.

jcfr commented 6 years ago

I'm using this on travis: https://travis-ci.com/Flamefire/release-tests/jobs/151523149

  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 558, in send
    r = adapter.send(request, **kwargs)

  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 385, in send
    raise SSLError(e)
requests.exceptions.SSLError: hostname 'uploads.github.com' doesn't match either of '*.githubusercontent.com', 'githubusercontent.com'

I tried to reproduce the problem downloading the docker image associated with the build reporting a failure but I couldn't reproduce the problem.

image

docker pull travisci/ci-garnet:packer-1512502276-986baf0
docker run -ti --rm travisci/ci-garnet:packer-1512502276-986baf0

And then execute the following without problem:

TRAVIS_REPO_SLUG=jcfr/ci-sandbox
TRAVIS_COMMIT=e73f4efd30e26def13cf1550998bed76dab08599
export GITHUB_TOKEN=XXXXXXXXXXXXX

/opt/python/2.7/bin/pip install githubrelease
githubrelease=/opt/python/2.7/bin/githubrelease

mkdir /work && cd work/
$githubrelease release $TRAVIS_REPO_SLUG create "Release-$TRAVIS_COMMIT" --name "Release for $TRAVIS_COMMIT" --body "BODY cool!" --target-commitish $TRAVIS_COMMIT

$githubrelease release $TRAVIS_REPO_SLUG create "Release-$TRAVIS_COMMIT" --name "Release for $TRAVIS_COMMIT" --body "BODY cool!" --target-commitish $TRAVIS_COMMIT

TRAVIS_OS_NAME=linux
echo "Foo" > $TRAVIS_COMMIT-$TRAVIS_OS_NAME.tar.gz

$githubrelease asset $TRAVIS_REPO_SLUG upload "Release-$TRAVIS_COMMIT" $TRAVIS_COMMIT-$TRAVIS_OS_NAME.tar.gz
jcfr commented 6 years ago

Is this consistently reproducible ? If you are restart the build, does it happen again ?

Flamefire commented 6 years ago

Yes it is. I made a separate test repo for this (https://travis-ci.com/Flamefire/travis_test) and reported this to the travis team (https://github.com/travis-ci/travis-ci/issues/10233)

I'm not very familiar with SSL certificates but I think that there might be some caching breaking this (Github reports wrong certificate, travis stores this and fails to use a newer one) or that it is due to the issues with SSL on Python 2.7:

Collecting githubrelease /home/travis/.local/lib/python2.7/site-packages/pip/vendor/urllib3/util/ssl.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings SNIMissingWarning /home/travis/.local/lib/python2.7/site-packages/pip/vendor/urllib3/util/ssl.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecurePlatformWarning /home/travis/.local/lib/python2.7/site-packages/pip/vendor/urllib3/util/ssl.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecurePlatformWarning

jcfr commented 2 years ago

Closing. We will be transitioning the continuous integration to GitHub Actions (or CircleCI)