wrapdb get_zip issue for modules that have no releases

[tp-m]

Just putting what's been reported on IRC into an issue here.

Problem

https://wrapdb.mesonbuild.com/v1/projects/chipmunk/6.2.2/1/get_zip returns an Internal Server Error page.

Server-side logs show wrapdb throwing an exception like this:

[2020-02-19 00:29:40,151] ERROR in app: Exception on /v1/projects/chipmunk/6.2.2/1/get_zip [GET]
Traceback (most recent call last):
  File "{basedir}/lib/python3.7/site-packages/mesonwrap-0.1.0-py3.7.egg/mesonwrap/githubdb.py", line 107, in _get_zip
    return _asset(org, project, branch, revision, PATCH_ZIP_LABEL)
  File "{basedir}/lib/python3.7/site-packages/cachetools/decorators.py", line 36, in wrapper
    v = func(*args, **kwargs)
  File "{basedir}/lib/python3.7/site-packages/mesonwrap-0.1.0-py3.7.egg/mesonwrap/githubdb.py", line 83, in _asset
    release = repo.get_release('{}-{}'.format(branch, revision))
  File "{basedir}/lib/python3.7/site-packages/github/Repository.py", line 2793, in get_release
    "GET", self.url + "/releases/tags/" + id
  File "{basedir}/lib/python3.7/site-packages/github/Requester.py", line 322, in requestJsonAndCheck
    verb, url, parameters, headers, input, self.__customConnection(url)
  File "{basedir}/lib/python3.7/site-packages/github/Requester.py", line 345, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.UnknownObjectException: 404 {"message": "Not Found", "documentation_url": "https://developer.github.com/v3/repos/releases/#get-a-release-by-tag-name"}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "{basedir}/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "{basedir}/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "{basedir}/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "{basedir}/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "{basedir}/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "{basedir}/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "{basedir}/lib/python3.7/site-packages/mesonwrap-0.1.0-py3.7.egg/wrapweb/api.py", line 84, in get_zip
    result = _database().get_zip(project, branch, revision)
  File "{basedir}/lib/python3.7/site-packages/mesonwrap-0.1.0-py3.7.egg/mesonwrap/githubdb.py", line 153, in get_zip
    return _get_zip(self._org, project, branch, revision)
  File "{basedir}/lib/python3.7/site-packages/mesonwrap-0.1.0-py3.7.egg/mesonwrap/githubdb.py", line 109, in _get_zip
    _log.error('get_zip(%s, %s, %d): %s', project, branch, revision, e)
NameError: name 'e' is not defined

It looks like this happens for modules that don't have releases on github for each branch in question, e.g. check or chipmunk.

I guess the old code path automatically created zips in that case?

[tp-m]

cc @legeana

[legeana]

I guess the old code path automatically created zips in that case?

No. The old code had all the data in its database. Most of it was migrated to a release form as a backup, unfortunately some of it was not. The new code is completely different, it just queries github. I will recreate the releases, unfortunately it will mean users will have to reinstall upstream.wrap files as the checksums will change.

[legeana]

@ignatenkobrain hey Igor, is there a way to retrieve the database file so that we can get the original wraps?

[tp-m]

Last I heard was that the server disappeared and no one has a backup of the database.

[2020-01-09T18:25] <ignatenkobrain> I think person who owns that hypervisor started to use that IP for something else…
[2020-01-09T18:25] <ignatenkobrain> or there is something seriously broken with that VM
[2020-01-12T12:22] <__tim> last I saw was ignatenkobrain saying he could make a database dump available
[2020-01-14T07:14] <@jpakkane> ignatenkobrain: what's the status on that?
[2020-01-14T07:25] <ignatenkobrain> jpakkane: https://send.firefox.com/download/c00ae51a1ac6465f/#LteSL6MaCUJ1N8fv2RtPmw (no idea what was in this link)
[2020-01-14T07:25] <ignatenkobrain> It seems that I lost that server entirely
[2020-01-14T07:56] <nirbheek> ignatenkobrain, ... you lost it?!
[2020-01-14T07:57] <nirbheek> ignatenkobrain, I mean, was there a disk failure or did it just disappear? :D
[2020-01-14T08:04] <ignatenkobrain> nirbheek: well, it was a VM on a server of my ex-ex-ex-ex-boss and apparently he did something with it and I don't have access there anymore.
[2020-01-14T08:04] <ignatenkobrain> I tried to ping him, but no luck so far

[legeana]

Alright I will just recreate them.

[legeana]

https://github.com/mesonbuild/chipmunk/releases/tag/6.2.2-1 is created. check doesn't have any releases.

[tp-m]

Thanks.

Should probably send an e-mail to the meson mailing list to announce the fact that the checksums have changed and why.