jupyterhub / binderhub

Run your code in the cloud, with technology so advanced, it feels like magic!
https://binderhub.readthedocs.io
BSD 3-Clause "New" or "Revised" License
2.54k stars 388 forks source link

ci: test failures again related to dataverse #1656

Closed consideRatio closed 1 year ago

consideRatio commented 1 year ago

Test failure seen in https://github.com/jupyterhub/binderhub/actions/runs/4649890726/jobs/8228546993 for example.

=================================== FAILURES ===================================
_ test_dataverse[10.25346/S6/DE95RT-10.25346/S6/DE95RT-20460\\.v\\d+\\.\\d+$-https://doi.org/10.25346/S6/DE95RT-dataverse-s6-2fde95rt] _

spec = '10.25346/S6/DE95RT', resolved_spec = '10.25346/S6/DE95RT'
resolved_ref = '20460\\.v\\d+\\.\\d+$'
resolved_ref_url = 'https://doi.org/10.25346/S6/DE95RT'
build_slug = 'dataverse-s6-2fde95rt'

    @pytest.mark.parametrize(
        # "10.7910/DVN/TJCLKP" is a DOI associated with all versions of the
        # dataverse dataset, including the latest version and previous versions (v3,
        # v2, etc). Dataverse doesn't mint DOIs for each version of a dataset, but
        # that has been discussed in https://github.com/IQSS/dataverse/issues/4499
        #
        "spec,resolved_spec,resolved_ref,resolved_ref_url,build_slug",
        [
            [
                "10.7910/DVN/TJCLKP",
                "10.7910/DVN/TJCLKP",
                r"3035124\.v\d+\.\d+$",
                "https://doi.org/10.7910/DVN/TJCLKP",
                "dataverse-dvn-2ftjclkp",
            ],
            [
                "10.25346/S6/DE95RT",
                "10.25346/S6/DE95RT",
                r"20460\.v\d+\.\d+$",
                "https://doi.org/10.25346/S6/DE95RT",
                "dataverse-s6-2fde95rt",
            ],
        ],
    )
    async def test_dataverse(
        spec, resolved_spec, resolved_ref, resolved_ref_url, build_slug
    ):
        provider = DataverseProvider(spec=spec)

        # have to resolve the ref first
>       ref = await provider.get_resolved_ref()

binderhub/tests/test_repoproviders.py:199: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
binderhub/repoproviders.py:336: in get_resolved_ref
    r = await client.fetch(req)
binderhub/tests/utils.py:87: in fetch
    raise error
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <binderhub.tests.utils.MockAsyncHTTPClient object at 0x7effa7601910>
req_or_url = <tornado.httpclient.HTTPRequest object at 0x7effa75b75d0>
args = (), kwargs = {}
request = <tornado.httpclient.HTTPRequest object at 0x7effa75b75d0>
url_key = 'https://doi.org/10.25346/S6/DE95RT'
fetch = <bound method AsyncHTTPClient.fetch of <binderhub.tests.utils.MockAsyncHTTPClient object at 0x7effa7601910>>
error = HTTP 503: Service Unavailable
response = HTTPResponse(_body=b'<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>503 Service Unavailable<..., 'appconnect': 0.388015, 'pretransfer': 0.388144, 'starttransfer': 0.921252, 'total': 0.92[177](https://github.com/jupyterhub/binderhub/actions/runs/4649890726/jobs/8228546993#step:29:178)8, 'redirect': 0.563803})

    async def fetch(self, req_or_url, *args, **kwargs):
        """Mocked HTTP fetch

        If the request URL is in self.mocks, build a response from the cached response.
        Otherwise, run the actual request and store the response in self.records.
        """
        if isinstance(req_or_url, HTTPRequest):
            request = req_or_url
        else:
            request = HTTPRequest(req_or_url, *args, **kwargs)

        url_key = self.url_key(request.url)

        if url_key in self.mocks:
            fetch = self.fetch_mock
        else:
            fetch = super().fetch

        error = None
        try:
>           response = await gen.maybe_future(
                fetch(request, raise_error=kwargs.get("raise_error", True))
            )
E           tornado.httpclient.HTTPClientError: HTTP 503: Service Unavailable
manics commented 1 year ago

error = HTTP 503: Service Unavailable

Looks like a temporary outage, tests are now passing.