astropy / astroquery

Functions and classes to access online data resources. Maintainers: @keflavich and @bsipocz and @ceb8
http://astroquery.readthedocs.org/en/latest/
BSD 3-Clause "New" or "Revised" License
701 stars 396 forks source link

Some tests on astroquery.nasa_exoplanet_archive fail depending on order in which they are run #2083

Closed rickynilsson closed 3 years ago

rickynilsson commented 3 years ago

Running remote and mock tests, respectively, for astroquery.nasa_exoplanet_archive has some tests failing depending on order in which the test files are run. Run separately they both pass. Run consecutively, like below, some tests in the last run file fail.

Case 1:

~/anaconda3/envs/astroquery_test/lib/python3.8/site-packages/astroquery-0.4.3.dev6850-py3.8.egg/astroquery/nasa_exoplanet_archive/tests » pytest test_nasa_exoplanet_archive_remote.py test_nasa_exoplanet_archive.py

...

collected 50 items                                                                                                                                       

test_nasa_exoplanet_archive_remote.py .............            
test_nasa_exoplanet_archive.py ....F................................

Case 2:

~/anaconda3/envs/astroquery_test/lib/python3.8/site-packages/astroquery-0.4.3.dev6850-py3.8.egg/astroquery/nasa_exoplanet_archive/tests » pytest test_nasa_exoplanet_archive.py test_nasa_exoplanet_archive_remote.py

...

collected 50 items                                                                                                                                       

test_nasa_exoplanet_archive.py .....................................     
test_nasa_exoplanet_archive_remote.py .......F..F.F
bsipocz commented 3 years ago

Traceback:

$ pytest astroquery/nasa_exoplanet_archive --remote-data
================================================= test session starts ==================================================
platform darwin -- Python 3.9.1, pytest-6.2.1, py-1.10.0, pluggy-0.13.1

Running tests with astroquery version 0.4.2_testrun.
Running tests in astroquery/nasa_exoplanet_archive.

Date: 2021-06-18T08:08:52

Platform: macOS-10.13.6-x86_64-i386-64bit

Executable: /usr/local/opt/python@3.9/bin/python3.9

Full Python Version: 
3.9.1 (default, Jan 11 2021, 15:38:10) 
[Clang 10.0.0 (clang-1000.11.45.5)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Package versions: 
Numpy: 1.20.2
Matplotlib: 3.3.3
Astropy: 4.2.1
APLpy: not available
pyregion: not available
regions: 0.4
pyVO: 1.1
mocpy: 0.8.5
astropy-healpix: 0.5
vamdclib: 0.1
astropy-helpers: 4.0.1

Using Astropy options: remote_data: any.

rootdir: /Users/bsipocz/munka/devel/astroquery, configfile: setup.cfg
plugins: requests-mock-1.9.2, arraydiff-0.3, doctestplus-0.8.0, remotedata-0.3.2, hypothesis-6.0.0, dependency-0.5.1, filter-subpackage-0.1.1, cov-2.10.1, openfiles-0.5.0, astropy-header-0.1.2
collected 50 items                                                                                                     

astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive.py .....................................     [ 74%]
astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py .......F..F.F                      [100%]

======================================================= FAILURES =======================================================
____________________________________________________ test_warnings _____________________________________________________

    @pytest.mark.remote_data
    def test_warnings():
        with pytest.warns(NoResultsWarning):
            NasaExoplanetArchive.query_criteria("ps", where="hostname='not a host'")

        with pytest.warns(InputWarning):
>           NasaExoplanetArchive.query_object("HAT-P-11 b", where="nothing")

astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py:106: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
astroquery/utils/class_or_instance.py:25: in f
    return self.fn(obj, *args, **kwds)
astroquery/utils/process_asyncs.py:29: in newmethod
    result = self._parse_result(response, verbose=verbose)
astroquery/nasa_exoplanet_archive/core.py:566: in _parse_result
    self._handle_error(text)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <astroquery.nasa_exoplanet_archive.core.NasaExoplanetArchiveClass object at 0x127887dc0>
text = 'ERROR<br>\nError Type: SystemError<br>\nMessage:    Error 904: 42S22 :[Oracle][ODBC][Ora]ORA-00904: "PL_HOSTNAME": invalid identifier . FAIL\n'

    def _handle_error(self, text):
        """
        Parse the response from a request to see if it failed

        Parameters
        ----------
        text : str
            The decoded body of the response.

        Raises
        ------
        InvalidColumnError :
            If ``select`` included an invalid column.
        InvalidTableError :
            If the queried ``table`` does not exist.
        RemoteServiceError :
            If anything else went wrong.
        """
        # Error messages will always be formatted starting with the word "ERROR"
        if not text.startswith("ERROR"):
            return

        # Some errors have the form:
        #   Error type: ...
        #   Message: ...
        # so we'll parse those to try to provide some reasonable feedback to the user
        error_type = None
        error_message = None
        for line in text.replace("<br>", "").splitlines():
            match = re.search(r"Error Type:\s(.+)$", line)
            if match:
                error_type = match.group(1).strip()
                continue

            match = re.search(r"Message:\s(.+)$", line)
            if match:
                error_message = match.group(1).strip()
                continue

        # If we hit this condition, that means that we weren't able to parse the error so we'll
        # just throw the full response
        if error_type is None or error_message is None:
            raise RemoteServiceError(text)

        # A useful special is if a column name is unrecognized. This has the format
        #   Error type: SystemError
        #   Message: ... "NAME_OF_COLUMN": invalid identifier ...
        if error_type.startswith("SystemError"):
            match = re.search(r'"(.*)": invalid identifier', error_message)
            if match:
>               raise InvalidQueryError(
                    (
                        "'{0}' is an invalid identifier. This error can be caused by invalid "
                        "column names, missing quotes, or other syntax errors"
                    ).format(match.group(1).lower())
                )
E               astroquery.exceptions.InvalidQueryError: 'pl_hostname' is an invalid identifier. This error can be caused by invalid column names, missing quotes, or other syntax errors

astroquery/nasa_exoplanet_archive/core.py:448: InvalidQueryError
__________________________________________________ test_query_region ___________________________________________________

    @pytest.mark.remote_data
    def test_query_region():
        coords = SkyCoord(ra=330.79488 * u.deg, dec=18.8843 * u.deg)
        radius = 0.001
>       table1 = NasaExoplanetArchive.query_region("pscomppars", coords, radius * u.deg)

astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py:148: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
astroquery/utils/class_or_instance.py:25: in f
    return self.fn(obj, *args, **kwds)
astroquery/utils/process_asyncs.py:29: in newmethod
    result = self._parse_result(response, verbose=verbose)
astroquery/nasa_exoplanet_archive/core.py:566: in _parse_result
    self._handle_error(text)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <astroquery.nasa_exoplanet_archive.core.NasaExoplanetArchiveClass object at 0x127887dc0>
text = 'ERROR<br>\nError Type: UserError - "table" parameter<br>\nMessage:    "pscomppars" is not a valid table.\n'

    def _handle_error(self, text):
        """
        Parse the response from a request to see if it failed

        Parameters
        ----------
        text : str
            The decoded body of the response.

        Raises
        ------
        InvalidColumnError :
            If ``select`` included an invalid column.
        InvalidTableError :
            If the queried ``table`` does not exist.
        RemoteServiceError :
            If anything else went wrong.
        """
        # Error messages will always be formatted starting with the word "ERROR"
        if not text.startswith("ERROR"):
            return

        # Some errors have the form:
        #   Error type: ...
        #   Message: ...
        # so we'll parse those to try to provide some reasonable feedback to the user
        error_type = None
        error_message = None
        for line in text.replace("<br>", "").splitlines():
            match = re.search(r"Error Type:\s(.+)$", line)
            if match:
                error_type = match.group(1).strip()
                continue

            match = re.search(r"Message:\s(.+)$", line)
            if match:
                error_message = match.group(1).strip()
                continue

        # If we hit this condition, that means that we weren't able to parse the error so we'll
        # just throw the full response
        if error_type is None or error_message is None:
            raise RemoteServiceError(text)

        # A useful special is if a column name is unrecognized. This has the format
        #   Error type: SystemError
        #   Message: ... "NAME_OF_COLUMN": invalid identifier ...
        if error_type.startswith("SystemError"):
            match = re.search(r'"(.*)": invalid identifier', error_message)
            if match:
                raise InvalidQueryError(
                    (
                        "'{0}' is an invalid identifier. This error can be caused by invalid "
                        "column names, missing quotes, or other syntax errors"
                    ).format(match.group(1).lower())
                )

        elif error_type.startswith("UserError"):
            # Another important one is when the table is not recognized. This has the format:
            #   Error type: UserError - "table" parameter
            #   Message: ... "NAME_OF_TABLE" is not a valid table.
            match = re.search(r'"(.*)" is not a valid table', error_message)
            if match:
>               raise InvalidTableError("'{0}' is not a valid table".format(match.group(1).lower()))
E               astroquery.nasa_exoplanet_archive.core.InvalidTableError: 'pscomppars' is not a valid table

astroquery/nasa_exoplanet_archive/core.py:461: InvalidTableError
_____________________________________________________ test_format ______________________________________________________

    @pytest.mark.remote_data
    def test_format():
>       table1 = NasaExoplanetArchive.query_object("HAT-P-11 b")

astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py:166: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
astroquery/utils/class_or_instance.py:25: in f
    return self.fn(obj, *args, **kwds)
astroquery/utils/process_asyncs.py:29: in newmethod
    result = self._parse_result(response, verbose=verbose)
astroquery/nasa_exoplanet_archive/core.py:566: in _parse_result
    self._handle_error(text)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <astroquery.nasa_exoplanet_archive.core.NasaExoplanetArchiveClass object at 0x127887dc0>
text = 'ERROR<br>\nError Type: SystemError<br>\nMessage:    Error 904: 42S22 :[Oracle][ODBC][Ora]ORA-00904: "PL_HOSTNAME": invalid identifier . FAIL\n'

    def _handle_error(self, text):
        """
        Parse the response from a request to see if it failed

        Parameters
        ----------
        text : str
            The decoded body of the response.

        Raises
        ------
        InvalidColumnError :
            If ``select`` included an invalid column.
        InvalidTableError :
            If the queried ``table`` does not exist.
        RemoteServiceError :
            If anything else went wrong.
        """
        # Error messages will always be formatted starting with the word "ERROR"
        if not text.startswith("ERROR"):
            return

        # Some errors have the form:
        #   Error type: ...
        #   Message: ...
        # so we'll parse those to try to provide some reasonable feedback to the user
        error_type = None
        error_message = None
        for line in text.replace("<br>", "").splitlines():
            match = re.search(r"Error Type:\s(.+)$", line)
            if match:
                error_type = match.group(1).strip()
                continue

            match = re.search(r"Message:\s(.+)$", line)
            if match:
                error_message = match.group(1).strip()
                continue

        # If we hit this condition, that means that we weren't able to parse the error so we'll
        # just throw the full response
        if error_type is None or error_message is None:
            raise RemoteServiceError(text)

        # A useful special is if a column name is unrecognized. This has the format
        #   Error type: SystemError
        #   Message: ... "NAME_OF_COLUMN": invalid identifier ...
        if error_type.startswith("SystemError"):
            match = re.search(r'"(.*)": invalid identifier', error_message)
            if match:
>               raise InvalidQueryError(
                    (
                        "'{0}' is an invalid identifier. This error can be caused by invalid "
                        "column names, missing quotes, or other syntax errors"
                    ).format(match.group(1).lower())
                )
E               astroquery.exceptions.InvalidQueryError: 'pl_hostname' is an invalid identifier. This error can be caused by invalid column names, missing quotes, or other syntax errors

astroquery/nasa_exoplanet_archive/core.py:448: InvalidQueryError
=============================================== short test summary info ================================================
FAILED astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py::test_warnings - astroquery.exce...
FAILED astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py::test_query_region - astroquery....
FAILED astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py::test_format - astroquery.except...
============================================ 3 failed, 47 passed in 28.44s =============================================
bsipocz commented 3 years ago

Traceback for case1 from above:

$ pytest astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive.py --remote-data
================================================= test session starts ==================================================
platform darwin -- Python 3.9.1, pytest-6.2.1, py-1.10.0, pluggy-0.13.1

Running tests with astroquery version 0.4.2_testrun.
Running tests in astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive.py.

Date: 2021-06-18T08:34:59

Platform: macOS-10.13.6-x86_64-i386-64bit

Executable: /usr/local/opt/python@3.9/bin/python3.9

Full Python Version: 
3.9.1 (default, Jan 11 2021, 15:38:10) 
[Clang 10.0.0 (clang-1000.11.45.5)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Package versions: 
Numpy: 1.20.2
Matplotlib: 3.3.3
Astropy: 4.2.1
APLpy: not available
pyregion: not available
regions: 0.4
pyVO: 1.1
mocpy: 0.8.5
astropy-healpix: 0.5
vamdclib: 0.1
astropy-helpers: 4.0.1

Using Astropy options: remote_data: any.

rootdir: /Users/bsipocz/munka/devel/astroquery, configfile: setup.cfg
plugins: requests-mock-1.9.2, arraydiff-0.3, doctestplus-0.8.0, remotedata-0.3.2, hypothesis-6.0.0, dependency-0.5.1, filter-subpackage-0.1.1, cov-2.10.1, openfiles-0.5.0, astropy-header-0.1.2
collected 50 items                                                                                                     

astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive_remote.py ............Exception ignored in: <ssl.SSLSocket fd=19, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('192.168.0.101', 49406), raddr=('134.4.54.151', 443)>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/_pytest/unraisableexception.py", line 59, in __exit__
    del self.unraisable
ResourceWarning: unclosed <ssl.SSLSocket fd=19, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('192.168.0.101', 49406), raddr=('134.4.54.151', 443)>
.                      [ 26%]
astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive.py ....F................................     [100%]

======================================================= FAILURES =======================================================
_____________________________________ test_api_tables[q1_q17_dr25_sup_koi-query2] ______________________________________

cls = <class '_pytest.runner.CallInfo'>, func = <function call_runtest_hook.<locals>.<lambda> at 0x1217be160>
when = 'call', reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: "Callable[[], TResult]",
        when: "Literal['collect', 'setup', 'call', 'teardown']",
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

/usr/local/lib/python3.9/site-packages/_pytest/runner.py:311: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.9/site-packages/_pytest/runner.py:255: in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
/usr/local/lib/python3.9/site-packages/pluggy/hooks.py:286: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/local/lib/python3.9/site-packages/pluggy/manager.py:93: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
/usr/local/lib/python3.9/site-packages/pluggy/manager.py:84: in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
/usr/local/lib/python3.9/site-packages/_pytest/unraisableexception.py:88: in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
        with catch_unraisable_exception() as cm:
            yield
            if cm.unraisable:
                if cm.unraisable.err_msg is not None:
                    err_msg = cm.unraisable.err_msg
                else:
                    err_msg = "Exception ignored in"
                msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
                msg += "".join(
                    traceback.format_exception(
                        cm.unraisable.exc_type,
                        cm.unraisable.exc_value,
                        cm.unraisable.exc_traceback,
                    )
                )
>               warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E               pytest.PytestUnraisableExceptionWarning: Exception ignored in: <ssl.SSLSocket fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>
E               
E               Traceback (most recent call last):
E                 File "/usr/local/lib/python3.9/site-packages/astropy/units/format/generic.py", line 566, in _do_parse
E                   return cls._parse_unit(s, detailed_exception=False)
E                 File "/usr/local/lib/python3.9/site-packages/astropy/units/format/generic.py", line 487, in _parse_unit
E                   raise ValueError()
E               ValueError
E               
E               During handling of the above exception, another exception occurred:
E               
E               Traceback (most recent call last):
E                 File "/usr/local/lib/python3.9/site-packages/astropy/utils/misc.py", line 460, in did_you_mean
E                   candidates_lower.setdefault(candidate_lower, [])
E               ResourceWarning: unclosed <ssl.SSLSocket fd=18, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('192.168.0.101', 49398), raddr=('134.4.54.151', 443)>

/usr/local/lib/python3.9/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning
=============================================== short test summary info ================================================
FAILED astroquery/nasa_exoplanet_archive/tests/test_nasa_exoplanet_archive.py::test_api_tables[q1_q17_dr25_sup_koi-query2]