search

pysat / pysatNASA

pysat support for NASA Instruments
BSD 3-Clause "New" or "Revised" License
21 stars 7 forks source link

BUG/TST: use sleep during instrument download tests #172

Closed jklenzing closed 1 year ago

jklenzing commented 2 years ago

Describe the bug Sometimes during the instrument library tests, download tests will fail with a "maximum connections exceeded" error. This is likely because the downloads all occur sequentially, and for small files could trigger this.

Note this does not occur on github actions, possibly because the servers are slower or they use a dynamic IP, so the host IP is potentially different with every request.

Adding time.sleep(1) to the end of pysat.tests.classes.cls_instrument_library.test_download solves this issue locally. This adds a 1-second pause to this test, which may be enough to keep this bug from triggering.

To Reproduce run pytest on the pysatNASA library from a local machine.

Expected behavior tests should succeed and not annoy the data servers.

Additional context Probably should review across packages to see if this affects others, or if the proposed solution is too onerous there.

UPDATE Noting this also occurs for downloading multiple small files, not just the tests

rstoneback commented 2 years ago

I ran the tests on my local machine and did not get the maximum connections error. I think it is a network proximity thing.

Anyway, I'm fine with the noted sleep commands so that you can run tests as well.

jklenzing commented 2 years ago

I'm going to tentatively close this issue, as the problem has gone away for now and is easily solved locally. May reopen with sleep as a user-defined option down the line if this becomes a problem again.

jklenzing commented 1 year ago

Reopening as this is affecting usage on campus for NASA developers

jklenzing commented 1 year ago

Transferred to pysatNASA as modifying the cdaweb download routine is the most direct way of solving.