search

astropy / pyvo

An Astropy affiliated package providing access to remote data and services of the Virtual Observatory (VO) using Python.
https://pyvo.readthedocs.io/en/latest
BSD 3-Clause "New" or "Revised" License
74 stars 50 forks source link

TST: fix doc example #546

Closed bsipocz closed 2 months ago

bsipocz commented 2 months ago

this should fix the currently failing CI

bsipocz commented 1 month ago

In the meantime the order in the response changed again, and now this one is breaking CI. So I'll just skip the output comparison on this in a quick new PR.

msdemlei commented 1 month ago

On Wed, May 15, 2024 at 08:44:13PM -0700, Brigitta Sipőcz wrote:

In the meantime the order in the response changed again, and now this one is breaking CI. So I'll just skip the output comparison on this in a quick new PR.

In general, we should never compare sequences (or literal output of sequences) generated from RDMS responses in tests; python sequences are ordered, database relationships are not.

One can ask the database server to sort, but our RegTAP interface doesn't support that yet (and frankly outside of doctesting, I don't see a case for that). Or one can sort or (preferably) make sets on the local side, but that's probably uncool for doctests.

So, I believe this is a case where doctesting finds its limits.

Talking about which: Is there a simple way to figure out which doctests take so long? If you tell me, I can try my hand choosing better examples.

bsipocz commented 1 month ago

I can easily tell which file takes long to test, but as far as pytest concerns unfortunately all the examples in a single file run as one.

And I already have a feature request issue to be more flexible with sets, though I doubt that will be easy to make work for cases like this that have an ellipse.