WorldWideTelescope / pywwt

Python interface to WorldWide Telescope :milky_way:
https://pywwt.readthedocs.io
BSD 3-Clause "New" or "Revised" License
51 stars 15 forks source link

Allow refreshing tile cache #375

Closed Carifio24 closed 6 months ago

Carifio24 commented 7 months ago

This PR exposes the cache-refreshing functionality of the research app to pywwt. We have a use case for this in CosmicDS (not urgent, but it was easy so I figured I'd get the ball rolling), but I think having the option available is generally useful. I did add a cautionary note in my docstring that this should really only be used when it's needed, though.

If we want a test, I'm not really sure what a good one would be - I don't see a straightforward way to use the image comparison method here.

Carifio24 commented 7 months ago

It looks like there's a pin in the CI (pin-1) that's requiring Python 3.12(?). I don't understand why conda would use Python 3.12 when we set the Python version here, though.

Carifio24 commented 7 months ago

@pkgw Ok, I think I have a beat on this now. So while we specify our Python version where I had linked above, the offending step is actually before that, in the setup. There we're installing brotlipy using the base environment (the comment says it's for a Windows workaround), which is now using Python 3.12 by default.

According to the runner info, it's using Miniconda 24.3.0. Miniconda added support for Python 3.12 in 24.1.2. The last CI prior to that was using Miniconda 23.10.0, so no Python 3.12, and thus not surprising that we didn't see this issue.

I'll admit I don't know exactly what we use brotlipy for, as it only seems to be referenced in that part of the CI. The Brotli project itself offers a brotli Python package, whereas brotlipy seems to have been become/been replaced by brotlicffi. Maybe it's time to upgrade?

pkgw commented 6 months ago

We wouldn't use Brotli(py) directly. In my own system, I see that urllib3 depends on it, though, so it probably comes in as a transitive dependency somewhere in the web stack (which makes sense, since Brotli is used as a compression format by modern web servers).

I'm afraid that I don't remember exactly what was going on that inspired me to add in the install step — probably some kind of missing library or something that prevented Conda from running. I've just pushed a commit to just remove the install since it was just a workaround to begin with.

Carifio24 commented 6 months ago

Removing the brotlipy install seems to have fixed this issue! My most recent commit deals with the actual test failures. The issues were:

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 50.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 56.51%. Comparing base (2c7377b) to head (5ec4686).

:exclamation: Current head 5ec4686 differs from pull request most recent head e1c6431. Consider uploading reports for the commit e1c6431 to get more accurate results

Files Patch % Lines
pywwt/core.py 50.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #375 +/- ## ========================================== - Coverage 56.51% 56.51% -0.01% ========================================== Files 25 25 Lines 2870 2872 +2 ========================================== + Hits 1622 1623 +1 - Misses 1248 1249 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

pkgw commented 6 months ago

The test that involves the sky_layers image starts by setting the foreground and background to the "Black Sky Background" imagery (here), so it's incorrect for stars to be showing up in the test image. It's not obvious to me why the behavior would be changing here. Clearly this PR shouldn't affect that, and I can't think of other recent changes that would either.

To my eye, the other image changes seem to be stemming from the background map not having tiled in to its full resolution just yet. Fetching those tiles is something that should be pretty reliable. It's possible that this could be worked around by increasing the delays in the tests, to give the engine more time to load images, but it might be good to double-check that this isn't indicating that our webservices are having some kind of issue.