search

WorldWideTelescope / wwt-web-client

The WorldWide Telescope web client lets you explore the universe in your browser.
https://worldwidetelescope.org/webclient/
MIT License
105 stars 35 forks source link

SDSS Galaxies Not Loading in Solar System View #341

Closed mfoley-astro closed 3 years ago

mfoley-astro commented 3 years ago

The following message appears in the Javascript console when in solar system view and zooming out beyond the Milky Way. This is using the Microsoft Edge browser. 

Uncaught TypeError: Cannot read property 'addPoint' of undefined
    at Function.Grids._createCosmosVertexBuffer (wwtsdk.min.js:1)
    at Function.Grids.initializeCosmos (wwtsdk.min.js:1)
    at FileReader.Grids._webFileGalaxy.get_state.Grids._webFileGalaxy.get_state.onloadend (wwtsdk.min.js:1)
astrojonathan commented 3 years ago

I confirmed this is happening on both Edge and Chrome. It appears the last line in the SDSS Cosmos file is corrupt, or the file is cut off since it has RA=0 and DEC=0 and the type is corrupt or undefined.

pkgw commented 3 years ago

This is strange to me because according to my database of WWT URLs, the content that the server is returning is the same as it ever was. But if we need to alter the file, that should be very easy with the upgraded server architecture.

astrojonathan commented 3 years ago

The file size on disk for downloaded file and the WWT windows client is 10,781,575, but the chunk of data that is being processed is 8,388,608, so the file is not being downloaded correctly. Looks like something broke in the WebFile class and it is firing before the whole file is downloaded.

astrojonathan commented 3 years ago

It appears that there may be some sort of new limit on the implementation of our http://www.worldwidetelescope.org/wwtweb/catalog.aspx?Q=cosmosbin that is limiting the download size to 8mb chunks.

pkgw commented 3 years ago

Aha, I figured out what's going on. The underlying server is fine, but the CDN has cached a bad version of the file. Compare:

http://www.worldwidetelescope.org/wwtweb/catalog.aspx?Q=cosmosnewbin

and

http://cdn.worldwidetelescope.org/wwtweb/catalog.aspx?Q=cosmosnewbin

I've just triggered a CDN purge of that path which I think might just fix things, although I wish I understood how the CDN got a bad copy to begin with.

astrojonathan commented 3 years ago

Still happens. Is there a 8MB limit on cache size ?

astrojonathan commented 3 years ago

I think this may be due to some behavior by the CDN and the browser that has changed... the exactly 8MB is what concerns me. I can't download anything but the 8MB. https://docs.microsoft.com/en-us/azure/cdn/cdn-large-file-optimization#object-chunking

pkgw commented 3 years ago

I think the chunking must be the culprit here. The updated server might not be handling byte-range requests correctly?

It would be ideal to fix the CDN issue, but an easy workaround would be to update the web engine to not use the CDN for this particular file. I'd also want to do a quick check for other static assets that might run into the same problem.

astrojonathan commented 3 years ago

So all the files over 8mb are not servering correctly on the CDN. This will break anyone with windows or web client who needs the files for the 3d Galaxy, Minor Planets(asteroids) ad the SDSS Galaxies, as well as the high-resolution clouds for the Earth. This needs rectified ASAP.

astrojonathan commented 3 years ago

It is hard to tell exactly when this started. I don't know if this happened with a CDN change, or if it was when we changed the catalog server, but I don't think it existed forever. We need to make sure we bypass CDN for files larger than 8MB for now..

pkgw commented 3 years ago

Yeah, I think it would have started with the server upgrade, although it's possible that it started spontaneously with some kind of CDN behavior change ... I don't expect that the older version of the server would have supported byte range requests either. I'm looking at fixing things up on the server side, and am hopeful I can get a fix deployed soon.

astrojonathan commented 3 years ago

Certainly, the code in the ASPX did not have that capability. Now it may have been that the server synthesized that behaviour and the leaner stack in the new code does not, or maybe the old CDN did not have the limit, and the new one does.

pkgw commented 3 years ago

I believe that the pull request WorldWideTelescope/wwt-website#280 should fix this. We may need to touch some other providers to use the new ServeStreamAsync function that activates support for byte-range requests — I'll need to do an audit of the URLs that get routed through the CDN.

pkgw commented 3 years ago

Looks like this has indeed been resolved!

astrojonathan commented 3 years ago

Woot! Woot! There be galaxies again! image