Solar astronomy data on GCS bucket

[mrocklin]

@MSKirk was kind enough to upload a number of FITS files that correspond to a solar astronomy dataset.

I can load a little bit of the data with the following steps:

import gcsfs
fs = gcsfs.GCSFileSystem()
urls = ['https://storage.googleapis.com/' + fn for fn in fs.ls('pangeo-data/SDO_AIA_Images/094')]

from astropy.io import fits
[img] = fits.open(urls[0])
x = img.data

cc @Cadair, @DanRyanIrish, @martindurant, @mmccarty

[MSKirk]

@wtbarnes and I are both very familiar with sunpy and map objects. I think for the Sunpy community, using some of their Maps code will foster a lot more good will than doing without it. Plus you can get the standard color tables for each of the images (I have my own issues, with them but won't get out my soap box just yet).

[rabernat]

Of course! Definitely best to leverage all the work that has gone into sunpy. It's clearly an amazing package.

For visualization, I am personally obsessed with holoviews / datashader and what they can do with cloud-backed datasets. (The screenshot I posted above was dynamically zoomable and scrollable in time.) For the long term, creating some integration layer between sunpy and holoviews might allow the best of both worlds.

[martindurant]

There are, though, further advantages to using Intake that may be attractive, if it is not too much of an imposition to use. As a location to frame and put data loading code, that is only part of the purpose, but consider the idea of encoding data locations and loading parameters into catalog files, which can potentially be stored remotely and updated in-place. Note that the headers, including WCS, is available in the data source objects. (just wanting to make sure that everyone has all the information!)

Indeed, the intake-astro outputs may be easy to coerce in to mapping objects, if that seems like a good idea.

[MSKirk]

I am giving an introductory sunpy tutorial at the end of the month. I will play around with holoviews / datashader to see if I can get a "hey look, I am flying" moment.

[rabernat]

Ok, so I will revisit this issue in a few weeks to see how things are progressing.

In the meantime, please feel free to put examples into https://github.com/pangeo-data/pangeo-astro-examples

[MSKirk]

@rabernat are you using a specific build of intake? I am not seeing the module open_fits_array in my version 0.2.6.

[martindurant]

@MSKirk , you also need intake-astro

[wtbarnes]

@wafels This discussion may be interesting to you as well.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it had not seen recent activity. The issue can always be reopened at a later date.