Supporting in-situ data in the wider heliophysics ecosystem

[dstansby]

This is a write up of a discussion on 02/04/2020 during the SunPy developer meeting. Comments or additions are very welcome and encouraged from those that were not present.

Background

SunPy is primarily a package for dealing with remote sensing data, however new missions (PSP, Solar Orbiter) increasingly require scientists to combine analysis of remote sensing data and in-situ data of the solar wind.

HelioPy was developed with the aim of making in-situ solar wind data easily discoverable and downloadable. Currently this is via. the heliopy.data module. Once downloaded, all data is presented to the user in sunpy.timeseries.TimeSeries objects.

Until recently, caching of data has been the main reason HelioPy has not used the Fido infrastructure in SunPy to find and download data. Now Fido automatically caches by filename, this blocker has been removed.

Question

In which package should code for discovering, downloading, and reading in-situ data live?

If SunPy performs this task, it would

• Need to implement new clients
• Add the ability to read .cdf files

If HelioPy performs this task it would

• Still need to implement new clients, but existing user facing API could still remain

Proposal

• SunPy does not provide code for discovering, downloading, or reading in-situ data
• HelioPy performs the above functionality
• HelioPy registers all clients with Fido, so that one can search for data provided by HelioPy and SunPy at the same time

[abhijeetmanhas]

Sorry if it sounds irrelevant here but I proposed implementing .cdf reader using cdflib and new clients in my GSoC proposal for Fido. However the clients were ACE Clients and some groundclients, . PRs https://github.com/sunpy/sunpy/pull/3705 and this https://github.com/sunpy/sunpy/pull/3763 .