spedas / pyspedas

Python-based Space Physics Environment Data Analysis Software
https://pyspedas.readthedocs.io/
MIT License
155 stars 60 forks source link

THEMIS data is currently inaccessible #1042

Closed kmrs-iitd closed 4 weeks ago

kmrs-iitd commented 4 weeks ago

I am trying to download THEMIS data.

pyspedas.themis.state(probe='a', trange=['2021-01-01', '2021-01-02']):

14-Oct-24 14:39:34: Unauthorized: http://themis.ssl.berkeley.edu/data/themis/tha/l1/state/2021/tha_l1_state_20210101.cdf 14-Oct-24 14:39:34: Searching for local files... 14-Oct-24 14:39:34: No local files found for http://themis.ssl.berkeley.edu/data/themis/tha/l1/state/2021/tha_l1_state_20210101.cdf

Data from other THEMIS instruments (fgm, scm, efi..) are experiencing the same accessibility issues.

How to fix that? Thanks

jameswilburlewis commented 4 weeks ago

There was a hardware failure on the THEMIS storage server over the weekend. Our IT people are working on it, but we don't have an ETA yet for restoring service. Sorry for the inconvenience!

jameswilburlewis commented 4 weeks ago

There is a mirror of the THEMIS data server at UCLA, which should have all the probe data that was mirrored before the storage server went down. You can get PySPEDAS to use it by editing pyspedas/projects/themis/config.py and changing this setting:

CONFIG = {'local_data_dir': 'themis_data/', 'remote_data_dir': 'http://themis.ssl.berkeley.edu/data/themis/'}

so that 'remote_data_dir' is set to 'https://themis-data.igpp.ucla.edu/'

Ideally, PySPEDAS would be able to let the user override this in their own config file, rather than editing the PySPEDAS source code, but we haven't implemented that yet. This might be enough to get you going in the short term, but I'd recommend changing it back to the original setting when the THEMIS server is back online. The UCLA mirror doesn't have all of the THEMIS all-sky imager data that's available from the Berkeley site.

kmrs-iitd commented 4 weeks ago

Thank you for the update. I appreciate your team’s efforts in addressing the issue, and will patiently await further updates.

Thank you for your support!

jameswilburlewis commented 3 weeks ago

We have reverted to an older storage server, while we wait for the failed server to be diagnosed and repaired. Most of the THEMIS probe and ground data is back online -- there is still a gap in L2 products between September 27 and October 11 2024. We expect that gap to be filled later today.

Please let us know if you notice anything wrong!

kmrs-iitd commented 3 weeks ago

Thank you very much.