pfsspy.sample_data.get_gong_map : looks like link is broken

[STBadman]

Did a fresh clone of pfsspy main (>>> pfsspy.__version__-> '0.6.6+36.gdd10d40' ) and getting an error for the hash provided for the loaded GONG map. I checked the URL and it is still working, but I guess the hash/underlying file has changed. Looks like something similar happened in #254 ? If the file url stays constant, is it necessary to have the hash matching requirement?

>>> import pfsspy                                                                                    
>>> pfsspy.sample_data.get_gong_map()                                                                
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-69a94c37ca22> in <module>
----> 1 pfsspy.sample_data.get_gong_map()

~/anaconda3/lib/python3.7/site-packages/sunpy/data/data_manager/manager.py in wrapper(*args, **kwargs)
     77                         # We match by urls to make sure that is not the case
     78                         if self._cache_has_file(urls):
---> 79                             raise ValueError(" Hash provided does not match the hash in database.")
     80                         file_path = self._cache.download(urls)
     81                         if hash_file(file_path) != sha_hash:

ValueError:  Hash provided does not match the hash in database.

[dstansby]

That's not a very helpful error message, I shall try and improve that in sunpy...

I think this is because the remote file changed, and I updated the hash. If you delete your local copy of the file, and re-run does it pass? If not I might get rid of the data manager stuff as it seems more hassle than it's worth.

[STBadman]

So after deleting sunpy/data_manager/data_manager.db it did a fresh download and now works. Before this, I tried the following which did not work :

1) I deleted only the file sunpy/data_manager/mrzqs200901t1304c2234_022.fits.gz which yielded the error listed below. This looks like the action of downloading the file previously has been cached but instead of checking if the file exists and re-downloading, it is just looking for a file with a matching hash?

2) I next tried downloading the .fits.gz file from the url in the error message below manually and dropping it into sunpy/data_manager, this yielded the same error message

It seems ideally, if a file is recorded as downloaded in data_manager but is missing in the directory, it should redownload and update the hash - is that something that is doable?

>>> import pfsspy                                                                                    
>>> pfsspy.sample_data.get_gong_map()                                                                
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-69a94c37ca22> in <module>
----> 1 pfsspy.sample_data.get_gong_map()

~/anaconda3/lib/python3.7/site-packages/sunpy/data/data_manager/manager.py in wrapper(*args, **kwargs)
     82                             # If we can't find a file matching sha_hash, but the url is already
     83                             # in the database
---> 84                             raise ValueError(f"{urls} has already been downloaded, but no file "
     85                                              f"matching the hash {sha_hash} can be found.")
     86                         file_path = self._cache.download(urls, self._namespace)

ValueError: ['https://gong2.nso.edu/oQR/zqs/202009/mrzqs200901/mrzqs200901t1304c2234_022.fits.gz'] has already been downloaded, but no file matching the hash aad927d8f617f32b72255b862c4910f13640fc7ca13edf98288cd0735a2db6a0 can be found.

[dstansby]

I think I came across the same issue here: https://github.com/sunpy/sunpy/issues/4597, so I'm going to close this in favor of that issue.