Failure to retrieve a Maps file given MaNGAID

[CalebRichardson2001]

Describe the bug Received an error message that Marvin was unable to retrieve the Maps file for given input parameters, with those input parameters being MaNGAIDs

To Reproduce This occured midway through a loop retrieving the HDU files for all unique MaNGAIDs, which are in a list generated by using the method recommended by the SDSS website and the drpall-v3_1_1 file. I don't know if this just affects certain MaNGAIDs, or which MaNGAID caused this unforunately

Expected behaviour I expect Marvin to not throw this error

Desktop (please complete the following information):

• Current version of Marvin, on SciServer

Additional context 0;31m--------------------------------------------------------------------------- MarvinError Traceback (most recent call last) /tmp/ipykernel_13/2463262468.py in   2 hdu_list = []  3 for val in uniq_vals: ----> 4 data1 = Maps(val)  5 hdu_list.append(fits.open(data1.filename))  6 counter1 + 1

~/miniconda3/envs/py38/lib/python3.8/site-packages/marvin/tools/maps.py in init(self, input, filename, mangaid, plateifu, mode, data, release, drpall, download, nsa_source, bintype, template, template_kin)  95 self._bitmasks = None  96  ---> 97 MarvinToolsClass.init(self, input=input, filename=filename,  98 mangaid=mangaid, plateifu=plateifu,  99 mode=mode, data=data, release=release,

~/miniconda3/envs/py38/lib/python3.8/site-packages/marvin/tools/core.py in init(self, input, filename, mangaid, plateifu, mode, data, release, drpall, download)  114 mode=None, data=None, release=None, drpall=None, download=None):  115  --> 116 MMAMixIn.init(self, input=input, filename=filename, mangaid=mangaid,  117 plateifu=plateifu, mode=mode, data=data, release=release,  118 download=download)

~/miniconda3/envs/py38/lib/python3.8/site-packages/marvin/tools/mixins/mma.py in init(self, input, filename, mangaid, plateifu, mode, data, release, drpall, download, ignore_db)  125   126 if self.mode == 'local': --> 127 self._doLocal()  128 elif self.mode == 'remote':  129 self._doRemote()

~/miniconda3/envs/py38/lib/python3.8/site-packages/marvin/tools/mixins/mma.py in _doLocal(self)  289 self.data_origin = 'file'  290 else: --> 291 raise MarvinError('failed to retrieve data using '  292 'input parameters.')  293 

MarvinError: failed to retrieve data using input parameters. You can submit this error to Marvin GitHub Issues (https://github.com/sdss/marvin/issues/new). Fill out a subject and some text describing the error that just occurred. If able, copy and paste the full traceback information into the issue as well. MarvinError: failed to retrieve data using input parameters. You can submit this error to Marvin GitHub Issues (https://github.com/sdss/marvin/issues/new). Fill out a subject and some text describing the error that just occurred. If able, copy and paste the full traceback information into the issue as well.

[havok2063]

@CalebRichardson2001 Are you trying to download the files? If you're on SciServer then all the files are already local. You can either access the files directly or through Marvin Maps. This may have been a temporary issue with SciServer? Do you have a better traceback than this? I can't easily read that.

[CalebRichardson2001]

@CalebRichardson2001 Are you trying to download the files? If you're on SciServer then all the files are already local. You can either access the files directly or through Marvin Maps. This may have been a temporary issue with SciServer? Do you have a better traceback than this? I can't easily read that.

Unfortunately no - SciServer does not give fantastic tracebacks when you're running a job

I'm trying to use Marvin (in local mode, with "database" manually turned off) in order to find the MAPS file name associated with a given MaNGAID.

[CalebRichardson2001]

I reran the job and it failed at the same point. My gut says it may be that Marvin is struggling with a specific galaxy (potentially multiple galaxies that appear later in the list)

[havok2063]

can you record which mangaid it fails at, or the iteration in your loop? If you know the plate-ifu, you can manually check for the file existence in SciServer.

[CalebRichardson2001]

can you record which mangaid it fails at, or the iteration in your loop? If you know the plate-ifu, you can manually check for the file existence in SciServer.

It fails at mandaid 1-210966. Since I ran it straight from the notebook and not through Jobs, I was also able to get a picture of the traceback

[CalebRichardson2001]

Having now iterated through the entire list, starting the loop up again at the next galaxy when one failed. The following all cause the failure: 1-210966, 1-249929, 1-312844, 1-456385, 1-634800, 1-80510, 54-35.

It was suggested to me that it's possible that these are seven galaxies where the velocity pipeline failed (thus no MAPS file) but spectra was able to be generated, so they WERE in the DRPall file I used to generate the list of available galaxies. I'm going to go poke some DAP files and see what what the value of DAPQUAL is for them

[CalebRichardson2001]

Yep, all of them lacked MAPS files. Problem solved, it wasn't Marvin's fault

[havok2063]

Yes not all plate-ifus in the drpall file made it through the DAP. Some made it through different DAP daptype stages, so the number maps files for SPX binning isn't necessarily the same as the HYB10 binning scheme, which is the default bintype that Marvin uses. To see which cubes made it through the DAP, you can check the dapall file. Look at the daptype extension you're interested in, then check the DAPDONE column. That will tell you if that cube made it through that stage of the DAP.