search

i4Ds / STIXCore

STIX Core functionalities
BSD 3-Clause "New" or "Revised" License
3 stars 3 forks source link

ESA file delivery #287

Closed samaloney closed 1 year ago

samaloney commented 1 year ago

Duplicated request e.g same start, end, request id (data fits hash?)

ESA file name collisions

Create new products with '-sup1' and if needed '-sup2' identifying these data supplementary so given:

solo_L1_stix-sci-xray-cpd_20220320T142822-20220320T144341_V01_2203206157-64599.fits
solo_L1_stix-sci-xray-cpd_20220320T142822-20220320T144341_V01_2203206157-64005.fits
solo_L1_stix-sci-xray-cpd_20220320T142822-20220320T144341_V01_2203206157-<diff num>.fits

we would rename the files delivered to ESA to be

solo_L1_stix-sci-xray-cpd_20220320T142822-20220320T144341_V01_2203206157-64599.fits
solo_L1_stix-sci-xray-cpd-sup1_20220320T142822-20220320T144341_V01_2203206157-64005.fits
solo_L1_stix-sci-xray-cpd_sup2_20220320T142822-20220320T144341_V01_2203206157-<diff num>.fits

and ESA will truncate it to

solo_L1_stix-sci-xray-cpd_20220320T142822-20220320T144341_V01.fits
solo_L1_stix-sci-xray-cpd-sup1_20220320T142822-20220320T144341_V01.fits
solo_L1_stix-sci-xray-cpd_sup2_20220320T142822-20220320T144341_V01.fits

flag if more than sup2 needed and handle manually.

ennosigaeus commented 1 year ago

@samaloney @nicHoch @drhlxiao

Hi guys, We, Säm, Hualin, and I, discussed this topic this morning in the STIX office. Since Hualin mentioned it, I just realized that the unique request ID is also added to the file name used to deliver to ESA - which seems to be in line with the example given above. I learned that four digits are random... so why did we go through all that trouble of introducing the sup1 and sup2 if, in the future, all requests, even a re-request of the same interval with a different configuration, will be assigned a unique ID due to the random component? Moreover, we had said to write a small file comparison tool... if the ID is unique in the future and we have a duplicated request, the file scanner will not find that since the ID is different...

Can somebody help me out here?

ennosigaeus commented 1 year ago

Had an exchange with Nicky. I must have overlooked the truncate part. Sorry.

nicHoch commented 1 year ago

implemented with #280