We observe that there are zero-sized acquisition monitor files (*pickles) produces by pax on xe1t-datamanager when e.g. a noise dataset is taken or an LED calibration is done. These datasets cause some trouble in the rucio catalog.
Rucio distributes the raw data (files attached to a rucio container) according to transfer rules. Once the transfer is finished the raw dataset (a rucio container) is marked from transferring to "ok" which means that all single files are transferred. Unfortunately this does not work when there is a zero sized file in the raw data. Some rucio storage elements (RSE) do not allow this. The status of the rucio catalog entry switches to "stuck" after a while and the dataset is not transferred correctly and therefore it does not appear in our runDB!
My main question if it possible to prevent pax from creating a zero sized file like this? The only alternative is to delete zero sized files before uploading to the rucio catalog (in ruciax). Or is this file needed for anything, nevertheless if it has a size or not?
feigaodm
commented
7 years ago
We observe that there are zero-sized acquisition monitor files (*pickles) produces by pax on xe1t-datamanager when e.g. a noise dataset is taken or an LED calibration is done. These datasets cause some trouble in the rucio catalog. Rucio distributes the raw data (files attached to a rucio container) according to transfer rules. Once the transfer is finished the raw dataset (a rucio container) is marked from transferring to "ok" which means that all single files are transferred. Unfortunately this does not work when there is a zero sized file in the raw data. Some rucio storage elements (RSE) do not allow this. The status of the rucio catalog entry switches to "stuck" after a while and the dataset is not transferred correctly and therefore it does not appear in our runDB! My main question if it possible to prevent pax from creating a zero sized file like this? The only alternative is to delete zero sized files before uploading to the rucio catalog (in ruciax). Or is this file needed for anything, nevertheless if it has a size or not?
Thank you,