indigo-astronomy / indigo

INDIGO is a system of standards and frameworks for multiplatform and distributed astronomy software development designed to scale with your needs.
http://www.indigo-astronomy.org
Other
139 stars 64 forks source link

Use a custom permanent set of placeholders in CCD file name "PREFIX" #467

Closed Paolo97Gll closed 1 year ago

Paolo97Gll commented 1 year ago

Using AIN imager, the PREFIX (ccd driver property -> Camera -> Save on server -> File name prefix) when starting a batch is set to name_%-D_%F_%C_%M, where name is the one inserted in AIN imager. Is it possible to use a custom permanent set of placeholders (e.g. _%-D_%-H_%F_%C_%M instead of _%-D_%F_%C_%M)?

rumengb commented 1 year ago

No, Ain uses its own set and it sets it each time the object name is set and when the exposure is started. Placeholders are for the client software to set them. At some point I will add placeholders to Ain but this has a low priority in my TODO.

Paolo97Gll commented 1 year ago

Ok, thanks! I thought of a solution like this because of a ""bug"" I found using INDIGO in clientless mode (and AIN for starting the batch). In particular, the progressive order in which AIN downloads (and so renames) the files differs from the real shot order, I think because on the server the images are saved with the _%-D_%F_%C_%M format and so, within the same day, the images are ordered by the hash. Using the _%-D_%-H_%F_%C_%M prefix set will solve this ""bug"" and so all the downloaded images are kept in order. For now, I have to rename all the files with a python script using astropy if I want to do temporal series analysis (like variable stars or comets astrometry).

Not knowing the source of this ""bug"" when I opened this issue, I chose the main INDIGO repository. Now that I know this is related to AIN, maybe using also the %-H placeholder here can be a valid idea to solve this?

rumengb commented 1 year ago

I never intended to make sorting by filename the same as sorting by date. Actually what you mentioned is only part of the "problem". The other part is merging the filenames with the already saved files. And still there is yet another problem when you relay on file names only. For example, if your preference is to see how many similar frames you have, including the time in the name will not make it obvious as the sequential numbers will be hallways "0". As it is now you look at the last frame number and you have it, but no grantee in the file name order. If you prefer to have the files sorted by name to be sorted by creation time you will not have clear view of how many frames you have in a single filter or how many darks you have etc. So this is a matter of personal preference... Also including the time makes the filename even longer and harder to distinguish. The sequential number gives a lot clearer distinction. On the other hand if the processing is acquisition time sensitive, like photometry, the software used should be able to read the header information and to get date, time, filter etc. Filename is not a reliable source of information it is just for convenience. A good example is the time: time placeholder is the file creation time, not the observation time which for a 30 min exposure is 30 minutes earlier :) Once I add custom file name templates everyone will be able to make the filenames as they wish :) So this is not a bug... this is just a preference :)

rumengb commented 1 year ago

With the next INDIGO release the files will be downloaded in order of creation not alphabetically. This way files with sequential numbers will will still be alphabetically ordered (as before) but the files with MD5 digests will be ordered in order of creation so no more random file sequencing using MD5. Still files will not be merged time wise with the locally saved but it is far better that having downloaded files scrambled.