search

panoptes / panoptes-pipeline

PANOPTES Data Processing Pipeline
MIT License
9 stars 8 forks source link

Choose naming convention for PSCs and lightcurves #12

Closed kstoreyf closed 4 years ago

kstoreyf commented 8 years ago

needs to identify image sequence and PIC.

example: PSC[PICID][SEQID] but this gets really long because SEQID = [unitid][cameraid][start_time_of_sequence]...

wtgee commented 8 years ago

Are we storing these in a Bucket and using the PICID for looking up PICs across units? If not, the PICID will be within the header metadata.

kstoreyf commented 8 years ago

this is related to the organization of the storage (#9). if we organize just by unit, won't there be multiple PSCs per image sequence? we can open the headers and look up the PIC in order to combine them, but don't we still need a unique name for the file?

kstoreyf commented 8 years ago

decided in https://github.com/panoptes/PIAA/wiki/File-Organization-for-Images,-PSCs-and-Light-Curves

followup: naming convention/organization for master light curves, which will be generated and stored on the cloud? these will be all of the light curves for a given object that have been taken over time, so naming could just be:

MLC/picid.json

(or whatever format we store them in, see #4 - would need other metadata stored in them) (MLC = master light curve, anyone have a better idea?)

wtgee commented 8 years ago

We should figure out how large the MLC will eventually be and how we plan to use it but that naming scheme seems fine.

@mimming Should we be thinking about different buckets for the MLC or it is fine to just have everything stored in panoptes-survey? The top-level within that bucket currently corresponds to a unitid and I have been using it to dump things in.