austinh2001
commented
2 years ago Ended up making this whole process into a set of objects which are created and utilized for the creation of manifest objects/ their associated file.
This relies on dataset objects which can be generated through Data and Metadata objects which themselves are essentially made up of two lists of keys and values, with Metadata having some other properties such as the ability to private some keys for Zooniverse. Data and Metadata are put together into a Dataset object, which is a generalized data-metadata-container object with potential beyond Cool Neighbors and even Zooniverse.
To specialize a bit, I used the Dataset object as a parent of a child class, Zooniverse_Dataset, which takes in a full filepath filename of a dataset.csv file (.fits file functionality is a future goal but is currently not implemented) and generates the Zooniverse_Dataset object from that.
What is particularly useful is the fact that a dataset.csv must contain an RA and DEC, but it can have any other metadata as well and it will still upload it to Zooniverse and give the metadata to Zooniverse to display for each subject. This opens up a lot of possibilities and should be explored further.
Additionally, there's checks that Datasets have Data/Metadata objects which all contain the same field_names to avoid conflicts and the option to have Defined_Manifests which will provide an error upon creation if the manifest being created has a different header than a "master header text file".
The comments for the new code is currently not done, but I am going to complete those in a new issue.
Creation of manifests is already implemented in the original refactored spout.py code, but it will be adjusted such that datasets can be constructed independently and verified with a master header text file.