Loading from JSON

[nonrice]

33

I finished a function in dataManager loadFromJSON which accepts a filepath to the json file. I imagined the json configuration looking like this:

{
    "load_A": ["1.txt", "2.txt"],
    "load_B": ["3.txt", "4.txt"],
    ...,  
    "children": ["subdir1/config.json"]
}

Here each data type e.g. "Load_A" contains an array of files which will be opened using the load function for that data type. Finally "children" allows the loadFromJSON function to recurse into another json file which may be useful for aggregating experiment data together.

To do the path name expansion with "*" and such I used https://github.com/p-ranav/glob which has a header only implementation. I just copied over the header and the license.

Right now in dataManager there are two dummy functions "load_A" and "load_B". Line 178 hooks them up to the keys in the JSON.

[paulmthompson]

Thank you for adding the glob-ing function and json library! I think we should have the json file have some additional (sometimes optional) fields to accommodate some of the different data types. I imagine something like an array where each entry has multiple fields

[
    {
        "filepath": "path_to_my_video.mp4",
        "data_type": "video",
        "name": "my_video"

    },
    {
        "filepath": "points.csv",
        "data_type": "points",
        "name": "jaw",
        "frame_column": 0,
        "x_column": 1,
        "y_column": 2
    },
    {
        "filepath": "another_json.json",
        "datatype": "config"
    }
]

Here the points entry has fields that are necessary to use the "load_points_from_csv" function in data_manager. That file could be selected because of the .csv file extension, but if this file was hdf5, a different points loading method could be selected. The "name" value could be used as a prefix for the name that is given to it in datamanager. If globbing is used, it would possibly generate data manager entries like "jaw_1", "jaw_2" etc.

I think having a data_type of "config" would be able to accomplish the "children" method you have outlined above.

I have added a couple of testing methods that use data manager to load in media and csv files. How about we try to get a test passing where it loads the video and points csv file now included? Then we can piecemeal add in additional loading methods and incorporate it into the testing.

Also, my Windows build wouldn't actually compile with the current JSON method because the std::filesystem::path was being implicitly converted to a string.