rosikand / rsbox

📦 A toolbox of utility functions I commonly use when programming in Python.
MIT License
0 stars 0 forks source link

Add `easyset` dataloaders #7

Open rosikand opened 2 years ago

rosikand commented 2 years ago

Initially I was going to make easyset a separate package but since the datasets are all pickled up, it doesn't make much sense to provide a whole new dependency. So you might as well just add a module here and have the following function:


def load_set(path_url):
    """
    Loads one of the pickled datasets. 
    Only pass in one of the viable options. 
    Arguments:
    -----------
    - path_url: url to the hosted .pkl file.
        - Can also specify one of the following 
        prefixes: 
            - 'cifar' which loads mini_cifar.pkl
            - 'cell' which loads mini_cell_segmentation.pkl
    """

    if path_url == "cifar":
        path_url = "https://stanford.edu/~rsikand/assets/datasets/mini_cifar.pkl"
    if path_url == "cell":
        path_url = "https://stanford.edu/~rsikand/assets/datasets/mini_cell_segmentation.pkl"

    loaded_object = cp.load(urlopen(path_url)) 

    return loaded_object
rosikand commented 2 years ago

Note: still have the separate repo to hold the hosted files (and put the above code snippet in __init__ maybe).