No file inside of space/ should be used as a script. In this case, datasets.py. What is currently inside of if __name__ == "__main__" in datasets.py should be used only for demonstration and development, not as a script.
We will want to use the space folder as a library (and pip install it). So the scripts should live elsewhere. Maybe add a folder scripts (or api) at the root and move there what is currently in data.sh and what's currently inside the __main__ in datasets.py as a new script that imports functions from our space library.
The logic in get_dataset_metadata_and_path is currently too hard coded. One solution is to copy the logic of ogb and add a masterfile that we can use as a lookup table.
mauriciogtec
commented
1 year ago
No file inside of space/ should be used as a script. In this case,
datasets.py. What is currently inside ofif __name__ == "__main__"indatasets.pyshould be used only for demonstration and development, not as a script.We will want to use the space folder as a library (and pip install it). So the scripts should live elsewhere. Maybe add a folder
scripts(orapi) at the root and move there what is currently indata.shand what's currently inside the__main__indatasets.pyas a new script that imports functions from ourspacelibrary.The logic in
get_dataset_metadata_and_pathis currently too hard coded. One solution is to copy the logic ofogband add a masterfile that we can use as a lookup table.