No file inside of space/ should be used as a script. In this case, datasets.py. What is currently inside of if __name__ == "__main__" in datasets.py should be used only for demonstration and development, not as a script.
We will want to use the space folder as a library (and pip install it). So the scripts should live elsewhere. Maybe add a folder scripts (or api) at the root and move there what is currently in data.sh and what's currently inside the __main__ in datasets.py as a new script that imports functions from our space library.
The logic in get_dataset_metadata_and_path is currently too hard coded. One solution is to copy the logic of ogb and add a masterfile that we can use as a lookup table.
No file inside of space/ should be used as a script. In this case,
datasets.py
. What is currently inside ofif __name__ == "__main__"
indatasets.py
should be used only for demonstration and development, not as a script.We will want to use the space folder as a library (and pip install it). So the scripts should live elsewhere. Maybe add a folder
scripts
(orapi
) at the root and move there what is currently indata.sh
and what's currently inside the__main__
indatasets.py
as a new script that imports functions from ourspace
library.The logic in
get_dataset_metadata_and_path
is currently too hard coded. One solution is to copy the logic ofogb
and add a masterfile that we can use as a lookup table.