updates scripts to download FSD FMA and RIR datasets; deletes unused files

iranroman / SpatialScaper

Creative Commons Attribution 4.0 International

25 stars 4 forks source link

updates scripts to download FSD FMA and RIR datasets; deletes unused files #39

Closed iranroman closed 10 months ago

iranroman commented 10 months ago

This PR implements the steps to generate the file structure proposed in #33

Lingering action items include:

[x] Refactor scripts to also download and prepare the TAU RIRs (currently only METU implemented) and moving functionality to scripts/prepare_rirs.py.
[x] Redefining a scripts/utils.py given all the functionalities needed for this data preparation.
[x] Unit tests for these functionalities.
[ ] Functions to cleanup temporary zip files that are generated as part of the process (nice to have; could skip and leave as open issue).

This PR could either point to main or be merged with #31

iranroman commented 10 months ago

Weird. The command-line shows the move, but it shows up as a new file here.

sivannavis commented 10 months ago

Changed the filepath to store metu sofa file because it wasn't consistent with example generation code in API 0.1.0, originally in PR #38

iranroman commented 10 months ago

I have started some unit tests, specifically for the scripts directory.

This is the current coverage, which can be replicated via

coverage run -m pytest --verbose scripts/test_script_utils.py && coverage report --show-missing

we should aim for 100% coverage.

@adrianSRoman @ChrisIck @sivannavis feel free to have a look

iranroman commented 10 months ago

the scripts tests should fully cover script/utils.py and perhaps the desired file structure after the preparation scripts have run.

iranroman commented 10 months ago

@ChrisIck please incorporate your reorganization changes too. Happy to undo and reconsider my commits in light of what you want to contribute.

iranroman commented 10 months ago

I've been working a bit more on the scripts to download and prepare FSD50K, FMA, TAU-SRIRs, and METU. These functionalities are now implemented.

In general I'm satisfied with scripts/prepare_rirs.py, scripts/utils.py, and scripts/test_script_utils.py.

scripts/prepare_fsd50k_fma.py works fine, but I think it could be improved to robustly handle cases where the data is already present in a directory, or partially downloaded from a previous try, or completely absent. Maybe this is just a "nice to have" for now and we could also leave open as an issue that we work on in the future.

Unit test coverage of scripts/utils.py is currently 100%.

There are still files to relocate given the functionalities we pursue.

Oh, and there are still many redundant files to delete, but I'll await review from @ChrisIck @adrianSRoman and @sivannavis to confirm which files are ready to go.

iranroman commented 10 months ago

@ChrisIck I'm relocating notebooks to demo_notebooks as requested in our weekly Spatial Scaper Notes.

Also, for now I'm keeping the db_configs directory, but I think we should aim at deprecating it and deleting it in a future version of the API.

iranroman commented 10 months ago

@ChrisIck I'm also moving the file called room_sim.py to spatialscaper.

room_scaper/tau_loading.py is now part of spatialscaper/tau_utils.py