azavea / raster-vision

An open source library and framework for deep learning on satellite and aerial imagery.
https://docs.rastervision.io
Other
2.03k stars 380 forks source link

BATCH_CPU_JOB_QUEUE requires a value parseable by str #2152

Closed bnascimento closed 2 months ago

bnascimento commented 2 months ago

🐛 Bug

I have build the docker and was running the docker via docker/run --gpu. aws sync the vegas data and I was following the documentation for the examples, but it failed with "BATCH_CPU_JOB_QUEUE requires a value parseable by str" when I checked locally upon the following one liner: (base) root@9e562187cf14:/opt/src# rastervision run batch rastervision.pytorch_backend.examples.semantic_segmentation.spacenet_vegas -a raw_uri $RAW_URI -a target $TARGET -a root_uri $ROOT_URI -a test True

To Reproduce

Steps to reproduce the behavior:

1.build and run docker 2.aws sync data 3.follow examples guide https://docs.rastervision.io/en/stable/framework/examples.html

... json_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img1627.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img2972.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img6031.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img5145.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img4637.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img3493.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img6320.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img1310.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img5192.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img6169.geojson', '/opt/data/spacenet/SN2_buildings/train/AOI_2_Vegas/geojson_buildings/SN2_buildings_train_AOI_2_Vegas_geojson_buildings_img6168.geojson'] Stages to run: ['analyze', 'train', 'predict', 'eval', 'bundle'] Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 250, in _main() File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 246, in _main main() File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 175, in run _run_pipeline(cfg, runner, tmp_dir, splits, commands, File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 125, in _run_pipeline runner.run( File "/opt/src/rastervision_aws_batch/rastervision/aws_batch/aws_batch_runner.py", line 47, in run job_id = self.run_command( ^^^^^^^^^^^^^^^^^ File "/opt/src/rastervision_aws_batch/rastervision/aws_batch/aws_batch_runner.py", line 148, in run_command job_queue = batch_config(f'{device}_job_queue') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/everett/manager.py", line 1397, in call raise ConfigurationMissingError(msg, namespace, key, parser) everett.ConfigurationMissingError: BATCH_CPU_JOB_QUEUE requires a value parseable by str Project docs: Check https://docs.rastervision.io/ for docs. Switch to the version being run and search for Raster Vision Configuration.

Expected behavior

Test if configuration would be correct

Environment

Running Raster Vision directly in Windows is not supported, and we recommend that you run it from within a Docker container.

Additional context

bnascimento commented 2 months ago

nevermind I needed to run local, instead of run batch

AdeelH commented 2 months ago

Great that you were able to figure it out! Closing this issue.