search

ImperialCollegeLondon / SWMManywhere

SWMManywhere is used to derive and simulate a sewer network anywhere in the world
https://imperialcollegelondon.github.io/SWMManywhere/
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

`whitebox` occasional error #268

Closed barneydobson closed 4 weeks ago

barneydobson commented 1 month ago

Occasionally the flow_dir calculation fails during clip_to_catchments. This has only happened to me on a PBS jobarray - so might be something to do with that, unsure.

Progress: 98%
Progress: 99%
Progress: 100%
Progress: 101%
Progress: 102%
Saving data...
Output file written
Elapsed Time (excluding I/O): 0.8s
./whitebox_tools --run="D8Pointer" --wd="/var/tmp/pbs.9947331[33].pbs/tmpce_buvq3" --dem='/var/tmp/pbs.9947331[33].pbs/tmpce_buvq3/breached_dem.tif' --output='/var/tmp/pbs.9947331[33].pbs/tmpce_buvq3/flow_direction.tif' -v --compress_rasters=False

****************************
* Welcome to D8Pointer     *
* Powered by WhiteboxTools *
* www.whiteboxgeo.com      *
****************************
Reading data...
thread 'main' panicked at whitebox-common/src/configs/mod.rs:43:45:
Failed to parse config_file.json file.: Error("EOF while parsing a value", line: 1, column: 0)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Probably aren't helpful - but logs attached for reference. logs.zip

I think it's a rust thing... which i don't really know about

cheginit commented 1 month ago

If it happened on the cluster only, it can be due to using the system tmp dir when using python's tempfile. It can fill up quickly on a cluster, so it's best to create a local tmp directory in your own scratch storage instead of using the system's. You can use the dir arg in, for example, tempfile.TemporaryDirectory.