Closed McWhity closed 1 month ago
https://github.com/isce-framework/dolphin/blob/main/docs/notebooks/walkthrough-basic.ipynb There might be a typo in section "Bounds subsetting". Highlighted in bold below
If you are interested in a smaller subset of the are, you can specify the --output-bounds to force the raster boudaries.
Interesting, that error is indeed weird.. What platform (linxus/mac/windows) are you running this on? Were you trying to use docker here? (I only ask because I saw the folder with docker
in the name)
Typos fixed here https://github.com/isce-framework/dolphin/pull/407
I am running it on Linux. No docker usage here. At first I tried to install it via docker but as I run into some errors I switched and just created a new environment. But I used "conda" instead of "mamba" to install dolphin. Tomorrow I will try it with mamba as well.
Right now I have to installations.
build-docker-image.sh
to build itUsing installation 1: I get the above mentioned error Using installation 2: I encountered other problems
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[4], line 25
23 url_file = slc_dir / \"urls.txt\"
24 with open(url_file, \"w\") as f:
---> 25 f.write(\"\
\".join(best_option.inputs))
28 chdir(slc_dir)
29 # Download 4in parallel
AttributeError: 'BurstSubsetOption' object has no attribute 'inputs'"
The problem is due to the version of package opera-utils
. Within the docker the version 0.8.0 is installed. When I checked Installation 1 I found version 0.12.0. Thus, you might need to update the package version in https://github.com/isce-framework/dolphin/blob/main/docker/specfile.txt
Another problem running the notebook was that I used as docker configuration python3
in which wget
is not installed. Thus I added the wget
installation to the Dockerfile
. You might consider adding
# Install wget
RUN apt-get update && apt-get install -y wget
to Dockerfile
With the mentioned adjustments I can run the notebook. However I am not able to visualize the output. There is no timeseries
folder in the folder work-walkthrough
. The folder structure looks like this (only 2 sub-directories are shown)
work-walkthrough/
├── dolphin.log
├── interferograms
│ ├── 20210606_20210618.int.cor.tif
│ ├── 20210606_20210618.int.tif
│ ├── 20210606_20210630.int.cor.tif
│ ├── 20210606_20210630.int.tif
│ ├── 20210606_20210712.int.cor.tif
│ ├── 20210606_20210712.int.tif
│ ├── 20210606_20210724.int.cor.tif
│ ├── 20210606_20210724.int.tif
│ ├── 20210606_20210805.int.cor.tif
│ ├── 20210606_20210805.int.tif
│ ├── 20210606_20210817.int.cor.tif
│ ├── 20210606_20210817.int.tif
│ ├── 20210606_20210829.int.cor.tif
│ ├── 20210606_20210829.int.tif
│ ├── 20210606_20210910.int.cor.tif
│ ├── 20210606_20210910.int.tif
│ ├── 20210606_20210922.int.cor.tif
│ ├── 20210606_20210922.int.tif
│ ├── 20210606_20211004.int.cor.tif
│ ├── 20210606_20211004.int.tif
│ ├── 20210606_20211016.int.cor.tif
│ ├── 20210606_20211016.int.tif
│ ├── 20210606_20211028.int.cor.tif
│ ├── 20210606_20211028.int.tif
│ ├── 20210606_20211109.int.cor.tif
│ ├── 20210606_20211109.int.tif
│ ├── 20210606_20211121.int.cor.tif
│ ├── 20210606_20211121.int.tif
│ ├── 20210606_20211203.int.cor.tif
│ ├── 20210606_20211203.int.tif
│ ├── 20210606_20211215.int.cor.tif
│ ├── 20210606_20211215.int.tif
│ ├── 20210606_20220219.int.cor.tif
│ ├── 20210606_20220219.int.tif
│ ├── 20210606_20220303.int.cor.tif
│ ├── 20210606_20220303.int.tif
│ ├── 20210606_20220315.int.cor.tif
│ ├── 20210606_20220315.int.tif
│ ├── 20210606_20220327.int.cor.tif
│ ├── 20210606_20220327.int.tif
│ ├── 20210606_20220408.int.cor.tif
│ ├── 20210606_20220408.int.tif
│ ├── 20210606_20220420.int.cor.tif
│ ├── 20210606_20220420.int.tif
│ ├── 20210606_20220502.int.cor.tif
│ ├── 20210606_20220502.int.tif
│ ├── 20210606_20220514.int.cor.tif
│ ├── 20210606_20220514.int.tif
│ ├── 20210606_20220526.int.cor.tif
│ ├── 20210606_20220526.int.tif
│ ├── amp_dispersion_looked.tif
│ ├── ps_mask_looked.tif
│ ├── shp_counts.tif
│ └── temporal_coherence.tif
├── t078_165573_iw2
│ ├── bounds_mask.tif
│ ├── combined_mask.tif
│ ├── interferograms
│ ├── linked_phase
│ ├── nodata_mask.tif
│ ├── PS
│ ├── slc_stack.vrt
│ ├── timeseries
│ └── unwrapped
├── t078_165574_iw2
│ ├── bounds_mask.tif
│ ├── combined_mask.tif
│ ├── interferograms
│ ├── linked_phase
│ ├── nodata_mask.tif
│ ├── PS
│ ├── slc_stack.vrt
│ ├── timeseries
│ └── unwrapped
└── unwrapped
├── 20210606_20210618.unw.conncomp.tif
├── 20210606_20210618.unw.tif
└── scratch
Here is the entire folder structure
work-walkthrough/
├── dolphin.log
├── interferograms
│ ├── 20210606_20210618.int.cor.tif
│ ├── 20210606_20210618.int.tif
│ ├── 20210606_20210630.int.cor.tif
│ ├── 20210606_20210630.int.tif
│ ├── 20210606_20210712.int.cor.tif
│ ├── 20210606_20210712.int.tif
│ ├── 20210606_20210724.int.cor.tif
│ ├── 20210606_20210724.int.tif
│ ├── 20210606_20210805.int.cor.tif
│ ├── 20210606_20210805.int.tif
│ ├── 20210606_20210817.int.cor.tif
│ ├── 20210606_20210817.int.tif
│ ├── 20210606_20210829.int.cor.tif
│ ├── 20210606_20210829.int.tif
│ ├── 20210606_20210910.int.cor.tif
│ ├── 20210606_20210910.int.tif
│ ├── 20210606_20210922.int.cor.tif
│ ├── 20210606_20210922.int.tif
│ ├── 20210606_20211004.int.cor.tif
│ ├── 20210606_20211004.int.tif
│ ├── 20210606_20211016.int.cor.tif
│ ├── 20210606_20211016.int.tif
│ ├── 20210606_20211028.int.cor.tif
│ ├── 20210606_20211028.int.tif
│ ├── 20210606_20211109.int.cor.tif
│ ├── 20210606_20211109.int.tif
│ ├── 20210606_20211121.int.cor.tif
│ ├── 20210606_20211121.int.tif
│ ├── 20210606_20211203.int.cor.tif
│ ├── 20210606_20211203.int.tif
│ ├── 20210606_20211215.int.cor.tif
│ ├── 20210606_20211215.int.tif
│ ├── 20210606_20220219.int.cor.tif
│ ├── 20210606_20220219.int.tif
│ ├── 20210606_20220303.int.cor.tif
│ ├── 20210606_20220303.int.tif
│ ├── 20210606_20220315.int.cor.tif
│ ├── 20210606_20220315.int.tif
│ ├── 20210606_20220327.int.cor.tif
│ ├── 20210606_20220327.int.tif
│ ├── 20210606_20220408.int.cor.tif
│ ├── 20210606_20220408.int.tif
│ ├── 20210606_20220420.int.cor.tif
│ ├── 20210606_20220420.int.tif
│ ├── 20210606_20220502.int.cor.tif
│ ├── 20210606_20220502.int.tif
│ ├── 20210606_20220514.int.cor.tif
│ ├── 20210606_20220514.int.tif
│ ├── 20210606_20220526.int.cor.tif
│ ├── 20210606_20220526.int.tif
│ ├── amp_dispersion_looked.tif
│ ├── ps_mask_looked.tif
│ ├── shp_counts.tif
│ └── temporal_coherence.tif
├── t078_165573_iw2
│ ├── bounds_mask.tif
│ ├── combined_mask.tif
│ ├── interferograms
│ │ ├── 20210606_20210618.int.vrt
│ │ ├── 20210606_20210630.int.vrt
│ │ ├── 20210606_20210712.int.vrt
│ │ ├── 20210606_20210724.int.vrt
│ │ ├── 20210606_20210805.int.vrt
│ │ ├── 20210606_20210817.int.vrt
│ │ ├── 20210606_20210829.int.vrt
│ │ ├── 20210606_20210910.int.vrt
│ │ ├── 20210606_20210922.int.vrt
│ │ ├── 20210606_20211004.int.vrt
│ │ ├── 20210606_20211016.int.vrt
│ │ ├── 20210606_20211028.int.vrt
│ │ ├── 20210606_20211109.int.vrt
│ │ ├── 20210606_20211121.int.vrt
│ │ ├── 20210606_20211203.int.vrt
│ │ ├── 20210606_20211215.int.vrt
│ │ ├── 20210606_20220219.int.vrt
│ │ ├── 20210606_20220303.int.vrt
│ │ ├── 20210606_20220315.int.vrt
│ │ ├── 20210606_20220327.int.vrt
│ │ ├── 20210606_20220408.int.vrt
│ │ ├── 20210606_20220420.int.vrt
│ │ ├── 20210606_20220502.int.vrt
│ │ ├── 20210606_20220514.int.vrt
│ │ └── 20210606_20220526.int.vrt
│ ├── linked_phase
│ │ ├── 20210606_20211121
│ │ │ ├── avg_coh_20210606_20211121.tif
│ │ │ ├── eigenvalues_20210606_20211121.tif
│ │ │ ├── estimator_20210606_20211121.tif
│ │ │ ├── shp_counts_20210606_20211121.tif
│ │ │ └── temporal_coherence_20210606_20211121.tif
│ │ ├── 20210606_20211121.vrt
│ │ ├── 20210606.slc.tif
│ │ ├── 20210618.slc.tif
│ │ ├── 20210630.slc.tif
│ │ ├── 20210712.slc.tif
│ │ ├── 20210724.slc.tif
│ │ ├── 20210805.slc.tif
│ │ ├── 20210817.slc.tif
│ │ ├── 20210829.slc.tif
│ │ ├── 20210910.slc.tif
│ │ ├── 20210922.slc.tif
│ │ ├── 20211004.slc.tif
│ │ ├── 20211016.slc.tif
│ │ ├── 20211028.slc.tif
│ │ ├── 20211109.slc.tif
│ │ ├── 20211121.slc.tif
│ │ ├── 20211203_20220526
│ │ │ ├── avg_coh_20211203_20220526.tif
│ │ │ ├── eigenvalues_20211203_20220526.tif
│ │ │ ├── estimator_20211203_20220526.tif
│ │ │ ├── shp_counts_20211203_20220526.tif
│ │ │ └── temporal_coherence_20211203_20220526.tif
│ │ ├── 20211203_20220526.vrt
│ │ ├── 20211203.slc.tif
│ │ ├── 20211215.slc.tif
│ │ ├── 20220219.slc.tif
│ │ ├── 20220303.slc.tif
│ │ ├── 20220315.slc.tif
│ │ ├── 20220327.slc.tif
│ │ ├── 20220408.slc.tif
│ │ ├── 20220420.slc.tif
│ │ ├── 20220502.slc.tif
│ │ ├── 20220514.slc.tif
│ │ ├── 20220526.slc.tif
│ │ ├── compressed_20210606_20210606_20211121.tif
│ │ ├── compressed_20210606_20211203_20220526.tif
│ │ ├── shp_counts_average_20210606_20220526.tif
│ │ └── temporal_coherence_average_20210606_20220526.tif
│ ├── nodata_mask.tif
│ ├── PS
│ │ ├── amp_dispersion_looked.tif
│ │ ├── amp_dispersion.tif
│ │ ├── amp_mean.tif
│ │ ├── ps_pixels_looked.tif
│ │ └── ps_pixels.tif
│ ├── slc_stack.vrt
│ ├── timeseries
│ └── unwrapped
├── t078_165574_iw2
│ ├── bounds_mask.tif
│ ├── combined_mask.tif
│ ├── interferograms
│ │ ├── 20210606_20210618.int.vrt
│ │ ├── 20210606_20210630.int.vrt
│ │ ├── 20210606_20210712.int.vrt
│ │ ├── 20210606_20210724.int.vrt
│ │ ├── 20210606_20210805.int.vrt
│ │ ├── 20210606_20210817.int.vrt
│ │ ├── 20210606_20210829.int.vrt
│ │ ├── 20210606_20210910.int.vrt
│ │ ├── 20210606_20210922.int.vrt
│ │ ├── 20210606_20211004.int.vrt
│ │ ├── 20210606_20211016.int.vrt
│ │ ├── 20210606_20211028.int.vrt
│ │ ├── 20210606_20211109.int.vrt
│ │ ├── 20210606_20211121.int.vrt
│ │ ├── 20210606_20211203.int.vrt
│ │ ├── 20210606_20211215.int.vrt
│ │ ├── 20210606_20220219.int.vrt
│ │ ├── 20210606_20220303.int.vrt
│ │ ├── 20210606_20220315.int.vrt
│ │ ├── 20210606_20220327.int.vrt
│ │ ├── 20210606_20220408.int.vrt
│ │ ├── 20210606_20220420.int.vrt
│ │ ├── 20210606_20220502.int.vrt
│ │ ├── 20210606_20220514.int.vrt
│ │ └── 20210606_20220526.int.vrt
│ ├── linked_phase
│ │ ├── 20210606_20211121
│ │ │ ├── avg_coh_20210606_20211121.tif
│ │ │ ├── eigenvalues_20210606_20211121.tif
│ │ │ ├── estimator_20210606_20211121.tif
│ │ │ ├── shp_counts_20210606_20211121.tif
│ │ │ └── temporal_coherence_20210606_20211121.tif
│ │ ├── 20210606_20211121.vrt
│ │ ├── 20210606.slc.tif
│ │ ├── 20210618.slc.tif
│ │ ├── 20210630.slc.tif
│ │ ├── 20210712.slc.tif
│ │ ├── 20210724.slc.tif
│ │ ├── 20210805.slc.tif
│ │ ├── 20210817.slc.tif
│ │ ├── 20210829.slc.tif
│ │ ├── 20210910.slc.tif
│ │ ├── 20210922.slc.tif
│ │ ├── 20211004.slc.tif
│ │ ├── 20211016.slc.tif
│ │ ├── 20211028.slc.tif
│ │ ├── 20211109.slc.tif
│ │ ├── 20211121.slc.tif
│ │ ├── 20211203_20220526
│ │ │ ├── avg_coh_20211203_20220526.tif
│ │ │ ├── eigenvalues_20211203_20220526.tif
│ │ │ ├── estimator_20211203_20220526.tif
│ │ │ ├── shp_counts_20211203_20220526.tif
│ │ │ └── temporal_coherence_20211203_20220526.tif
│ │ ├── 20211203_20220526.vrt
│ │ ├── 20211203.slc.tif
│ │ ├── 20211215.slc.tif
│ │ ├── 20220219.slc.tif
│ │ ├── 20220303.slc.tif
│ │ ├── 20220315.slc.tif
│ │ ├── 20220327.slc.tif
│ │ ├── 20220408.slc.tif
│ │ ├── 20220420.slc.tif
│ │ ├── 20220502.slc.tif
│ │ ├── 20220514.slc.tif
│ │ ├── 20220526.slc.tif
│ │ ├── compressed_20210606_20210606_20211121.tif
│ │ ├── compressed_20210606_20211203_20220526.tif
│ │ ├── shp_counts_average_20210606_20220526.tif
│ │ └── temporal_coherence_average_20210606_20220526.tif
│ ├── nodata_mask.tif
│ ├── PS
│ │ ├── amp_dispersion_looked.tif
│ │ ├── amp_dispersion.tif
│ │ ├── amp_mean.tif
│ │ ├── ps_pixels_looked.tif
│ │ └── ps_pixels.tif
│ ├── slc_stack.vrt
│ ├── timeseries
│ └── unwrapped
└── unwrapped
├── 20210606_20210618.unw.conncomp.tif
├── 20210606_20210618.unw.tif
└── scratch
└── scratch-20210606_20210618.int
Thanks for the details @McWhity ,
wget
to docker.inputs
error was from docker/specfile.txt
being outdated. the docker build uses the explicitly saved dependencies, so I updated that in #409 The output doesn't look like it was made, since it errored during the step making unwrapped/
.
I still don't understand what that error means, as I've never seen that and haven't run into it running it on my Mac, linux, or on Colab.. is there a chance you started running it outside of docker, then got the error during a docker run? If so, it might be a folder permissions thing. Do you get that same error running it again?
@scottstanie: The installation process with mamba as well as with docker works now.
However, using the workflow from the walkthrough-basic.ipynb
the output after Section Running the workflow
is still as mentioned above. Unfortunately, the workflow does not produce an folder work-walkthrough/timeseries
. The folder work-walkthough/t078_165573_iw2/timeseries
is created but there are no files stored in that folder. As you can see in the comment above the Interferograms and Linked phase files are there.
Hmm I'm sorry this still isn't working for you, through I'm struggling to reproduce this error.
Would you mind putting the exact steps you're running to hit this error repeatedly? That would help me debug if it's something with the notebook, with docker, or with your setup that I've under-specified. Are you using docker, but somehow executing the notebook? Running the notebook steps manually within docker?
Something to try- stripping everything around the notebook to see if dolphin
is failing for you. You could try (assuming that you still have the directory of HDF5 input files) just entering that directory, then running:
cd input_slcs/ # wherever the "OPERA_L2_CSLC-S1..." files are
rm -rf work-walkthrough
dolphin config --slc-files *.h5 --subdataset "/data/VV" --threads-per-worker 8 --strides 6 3 --n-parallel-bursts 2 --output-bounds -102.8 31.3 -102.6 31.5 --work-directory work-walkthrough
dolphin run dolphin_config.yaml
This runs in 4-5 mins on my macbook.
Alternatively, if you really want to use docker, pull the github-built image,
docker pull ghcr.io/isce-framework/dolphin:develop
docker run --user $(id -u):$(id -g) -v $PWD:/work --rm -it ghcr.io/isce-framework/dolphin:develop bash
# ...as above with config, run
If that still fails, then there's definitely something wrong with this setup.
I get the same problem if I run it with docker or in a virtualenv configuration with mamba installation process. But I think I found the problem. As suggested I did run the application in the terminal and there I got an error that is unfortunately not shown if I run the command in the jupyter notebook.
Traceback (most recent call last):
File "/home/tweiss/anaconda3/envs/dolphin-env/bin/dolphin", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/cli.py", line 32, in main
run_func(**arg_dict)
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/workflows/_cli_run.py", line 30, in run
displacement.run(cfg, debug=debug)
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/_log.py", line 90, in wrapper
result = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/workflows/displacement.py", line 233, in run
unwrapped_paths, conncomp_paths = unwrapping.run(
^^^^^^^^^^^^^^^
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/_log.py", line 90, in wrapper
result = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/workflows/unwrapping.py", line 82, in run
unwrapped_paths, conncomp_paths = unwrap.run(
^^^^^^^^^^^
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/unwrap/_unwrap.py", line 158, in run
exc.submit(
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/utils.py", line 680, in submit
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/site-packages/dolphin/unwrap/_unwrap.py", line 491, in unwrap
shutil.rmtree(scratchdir)
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/shutil.py", line 759, in rmtree
_rmtree_safe_fd(stack, onexc)
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/shutil.py", line 703, in _rmtree_safe_fd
onexc(func, path, err)
File "/home/tweiss/anaconda3/envs/dolphin-env/lib/python3.12/shutil.py", line 662, in _rmtree_safe_fd
os.rmdir(name, dir_fd=dirfd)
OSError: [Errno 39] Directory not empty: PosixPath('/media/tweiss/data/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/work-walkthrough~/unwrapped/scratch/scratch-20210606_20210712.int')
The problem is with shutil
. By trying to delete not needed processing files (https://github.com/isce-framework/dolphin/blob/main/src/dolphin/unwrap/_unwrap.py line 493) shutil
gives me an OSError
. If I check the directory afterwards it is actually empty. I don't know why the Error occurs but it does.
If I remove the delete option then everything works fine. I finally get the output in the timeseries
folder.
huh, so that shutil
error happens on a fresh environment after deleting the working directory, and running one unwrapping job at a time (the default)? that is so weird... I was wondering if it's some python 3.12 problem, but i don't see any relevant changes in this version. I had been running 3.11, but i tried with a new environment, on both a Macbook and a linux server, and both run through
What does your environment look like with python -c 'import dolphin; dolphin.show_versions()'
? here's mine after i did
mamba env create --file conda-env.yml
conda activate dolphin-env
pip install . snaphu
python -c 'import dolphin; dolphin.show_versions()'
dolphin version: 0.26.0.post1.dev31+g10d1019
Python deps:
h5py: 3.11.0
jax: 0.4.31
numba: 0.60.0
numpy: 2.0.2
opera-utils: 0.12.1
pydantic: 2.8.2
pyproj: 3.6.1
rasterio: 1.3.11
ruamel.yaml: 0.18.6
scipy: 1.14.1
threadpoolctl: 3.5.0
tqdm: 4.66.5
osgeo.gdal: 3.9.2
System:
python: 3.12.5 | packaged by conda-forge | (main, Aug 8 2024, 18:36:51) [GCC 12.4.0]
executable: /u/aurora-r0/staniewi/miniconda3/envs/dolphin-env/bin/python
machine: Linux-3.10.0-1160.118.1.el7.x86_64-x86_64-with-glibc2.17
optional GPU info:
jax: 0.4.31
gpu_is_available: True
also, could you post the dolphin_config.yaml
you are running?
dolphin version: 0.27.1.post1.dev27+geeabbf9
Python deps:
h5py: 3.11.0
jax: 0.4.31
numba: 0.60.0
numpy: 2.0.2
opera-utils: 0.12.1
pydantic: 2.8.2
pyproj: 3.6.1
rasterio: 1.3.11
ruamel.yaml: 0.18.6
scipy: 1.14.1
threadpoolctl: 3.5.0
tqdm: 4.66.5
osgeo.gdal: 3.9.2
System:
python: 3.12.5 | packaged by conda-forge | (main, Aug 8 2024, 18:36:51) [GCC 12.4.0]
executable: /home/tweiss/miniforge3/envs/dolphin-env/bin/python
machine: Linux-6.8.0-40-generic-x86_64-with-glibc2.35
optional GPU info:
jax: 0.4.31
gpu_is_available: False
The only difference that I see is in the dolphin version number and the used machine.
the dolphin_config.yaml looks like
input_options:
# If passing HDF5/NetCDF files, subdataset to use from CSLC files. .
# Type: string | null.
subdataset: /data/VV
# Format of dates contained in CSLC filenames.
# Type: string.
cslc_date_fmt: '%Y%m%d'
# Mask file used to ignore low correlation/bad data (e.g water mask). Convention is 0 for no
# data/invalid, and 1 for good data. Dtype must be uint8.
# Type: string | null.
mask_file:
# Name of sub-directory to use for writing output files.
# Type: string.
work_directory:
/media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/work-walkthrough~
# Don't resolve filepaths that are given as relative to be absolute.
# Type: boolean.
keep_paths_relative: false
worker_settings:
# Whether to use GPU for processing (if available).
# Type: boolean.
gpu_enabled: false
# Number of threads to use per worker. This sets the OMP_NUM_THREADS environment variable in
# each python process.
# Type: integer.
threads_per_worker: 8
# If processing separate spatial bursts, number of bursts to run in parallel for wrapped-
# phase-estimation.
# Type: integer.
n_parallel_bursts: 2
# Size (rows, columns) of blocks of data to load at a time.
# Type: array.
block_shape:
- 512
- 512
# Path to output log file (in addition to logging to `stderr`). Default logs to
# `dolphin.log` within `work_directory`.
# Type: string | null.
log_file:
# Time the config file was created.
# Type: string.
creation_time_utc: '2024-09-04T14:33:25.410139Z'
# REQUIRED: list of CSLC files, or newline-delimited file containing list of CSLC files.
# Type: array.
cslc_file_list:
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210606T005130Z_20240726T084639Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210606T005133Z_20240726T084639Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210618T005131Z_20240726T114540Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210618T005133Z_20240726T114540Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210630T005131Z_20240726T145129Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210630T005134Z_20240726T145129Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210712T005132Z_20240726T175557Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210712T005135Z_20240726T175557Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210724T005133Z_20240726T204530Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210724T005136Z_20240726T204530Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210805T005133Z_20240726T235123Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210805T005136Z_20240726T235123Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210817T005134Z_20240727T024621Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210817T005137Z_20240727T024621Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210829T005135Z_20240727T054138Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210829T005138Z_20240727T054138Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210910T005135Z_20240727T083819Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210910T005138Z_20240727T083819Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20210922T005136Z_20240727T115521Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20210922T005138Z_20240727T115521Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20211004T005136Z_20240727T150409Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20211004T005139Z_20240727T150409Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20211016T005136Z_20240727T180616Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20211016T005139Z_20240727T180616Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20211028T005136Z_20240727T205202Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20211028T005139Z_20240727T205202Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20211109T005136Z_20240728T001911Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20211109T005139Z_20240728T001911Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20211121T005136Z_20240728T024947Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20211121T005138Z_20240728T024947Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20211203T005135Z_20240728T060328Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20211203T005138Z_20240728T060328Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20211215T005135Z_20240728T084527Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20211215T005137Z_20240728T084527Z_S1B_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220219T005214Z_20240731T200844Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220219T005217Z_20240731T200844Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220303T005214Z_20240731T223010Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220303T005216Z_20240731T223010Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220315T005214Z_20240801T022007Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220315T005217Z_20240801T022007Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220327T005214Z_20240801T052724Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220327T005217Z_20240801T052724Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220408T005214Z_20240801T090520Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220408T005217Z_20240801T090520Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220420T005215Z_20240801T130307Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220420T005218Z_20240801T130307Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220502T005215Z_20240801T152841Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220502T005218Z_20240801T152841Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220514T005216Z_20240801T194925Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220514T005219Z_20240801T194925Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165573-IW2_20220526T005217Z_20240801T223223Z_S1A_VV_v1.1.h5
- /media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/OPERA_L2_CSLC-S1_T078-165574-IW2_20220526T005219Z_20240801T223223Z_S1A_VV_v1.1.h5
output_options:
# Output (x, y) resolution (in units of input data).
# Type: object | null.
output_resolution:
# Alternative to specifying output resolution: Specify the (x, y) strides (decimation
# factor) to perform while processing input. For example, strides of [4, 2] would turn an
# input resolution of [5, 10] into an output resolution of [20, 20].
# Type: object.
strides:
x: 6
y: 3
# Area of interest: [left, bottom, right, top] coordinates. e.g.
# `bbox=[-150.2,65.0,-150.1,65.5]`.
# Type: array | null.
bounds:
- -102.8
- 31.3
- -102.6
- 31.5
# EPSG code for the `bounds` coordinates, if specified.
# Type: integer.
bounds_epsg: 4326
# Options for `create_dataset` with h5py.
# Type: object.
hdf5_creation_options:
chunks:
- 128
- 128
compression: gzip
compression_opts: 4
shuffle: true
# GDAL creation options for GeoTIFF files.
# Type: array.
gtiff_creation_options:
- COMPRESS=lzw
- ZLEVEL=4
- BIGTIFF=yes
- TILED=yes
- INTERLEAVE=band
- BLOCKXSIZE=128
- BLOCKYSIZE=128
# Whether to add overviews to the output GeoTIFF files. This will increase file size, but
# can be useful for visualizing the data with web mapping tools. See
# https://gdal.org/programs/gdaladdo.html for more.
# Type: boolean.
add_overviews: true
# List of overview levels to create (if `add_overviews=True`).
# Type: array.
overview_levels:
- 4
- 8
- 16
- 32
- 64
ps_options:
# Amplitude dispersion threshold to consider a pixel a PS.
# Type: number.
amp_dispersion_threshold: 0.25
# Paths to existing Amplitude Dispersion file (1 per SLC region) for PS update calculation.
# If none provided, computed using the input SLC stack.
# Type: array.
amplitude_dispersion_files: []
# Paths to an existing Amplitude Mean files (1 per SLC region) for PS update calculation. If
# none provided, computed using the input SLC stack.
# Type: array.
amplitude_mean_files: []
phase_linking:
# Size of the ministack for sequential estimator.
# Type: integer.
ministack_size: 15
# Maximum number of compressed images to use in sequential estimator. If there are more
# ministacks than this, the earliest CCSLCs will be left out of the later stacks.
# Type: integer.
max_num_compressed: 5
half_window:
# Half window size (in pixels) for x direction.
# Type: integer.
x: 11
# Half window size (in pixels) for y direction.
# Type: integer.
y: 5
# Use EVD on the coherence instead of using the EMI algorithm.
# Type: boolean.
use_evd: false
# Beta regularization parameter for correlation matrix inversion. 0 is no regularization.
# Type: number.
beta: 0.0
# Method for finding SHPs during phase linking.
# Type: string.
# Options: ['glrt', 'ks', 'rect', 'rect'].
shp_method: glrt
# Significance level (probability of false alarm) for SHP tests.
# Type: number.
shp_alpha: 0.005
# StBAS parameter to include only nearest-N interferograms forphase linking. A
# `baseline_lag` of `n` will only include the closest`n` interferograms. `baseline_line`
# must be positive.
# Type: integer | null.
baseline_lag:
interferogram_network:
# For single-reference network: Index of the reference image in the network.
# Type: integer | null.
reference_idx: 0
# Max `n` to form the nearest-`n` interferograms by index.
# Type: integer | null.
max_bandwidth:
# Maximum temporal baseline of interferograms.
# Type: integer | null.
max_temporal_baseline:
# For manual-index network: list of (ref_idx, sec_idx) defining the interferograms to form.
# Type: array | null.
indexes:
unwrap_options:
# Whether to run the unwrapping step after wrapped phase estimation.
# Type: boolean.
run_unwrap: true
# Whether to run Goldstein filtering step on wrapped interferogram.
# Type: boolean.
run_goldstein: false
# Whether to run interpolation step on wrapped interferogram.
# Type: boolean.
run_interpolation: false
# Phase unwrapping method.
# Type: string.
# Options: ['snaphu', 'icu', 'phass', 'spurt', 'whirlwind'].
unwrap_method: snaphu
# Number of interferograms to unwrap in parallel.
# Type: integer.
n_parallel_jobs: 1
# Set wrapped phase/correlation to 0 where mask is 0 before unwrapping. .
# Type: boolean.
zero_where_masked: false
preprocess_options:
# Adaptive phase (Goldstein) filter exponent parameter.
# Type: number.
alpha: 0.5
# (for interpolation) Maximum radius to find scatterers.
# Type: integer.
max_radius: 51
# Threshold on the correlation raster to use for interpolation. Pixels with less than this
# value are replaced by a weighted combination of neighboring pixels.
# Type: number.
interpolation_cor_threshold: 0.5
snaphu_options:
# Number of tiles to split the inputs into using SNAPHU's internal tiling.
# Type: array.
ntiles:
- 1
- 1
# Amount of tile overlap (in pixels) along the (row, col) directions.
# Type: array.
tile_overlap:
- 0
- 0
# Number of tiles to unwrap in parallel for each interferogram.
# Type: integer.
n_parallel_tiles: 1
# Initialization method for SNAPHU.
# Type: string.
# Options: ['mcf', 'mst'].
init_method: mcf
# Statistical cost mode method for SNAPHU.
# Type: string.
# Options: ['defo', 'smooth'].
cost: smooth
tophu_options:
# Number of tiles to split the inputs into.
# Type: array.
ntiles:
- 1
- 1
# Extra multilook factor to use for the coarse unwrap.
# Type: array.
downsample_factor:
- 1
- 1
# Initialization method for SNAPHU.
# Type: string.
# Options: ['mcf', 'mst'].
init_method: mcf
# Statistical cost mode method for SNAPHU.
# Type: string.
# Options: ['defo', 'smooth'].
cost: smooth
spurt_options:
# Temporal coherence to pick pixels used on an irregular grid.
# Type: number.
temporal_coherence_threshold: 0.6
general_settings:
# Tile up data spatially.
# Type: boolean.
use_tiles: true
tiler_settings:
# Maximum number of tiles allowed.
# Type: integer.
max_tiles: 16
# Number of points used for determining tiles based on density.
# Type: integer.
target_points_for_generation: 120000
# Target points per tile when generating tiles.
# Type: integer.
target_points_per_tile: 800000
# Dilation factor of non-overlapping tiles. 0.05 would lead to 5 percent dilation of the
# tile.
# Type: number.
dilation_factor: 0.05
solver_settings:
# Number of workers for temporal unwrapping in parallel. Set value to <=0 to let workflow
# use default workers (ncpus - 1).
# Type: integer.
t_worker_count: 1
# Number of workers for spatial unwrapping in parallel. Set value to <=0 to let workflow use
# (ncpus - 1).
# Type: integer.
s_worker_count: 1
# Temporal unwrapping operations over spatial links are performed in batches and each batch
# is solved in parallel.
# Type: integer.
links_per_batch: 50000
# Temporal unwrapping costs.
# Type: string.
# Options: ['constant', 'distance', 'centroid'].
t_cost_type: constant
# Scale factor used to compute edge costs for temporal unwrapping.
# Type: number.
t_cost_scale: 100.0
# Spatial unwrapping costs.
# Type: string.
# Options: ['constant', 'distance', 'centroid'].
s_cost_type: constant
# Scale factor used to compute edge costs for spatial unwrapping.
# Type: number.
s_cost_scale: 100.0
merger_settings:
# Minimum number of overlap pixels to be considered valid.
# Type: integer.
min_overlap_points: 25
# Currently, only 'dirichlet' is supported.
# Type: dirichlet.
# Options: ['dirichlet'].
method: dirichlet
# Method used to estimate bulk offset between tiles.
# Type: string.
# Options: ['integer', 'L2'].
bulk_method: L2
timeseries_options:
# Whether to run the inversion step after unwrapping, if more than a single-reference
# network is used.
# Type: boolean.
run_inversion: true
# Reference point (row, col) used if performing a time series inversion. If not provided, a
# point will be selected from a consistent connected component with low amplitude
# dispersion.
# Type: array | null.
reference_point:
# Run the velocity estimation from the phase time series.
# Type: boolean.
run_velocity: true
# Pixels with correlation below this value will be masked out.
# Type: number.
correlation_threshold: 0.2
correction_options:
# List of weather-model files (one per date) for tropospheric corrections.
# Type: array.
troposphere_files: []
# Format of dates contained in weather-model filenames.
# Type: string.
tropo_date_fmt: '%Y%m%d'
# Package for tropospheric correction. Choices: pyaps, raider.
# Type: string.
tropo_package: pyaps
# Enumeration representing different tropospheric models.
# Type: string.
# Options: ['ECMWF', 'ERA5', 'HRES', 'ERAINT', 'ERAI', 'MERRA', 'NARR', 'HRRR', 'GMAO'].
tropo_model: ERA5
# Type of tropospheric delay.
# Type: string.
# Options: ['wet', 'dry', 'hydrostatic', 'comb'].
tropo_delay_type: comb
# List of GNSS-derived TEC maps for ionospheric corrections (one per date). Source is
# https://cddis.nasa.gov/archive/gnss/products/ionex/.
# Type: array.
ionosphere_files: []
# Line-of-sight geometry files for each burst/SLC stack area, for use in correction
# computations.
# Type: array.
geometry_files: []
# DEM file for tropospheric/ topographic phase corrections.
# Type: string | null.
dem_file:
Ok two questions-
/media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/work-walkthrough~
This is the work directory, is there supposed to be a tilde at the end?
Also, is /media a symlink to another drive? I noticed shutil had some setting about avoid symlink attacks but I haven't tried to understand that yet
/media/tweiss/data/Arbeit_einordnen/review_joss_dolphin/20240903_dolphin/dolphin/docs/notebooks/input_slcs/work-walkthrough~ This is the work directory, is there supposed to be a tilde at the end?
I don't know why there is a tilde at the end. Probably a typing error from my side. I removed the tilde and I did run the process again with the same Error as output.
Also, is /media a symlink to another drive? I noticed shutil had some setting about avoid symlink attacks but I haven't tried to understand that yet
Ok. This seems to be the problem. /media/tweiss/data
is not actually a symlink but a mounted internal hard drive. Nevertheless, I moved the output folder (work-warkthrough) to my home directory and there everything works fine. It is somehow strange.
I tried a quick and dirty fix by adding ignore_errors=True
. If I change the shutil command shutil.rmtree(scratchdir, ignore_errors=True)
, the error is ignored. The shutil command is still removing all files within folders like scatch-20210606_20220526.int
. But as the Error seems to occur during the removing process of the empty folder, the folder itself is not deleted.
Thanks for testing through to find that! A very strange bug indeed, but I don't see the hard in ignore errors for that step, so I'll add that in a PR
@openjournals/joss-reviews#6997
Regarding walkthrough-basic.ipynb
Within section "Running the workflow" if running command
!dolphin run dolphin_config.yaml
I get following errorSomehow the directory seems to be not empty. But when I checkout the folder "scratch-20210606_20210712.int" afterwards it is empty