notebook with example visuals: docker build failed on dfm_tools pip install

[backeb]

• [x] load all raw data from run (i.e. load tttz_waq_*_map.nc)
• [x] plot SST and surface currents on a map
• [x] plot some water quality variable on a map together with currents
• [x] plot a timeseries of SST for a specific location in the domain
• [x] plot other variables of interest to https://hiseaproject.com/ (aquaculture and ports)
• [ ] update Dockerfile and environment.yml for example_visuals.ipynb

[backeb]

hi @lorincmeszaros,

hope you're enjoying your holiday.

if you have example visualisations from the HiSea project, could you please share them here?

thanks Björn

[lorincmeszaros]

While exploring the dedicated Hisea font-end: https://hiseaproject.com/the-platform/overview/

You can check different visualization possibilities we used but the basic elements are:

• map of the top layer
• time series at selected locations

[backeb]

When running docker build in /notebooks directory I get the below error.

Seems to be related to:

• https://github.com/Deltares/HYDROLIB-core/issues/444#issuecomment-1488542448

Tried workaround:

• https://github.com/openearth/bed-data-preparation/issues/1#issuecomment-1545822906

Error:

#0 630.9 Building wheels for collected packages: dfm-tools, meshkernel, xugrid, bottleneck, cdsapi, cftime, docopt, numba-celltree
#0 630.9   Building wheel for dfm-tools (setup.py): started
#0 630.9   Building wheel for dfm-tools (setup.py): finished with status 'done'
#0 630.9   Created wheel for dfm-tools: filename=dfm_tools-0.11.7-py3-none-any.whl size=75081 sha256=8451e11e784cd74b45bb940e145faac04d80f1ad7947abd4d05099dd769a0cfd
#0 630.9   Stored in directory: /tmp/pip-ephem-wheel-cache-tbzj9_x3/wheels/06/78/9c/6c597a658652da7ad399f0d749274405781c024d23dd13582a
#0 630.9   Building wheel for meshkernel (pyproject.toml): started
#0 630.9   Building wheel for meshkernel (pyproject.toml): still running...
#0 630.9   Building wheel for meshkernel (pyproject.toml): finished with status 'error'
#0 630.9   Building wheel for xugrid (pyproject.toml): started
#0 630.9   Building wheel for xugrid (pyproject.toml): finished with status 'done'
#0 630.9   Created wheel for xugrid: filename=xugrid-0.5.0-py3-none-any.whl size=91095 sha256=8fb5d2112bd84e77655ec9a80023e13e8faacc0b21e39315b8b3b761c8756c49
#0 630.9   Stored in directory: /tmp/pip-ephem-wheel-cache-tbzj9_x3/wheels/be/52/a6/54e669bd12cb864cb49f0c610430381ab976ee7e5c8260289c
#0 630.9   Building wheel for bottleneck (pyproject.toml): started
#0 630.9   Building wheel for bottleneck (pyproject.toml): finished with status 'error'
#0 630.9   Building wheel for cdsapi (setup.py): started
#0 630.9   Building wheel for cdsapi (setup.py): finished with status 'done'
#0 630.9   Created wheel for cdsapi: filename=cdsapi-0.6.1-py2.py3-none-any.whl size=12021 sha256=f9943be8187b870a482e29ebbf279937a0faa039ef881af49e4eb0c789dc5ee2
#0 630.9   Stored in directory: /tmp/pip-ephem-wheel-cache-tbzj9_x3/wheels/4d/b6/82/16fe50c5ba1bc796d79375fabe24c0a5f99340d50fea2bc4bb
#0 630.9   Building wheel for cftime (pyproject.toml): started
#0 630.9   Building wheel for cftime (pyproject.toml): finished with status 'error'
#0 630.9   Building wheel for docopt (setup.py): started
#0 630.9   Building wheel for docopt (setup.py): finished with status 'done'
#0 630.9   Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13723 sha256=c140bf356b5418f4eace187d456199f14e38ff314129dc02dbd5b789fc6e62dc
#0 630.9   Stored in directory: /tmp/pip-ephem-wheel-cache-tbzj9_x3/wheels/70/4a/46/1309fc853b8d395e60bafaf1b6df7845bdd82c95fd59dd8d2b
#0 630.9   Building wheel for numba-celltree (pyproject.toml): started
#0 630.9   Building wheel for numba-celltree (pyproject.toml): finished with status 'done'
#0 630.9   Created wheel for numba-celltree: filename=numba_celltree-0.1.6-py3-none-any.whl size=33960 sha256=147117036e68f5dfde2be71c6670c7f70da900559c57e9750a96d82461aaad4d
#0 630.9   Stored in directory: /tmp/pip-ephem-wheel-cache-tbzj9_x3/wheels/fd/40/6d/b11983289e9b230c428a1c17002bb4d6487c14f45555a624af
#0 630.9 Successfully built dfm-tools xugrid cdsapi docopt numba-celltree
#0 630.9 Failed to build meshkernel bottleneck cftime
#0 630.9 Pip subprocess error:
#0 630.9 WARNING: The directory '/home/jovyan/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
#0 630.9   Running command git clone --filter=blob:none --quiet https://github.com/openearth/dfm_tools.git /tmp/pip-req-build-1l2yh9l6
#0 630.9   error: subprocess-exited-with-error
#0 630.9   
#0 630.9   × Building wheel for meshkernel (pyproject.toml) did not run successfully.
#0 630.9   │ exit code: 1
#0 630.9   ╰─> [19 lines of output]
#0 630.9       running bdist_wheel
#0 630.9       running build
#0 630.9       running build_py
#0 630.9       creating build
#0 630.9       creating build/lib.linux-aarch64-cpython-39
#0 630.9       creating build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       copying meshkernel/factories.py -> build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       copying meshkernel/meshkernel.py -> build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       copying meshkernel/py_structures.py -> build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       copying meshkernel/utils.py -> build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       copying meshkernel/c_structures.py -> build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       copying meshkernel/__init__.py -> build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       copying meshkernel/version.py -> build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       copying meshkernel/errors.py -> build/lib.linux-aarch64-cpython-39/meshkernel
#0 630.9       running build_ext
#0 630.9       git clone https://github.com/Deltares/MeshKernel
#0 630.9       Cloning into 'MeshKernel'...
#0 630.9       cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DADD_UNIT_TESTS_PROJECTS=OFF
#0 630.9       error: command 'cmake' failed: No such file or directory
#0 630.9       [end of output]
#0 630.9   
#0 630.9   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 630.9   ERROR: Failed building wheel for meshkernel
#0 630.9   error: subprocess-exited-with-error
#0 630.9   
#0 630.9   × Building wheel for bottleneck (pyproject.toml) did not run successfully.
#0 630.9   │ exit code: 1
#0 630.9   ╰─> [128 lines of output]
#0 630.9       running bdist_wheel
#0 630.9       running build
#0 630.9       running build_py
#0 630.9       creating build
#0 630.9       creating build/lib.linux-aarch64-cpython-39
#0 630.9       creating build/lib.linux-aarch64-cpython-39/bottleneck
#0 630.9       copying bottleneck/__init__.py -> build/lib.linux-aarch64-cpython-39/bottleneck
#0 630.9       copying bottleneck/_version.py -> build/lib.linux-aarch64-cpython-39/bottleneck
#0 630.9       copying bottleneck/_pytesttester.py -> build/lib.linux-aarch64-cpython-39/bottleneck
#0 630.9       creating build/lib.linux-aarch64-cpython-39/bottleneck/slow
#0 630.9       copying bottleneck/slow/nonreduce_axis.py -> build/lib.linux-aarch64-cpython-39/bottleneck/slow
#0 630.9       copying bottleneck/slow/nonreduce.py -> build/lib.linux-aarch64-cpython-39/bottleneck/slow
#0 630.9       copying bottleneck/slow/__init__.py -> build/lib.linux-aarch64-cpython-39/bottleneck/slow
#0 630.9       copying bottleneck/slow/move.py -> build/lib.linux-aarch64-cpython-39/bottleneck/slow
#0 630.9       copying bottleneck/slow/reduce.py -> build/lib.linux-aarch64-cpython-39/bottleneck/slow
#0 630.9       creating build/lib.linux-aarch64-cpython-39/bottleneck/benchmark
#0 630.9       copying bottleneck/benchmark/bench_detailed.py -> build/lib.linux-aarch64-cpython-39/bottleneck/benchmark
#0 630.9       copying bottleneck/benchmark/autotimeit.py -> build/lib.linux-aarch64-cpython-39/bottleneck/benchmark
#0 630.9       copying bottleneck/benchmark/bench.py -> build/lib.linux-aarch64-cpython-39/bottleneck/benchmark
#0 630.9       copying bottleneck/benchmark/__init__.py -> build/lib.linux-aarch64-cpython-39/bottleneck/benchmark
#0 630.9       creating build/lib.linux-aarch64-cpython-39/bottleneck/src
#0 630.9       copying bottleneck/src/bn_template.py -> build/lib.linux-aarch64-cpython-39/bottleneck/src
#0 630.9       copying bottleneck/src/bn_config.py -> build/lib.linux-aarch64-cpython-39/bottleneck/src
#0 630.9       copying bottleneck/src/__init__.py -> build/lib.linux-aarch64-cpython-39/bottleneck/src
#0 630.9       creating build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/test_template.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/list_input_test.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/scalar_input_test.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/input_modification_test.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/memory_test.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/reduce_test.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/nonreduce_axis_test.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/__init__.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/move_test.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/nonreduce_test.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/util.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       copying bottleneck/tests/common.py -> build/lib.linux-aarch64-cpython-39/bottleneck/tests
#0 630.9       creating build/lib.linux-aarch64-cpython-39/bottleneck/tests/data
#0 630.9       creating build/lib.linux-aarch64-cpython-39/bottleneck/tests/data/template_test
#0 630.9       copying bottleneck/tests/data/template_test/truth.c -> build/lib.linux-aarch64-cpython-39/bottleneck/tests/data/template_test
#0 630.9       copying bottleneck/tests/data/template_test/test_template.c -> build/lib.linux-aarch64-cpython-39/bottleneck/tests/data/template_test
#0 630.9       UPDATING build/lib.linux-aarch64-cpython-39/bottleneck/_version.py
#0 630.9       set build/lib.linux-aarch64-cpython-39/bottleneck/_version.py to '1.3.7'
#0 630.9       running build_ext
#0 630.9       running config
#0 630.9       compiling '_configtest.c':
#0 630.9       
#0 630.9       #pragma GCC diagnostic error "-Wattributes"
#0 630.9       
#0 630.9       int __attribute__((optimize("O3"))) have_attribute_optimize_opt_3(void*);
#0 630.9       
#0 630.9       int main(void)
#0 630.9       {
#0 630.9           return 0;
#0 630.9       }
#0 630.9       
#0 630.9       gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O3 -Wall -fPIC -O3 -isystem /opt/conda/include -fPIC -O3 -isystem /opt/conda/include -fPIC -c _configtest.c -o _configtest.o
#0 630.9       failure.
#0 630.9       removing: _configtest.c _configtest.o
#0 630.9       compiling '_configtest.c':
#0 630.9       
#0 630.9       #ifndef __cplusplus
#0 630.9       static inline int static_func (void)
#0 630.9       {
#0 630.9           return 0;
#0 630.9       }
#0 630.9       inline int nostatic_func (void)
#0 630.9       {
#0 630.9           return 0;
#0 630.9       }
#0 630.9       #endif
#0 630.9       int main(void) {
#0 630.9           int r1 = static_func();
#0 630.9           int r2 = nostatic_func();
#0 630.9           return r1 + r2;
#0 630.9       }
#0 630.9       
#0 630.9       gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O3 -Wall -fPIC -O3 -isystem /opt/conda/include -fPIC -O3 -isystem /opt/conda/include -fPIC -c _configtest.c -o _configtest.o
#0 630.9       failure.
#0 630.9       removing: _configtest.c _configtest.o
#0 630.9       compiling '_configtest.c':
#0 630.9       
#0 630.9       #ifndef __cplusplus
#0 630.9       static __inline__ int static_func (void)
#0 630.9       {
#0 630.9           return 0;
#0 630.9       }
#0 630.9       __inline__ int nostatic_func (void)
#0 630.9       {
#0 630.9           return 0;
#0 630.9       }
#0 630.9       #endif
#0 630.9       int main(void) {
#0 630.9           int r1 = static_func();
#0 630.9           int r2 = nostatic_func();
#0 630.9           return r1 + r2;
#0 630.9       }
#0 630.9       
#0 630.9       gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O3 -Wall -fPIC -O3 -isystem /opt/conda/include -fPIC -O3 -isystem /opt/conda/include -fPIC -c _configtest.c -o _configtest.o
#0 630.9       failure.
#0 630.9       removing: _configtest.c _configtest.o
#0 630.9       compiling '_configtest.c':
#0 630.9       
#0 630.9       #ifndef __cplusplus
#0 630.9       static __inline int static_func (void)
#0 630.9       {
#0 630.9           return 0;
#0 630.9       }
#0 630.9       __inline int nostatic_func (void)
#0 630.9       {
#0 630.9           return 0;
#0 630.9       }
#0 630.9       #endif
#0 630.9       int main(void) {
#0 630.9           int r1 = static_func();
#0 630.9           int r2 = nostatic_func();
#0 630.9           return r1 + r2;
#0 630.9       }
#0 630.9       
#0 630.9       gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O3 -Wall -fPIC -O3 -isystem /opt/conda/include -fPIC -O3 -isystem /opt/conda/include -fPIC -c _configtest.c -o _configtest.o
#0 630.9       failure.
#0 630.9       removing: _configtest.c _configtest.o
#0 630.9       building 'bottleneck.reduce' extension
#0 630.9       creating build/temp.linux-aarch64-cpython-39
#0 630.9       creating build/temp.linux-aarch64-cpython-39/bottleneck
#0 630.9       creating build/temp.linux-aarch64-cpython-39/bottleneck/src
#0 630.9       gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O3 -Wall -fPIC -O3 -isystem /opt/conda/include -fPIC -O3 -isystem /opt/conda/include -fPIC -I/tmp/pip-build-env-pcqmuhts/overlay/lib/python3.9/site-packages/numpy/core/include -I/opt/conda/include/python3.9 -Ibottleneck/src -c bottleneck/src/reduce.c -o build/temp.linux-aarch64-cpython-39/bottleneck/src/reduce.o -O2
#0 630.9       error: command 'gcc' failed: No such file or directory
#0 630.9       [end of output]
#0 630.9   
#0 630.9   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 630.9   ERROR: Failed building wheel for bottleneck
#0 630.9   error: subprocess-exited-with-error
#0 630.9   
#0 630.9   × Building wheel for cftime (pyproject.toml) did not run successfully.
#0 630.9   │ exit code: 1
#0 630.9   ╰─> [18 lines of output]
#0 630.9       running bdist_wheel
#0 630.9       running build
#0 630.9       running build_py
#0 630.9       creating build
#0 630.9       creating build/lib.linux-aarch64-cpython-39
#0 630.9       creating build/lib.linux-aarch64-cpython-39/cftime
#0 630.9       copying src/cftime/__init__.py -> build/lib.linux-aarch64-cpython-39/cftime
#0 630.9       copying src/cftime/_strptime.py -> build/lib.linux-aarch64-cpython-39/cftime
#0 630.9       running build_ext
#0 630.9       building 'cftime._cftime' extension
#0 630.9       creating build/temp.linux-aarch64-cpython-39
#0 630.9       creating build/temp.linux-aarch64-cpython-39/tmp
#0 630.9       creating build/temp.linux-aarch64-cpython-39/tmp/pip-install-_e1ugemb
#0 630.9       creating build/temp.linux-aarch64-cpython-39/tmp/pip-install-_e1ugemb/cftime_7080bfb85fbe45828e01983fdf070554
#0 630.9       creating build/temp.linux-aarch64-cpython-39/tmp/pip-install-_e1ugemb/cftime_7080bfb85fbe45828e01983fdf070554/src
#0 630.9       creating build/temp.linux-aarch64-cpython-39/tmp/pip-install-_e1ugemb/cftime_7080bfb85fbe45828e01983fdf070554/src/cftime
#0 630.9       gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O3 -Wall -fPIC -O3 -isystem /opt/conda/include -fPIC -O3 -isystem /opt/conda/include -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/tmp/pip-build-env-wgb913kp/overlay/lib/python3.9/site-packages/numpy/core/include -I/opt/conda/include/python3.9 -c /tmp/pip-install-_e1ugemb/cftime_7080bfb85fbe45828e01983fdf070554/src/cftime/_cftime.c -o build/temp.linux-aarch64-cpython-39/tmp/pip-install-_e1ugemb/cftime_7080bfb85fbe45828e01983fdf070554/src/cftime/_cftime.o
#0 630.9       error: command 'gcc' failed: No such file or directory
#0 630.9       [end of output]
#0 630.9   
#0 630.9   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 630.9   ERROR: Failed building wheel for cftime
#0 630.9 ERROR: Could not build wheels for meshkernel, bottleneck, cftime, which is required to install pyproject.toml-based projects
#0 630.9 
#0 630.9 
#0 630.9 failed
#0 630.9 
#0 630.9 CondaEnvException: Pip failed
#0 630.9 
------
Dockerfile:9
--------------------
   7 |     
   8 |     # update the base environment with dependencies from environment.yaml
   9 | >>> RUN mamba env update --name base --file /tmp/environment.yaml --prune
  10 |     RUN jupyter labextension install jupyter-matplotlib
  11 |     
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c mamba env update --name base --file /tmp/environment.yaml --prune" did not complete successfully: exit code: 1