bioimage-io / collection-bioimage-io

(deprecated in favor of bioimage-io/collection) RDF collection for BioImage.IO
5 stars 9 forks source link

Update Mitochondria TOM20 artificial labelling - pix2pix #700

Closed bioimageiobot closed 8 months ago

bioimageiobot commented 8 months ago

This is an automatic PR created by the @bioimageiobot regarding changes to the resource item 10.5281/zenodo.10366411. The following version(s) will be added:

Please review the changes and make sure the new item or version(s) pass the following check list:

Maintainers: @esgomezm

Note: If you updated or re-uploaded another version for the current item on Zenodo, this PR won't be changed automatically. To proceed, you can do the following:

  1. Block this version, but keep looking for future versions: Edit the current resource.yaml and keep the top-level status field as accepted, but change the status under the current version to blocked.
  2. Accept this version and keep looking for future versions: Merge this PR for now.
  3. Keep proposed version(s) (and this resource in general if it is new) as pending: Close this PR without merging.

    Then wait for the CI on the main branch to complete. It should detect the new version(s) and create another PR for the new version(s).

Previous PRs of this resource: https://github.com/bioimage-io/collection-bioimage-io/pull/695

FynnBe commented 8 months ago

@esgomezm why does the zenodo record also include the packaged bioimage-model.zip? Looks like it contains the same files as are in the record. This file duplication makes it unclear which ones were tested, etc.. I'm going to ignore the bioimageio-model.zip as our convention is to upload the files individually to zenodo to allow selective download.

back to the real issue:

The error suggests that it is a pytorch bug:

terminate called after throwing an instance of 'c10::Error'
  what():  isTuple()INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1640811723911/work/aten/src/ATen/core/ivalue_inl.h":1400, please report a bug to PyTorch. Expected Tuple but got String
Exception raised from toTuple at /opt/conda/conda-bld/pytorch_1640811723911/work/aten/src/ATen/core/ivalue_inl.h:1400
full conda env used in CI ``` _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_kmp_llvm conda-forge aom 3.7.1 h59595ed_0 conda-forge bioimageio.core 0.5.11 pyhd8ed1ab_0 conda-forge bioimageio.spec 0.4.9.post5 pyhd8ed1ab_0 conda-forge blas 2.120 mkl conda-forge blas-devel 3.9.0 20_linux64_mkl conda-forge blosc 1.21.5 h0f2a231_0 conda-forge brotli 1.1.0 hd590300_1 conda-forge brotli-bin 1.1.0 hd590300_1 conda-forge brotli-python 1.1.0 py39h3d6467e_1 conda-forge brunsli 0.1 h9c3ff4c_0 conda-forge bzip2 1.0.8 hd590300_5 conda-forge c-blosc2 2.12.0 hb4ffafa_0 conda-forge ca-certificates 2023.11.17 hbcca054_0 conda-forge certifi 2023.11.17 pyhd8ed1ab_0 conda-forge charls 2.4.2 h59595ed_0 conda-forge charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge click 8.1.7 unix_pyh707e725_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge cpuonly 2.0 0 pytorch dav1d 1.2.1 hd590300_0 conda-forge freetype 2.12.1 h267a509_2 conda-forge giflib 5.2.1 h0b41bf4_3 conda-forge icu 73.2 h59595ed_0 conda-forge idna 3.6 pyhd8ed1ab_0 conda-forge imagecodecs 2024.1.1 py39hf9b8f0e_0 conda-forge imageio 2.33.1 pyh8c1a49c_0 conda-forge jxrlib 1.1 hd590300_3 conda-forge lcms2 2.16 hb7c19ff_0 conda-forge ld_impl_linux-64 2.40 h41732ed_0 conda-forge lerc 4.0.0 h27087fc_0 conda-forge libaec 1.1.2 h59595ed_1 conda-forge libavif16 1.0.3 hef5bec9_1 conda-forge libblas 3.9.0 20_linux64_mkl conda-forge libbrotlicommon 1.1.0 hd590300_1 conda-forge libbrotlidec 1.1.0 hd590300_1 conda-forge libbrotlienc 1.1.0 hd590300_1 conda-forge libcblas 3.9.0 20_linux64_mkl conda-forge libdeflate 1.19 hd590300_0 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 13.2.0 h807b86a_3 conda-forge libgfortran-ng 13.2.0 h69a702a_3 conda-forge libgfortran5 13.2.0 ha4646dd_3 conda-forge libhwloc 2.9.3 default_h554bfaf_1009 conda-forge libiconv 1.17 hd590300_2 conda-forge libjpeg-turbo 3.0.0 hd590300_1 conda-forge liblapack 3.9.0 20_linux64_mkl conda-forge liblapacke 3.9.0 20_linux64_mkl conda-forge libnsl 2.0.1 hd590300_0 conda-forge libpng 1.6.39 h753d276_0 conda-forge libsqlite 3.44.2 h2797004_0 conda-forge libstdcxx-ng 13.2.0 h7e041cc_3 conda-forge libtiff 4.6.0 ha9c0a0a_2 conda-forge libuuid 2.38.1 h0b41bf4_0 conda-forge libuv 1.46.0 hd590300_0 conda-forge libwebp-base 1.3.2 hd590300_0 conda-forge libxcb 1.15 h0b41bf4_0 conda-forge libxcrypt 4.4.36 hd590300_1 conda-forge libxml2 2.11.6 h232c23b_0 conda-forge libzlib 1.2.13 hd590300_5 conda-forge libzopfli 1.0.3 h9c3ff4c_0 conda-forge llvm-openmp 17.0.6 h4dfa4b3_0 conda-forge lz4-c 1.9.4 hcb278e6_0 conda-forge markdown-it-py 3.0.0 pyhd8ed1ab_0 conda-forge marshmallow 3.20.2 pyhd8ed1ab_0 conda-forge marshmallow-jsonschema 0.13.0 pyhd8ed1ab_0 conda-forge marshmallow-union 0.1.15.post1 pyhd8ed1ab_0 conda-forge mdurl 0.1.2 pyhd8ed1ab_0 conda-forge mkl 2023.2.0 h84fe81f_50496 conda-forge mkl-devel 2023.2.0 ha770c72_50496 conda-forge mkl-include 2023.2.0 h84fe81f_50496 conda-forge ncurses 6.4 h59595ed_2 conda-forge numpy 1.26.3 py39h474f0d3_0 conda-forge openjpeg 2.5.0 h488ebb8_3 conda-forge openssl 3.2.0 hd590300_1 conda-forge packaging 23.2 pyhd8ed1ab_0 conda-forge pandas 2.1.4 py39hddac248_0 conda-forge pillow 10.2.0 py39had0adad_0 conda-forge pip 23.3.2 pyhd8ed1ab_0 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge pygments 2.17.2 pyhd8ed1ab_0 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge python 3.9.18 h0755675_1_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-stdnum 1.19 pyhd8ed1ab_0 conda-forge python-tzdata 2023.4 pyhd8ed1ab_0 conda-forge python_abi 3.9 4_cp39 conda-forge pytorch 1.10.2 py3.9_cpu_0 pytorch pytorch-mutex 1.0 cpu pytorch pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge pyyaml 6.0.1 py39hd1e30aa_1 conda-forge rav1e 0.6.6 he8a937b_2 conda-forge readline 8.2 h8228510_1 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge rich 13.7.0 pyhd8ed1ab_0 conda-forge ruamel.yaml 0.18.5 py39hd1e30aa_0 conda-forge ruamel.yaml.clib 0.2.7 py39hd1e30aa_2 conda-forge setuptools 69.0.3 pyhd8ed1ab_0 conda-forge shellingham 1.5.4 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.10 h9fff704_0 conda-forge svt-av1 1.8.0 h59595ed_0 conda-forge tbb 2021.11.0 h00ab1b0_0 conda-forge tifffile 2023.12.9 pyhd8ed1ab_0 conda-forge tk 8.6.13 noxft_h4845f30_101 conda-forge tqdm 4.66.1 pyhd8ed1ab_0 conda-forge typer 0.9.0 pyhd8ed1ab_0 conda-forge typing-extensions 4.9.0 hd8ed1ab_0 conda-forge typing_extensions 4.9.0 pyha770c72_0 conda-forge tzdata 2023d h0c530f3_0 conda-forge urllib3 2.1.0 pyhd8ed1ab_0 conda-forge wheel 0.42.0 pyhd8ed1ab_0 conda-forge xarray 2023.12.0 pyhd8ed1ab_0 conda-forge xorg-libxau 1.0.11 hd590300_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xz 5.2.6 h166bdaf_0 conda-forge yaml 0.2.5 h7f98852_2 conda-forge zfp 1.0.1 h59595ed_0 conda-forge zlib-ng 2.0.7 h0b41bf4_0 conda-forge zstd 1.5.5 hfc55251_0 conda-forge ```

@esgomezm could you please post your working, local environment here for comparison? (mamba list -r)

esgomezm commented 8 months ago

Hi hi, The zip file contains the same files that are individually uploaded. It's a silly duplication but it is a way for me to have a plan B in any course whenever the bioengine or the zoo stops working.

Here it goes:

Mamba env ```` (bioimageio-core) esti@estimacbookair ~ % mamba list -r 2024-01-11 14:29:43 (rev 0) +ca-certificates-2023.11.17 (conda-forge/osx-arm64) +libcxx-16.0.6 (conda-forge/osx-arm64) +libffi-3.3 (conda-forge/osx-arm64) +libsqlite-3.44.2 (conda-forge/osx-arm64) +libzlib-1.2.13 (conda-forge/osx-arm64) +ncurses-6.4 (conda-forge/osx-arm64) +openssl-1.1.1w (conda-forge/osx-arm64) +pip-23.3.2 (conda-forge/noarch) +python-3.9.0 (conda-forge/osx-arm64) +readline-8.2 (conda-forge/osx-arm64) +setuptools-69.0.3 (conda-forge/noarch) +sqlite-3.44.2 (conda-forge/osx-arm64) +tk-8.6.13 (conda-forge/osx-arm64) +tzdata-2023d (conda-forge/noarch) +wheel-0.42.0 (conda-forge/noarch) +xz-5.2.6 (conda-forge/osx-arm64) +zlib-1.2.13 (conda-forge/osx-arm64) 2024-01-11 14:30:41 (rev 1) +aom-3.7.1 (conda-forge/osx-arm64) +bioimageio.core-0.5.11 (conda-forge/noarch) +bioimageio.spec-0.4.9.post5 (conda-forge/noarch) +blosc-1.21.5 (conda-forge/osx-arm64) +brotli-1.1.0 (conda-forge/osx-arm64) +brotli-bin-1.1.0 (conda-forge/osx-arm64) +brotli-python-1.1.0 (conda-forge/osx-arm64) +brunsli-0.1 (conda-forge/osx-arm64) +bzip2-1.0.8 (conda-forge/osx-arm64) +c-blosc2-2.12.0 (conda-forge/osx-arm64) +certifi-2023.11.17 (conda-forge/noarch) +charls-2.4.2 (conda-forge/osx-arm64) +charset-normalizer-3.3.2 (conda-forge/noarch) +click-8.1.7 (conda-forge/noarch) +colorama-0.4.6 (conda-forge/noarch) +cpuonly-2.0 (pytorch/noarch) +dav1d-1.2.1 (conda-forge/osx-arm64) +filelock-3.13.1 (conda-forge/noarch) +freetype-2.12.1 (conda-forge/osx-arm64) +fsspec-2023.12.2 (conda-forge/noarch) +giflib-5.2.1 (conda-forge/osx-arm64) +gmp-6.3.0 (conda-forge/osx-arm64) +gmpy2-2.1.2 (conda-forge/osx-arm64) +idna-3.6 (conda-forge/noarch) +imagecodecs-2024.1.1 (conda-forge/osx-arm64) +imageio-2.33.1 (conda-forge/noarch) +jinja2-3.1.3 (conda-forge/noarch) +jxrlib-1.1 (conda-forge/osx-arm64) +lcms2-2.16 (conda-forge/osx-arm64) +lerc-4.0.0 (conda-forge/osx-arm64) +libabseil-20230802.1 (conda-forge/osx-arm64) +libaec-1.1.2 (conda-forge/osx-arm64) +libavif16-1.0.3 (conda-forge/osx-arm64) +libblas-3.9.0 (conda-forge/osx-arm64) +libbrotlicommon-1.1.0 (conda-forge/osx-arm64) +libbrotlidec-1.1.0 (conda-forge/osx-arm64) +libbrotlienc-1.1.0 (conda-forge/osx-arm64) +libcblas-3.9.0 (conda-forge/osx-arm64) +libdeflate-1.19 (conda-forge/osx-arm64) +libgfortran-5.0.0 (conda-forge/osx-arm64) +libgfortran5-13.2.0 (conda-forge/osx-arm64) +libjpeg-turbo-3.0.0 (conda-forge/osx-arm64) +liblapack-3.9.0 (conda-forge/osx-arm64) +libopenblas-0.3.25 (conda-forge/osx-arm64) +libpng-1.6.39 (conda-forge/osx-arm64) +libprotobuf-4.24.4 (conda-forge/osx-arm64) +libtiff-4.6.0 (conda-forge/osx-arm64) +libuv-1.46.0 (conda-forge/osx-arm64) +libwebp-base-1.3.2 (conda-forge/osx-arm64) +libxcb-1.15 (conda-forge/osx-arm64) +libzopfli-1.0.3 (conda-forge/osx-arm64) +llvm-openmp-17.0.6 (conda-forge/osx-arm64) +lz4-c-1.9.4 (conda-forge/osx-arm64) +markdown-it-py-3.0.0 (conda-forge/noarch) +markupsafe-2.1.3 (conda-forge/osx-arm64) +marshmallow-3.20.2 (conda-forge/noarch) +marshmallow-jsonschema-0.13.0 (conda-forge/noarch) +marshmallow-union-0.1.15.post1 (conda-forge/noarch) +mdurl-0.1.2 (conda-forge/noarch) +mpc-1.3.1 (conda-forge/osx-arm64) +mpfr-4.2.1 (conda-forge/osx-arm64) +mpmath-1.3.0 (conda-forge/noarch) +networkx-3.2.1 (conda-forge/noarch) +nomkl-1.0 (conda-forge/noarch) +numpy-1.26.3 (conda-forge/osx-arm64) +openjpeg-2.5.0 (conda-forge/osx-arm64) +packaging-23.2 (conda-forge/noarch) +pandas-2.1.4 (conda-forge/osx-arm64) +pillow-10.2.0 (conda-forge/osx-arm64) +pthread-stubs-0.4 (conda-forge/osx-arm64) +pygments-2.17.2 (conda-forge/noarch) +pysocks-1.7.1 (conda-forge/noarch) +python-dateutil-2.8.2 (conda-forge/noarch) +python-stdnum-1.19 (conda-forge/noarch) +python-tzdata-2023.4 (conda-forge/noarch) +python_abi-3.9 (conda-forge/osx-arm64) +pytorch-2.1.0 (conda-forge/osx-arm64) +pytorch-mutex-1.0 (pytorch/noarch) +pytz-2023.3.post1 (conda-forge/noarch) +pyyaml-6.0.1 (conda-forge/osx-arm64) +rav1e-0.6.6 (conda-forge/osx-arm64) +requests-2.31.0 (conda-forge/noarch) +rich-13.7.0 (conda-forge/noarch) +ruamel.yaml-0.18.5 (conda-forge/osx-arm64) +ruamel.yaml.clib-0.2.7 (conda-forge/osx-arm64) +shellingham-1.5.4 (conda-forge/noarch) +six-1.16.0 (conda-forge/noarch) +sleef-3.5.1 (conda-forge/osx-arm64) +snappy-1.1.10 (conda-forge/osx-arm64) +svt-av1-1.8.0 (conda-forge/osx-arm64) +sympy-1.12 (conda-forge/noarch) +tifffile-2023.12.9 (conda-forge/noarch) +torchvision-0.16.1 (conda-forge/osx-arm64) +tqdm-4.66.1 (conda-forge/noarch) +typer-0.9.0 (conda-forge/noarch) +typing-extensions-4.9.0 (conda-forge/noarch) +typing_extensions-4.9.0 (conda-forge/noarch) +urllib3-2.1.0 (conda-forge/noarch) +xarray-2023.12.0 (conda-forge/noarch) +xorg-libxau-1.0.11 (conda-forge/osx-arm64) +xorg-libxdmcp-1.1.3 (conda-forge/osx-arm64) +yaml-0.2.5 (conda-forge/osx-arm64) +zfp-1.0.1 (conda-forge/osx-arm64) +zlib-ng-2.0.7 (conda-forge/osx-arm64) +zstd-1.5.5 (conda-forge/osx-arm64) ````
FynnBe commented 8 months ago

ah, well. A trivial issue!

The collection's default pytorch version is 1.10 (which was resolved to 1.10.2), but you have 2.1.0 in your env... The default has been used as pytorch_version is not specified in the rdf.yaml. (We don't have a default in spec, but currently do allow to not specify the pytorch version. So for practical purposes the collection repo had to choose a default).

The next minor bioimageio.spec release will make pytorch_version a mandatory field which will prevent this issue in the future.

FynnBe commented 8 months ago

thanks for sharing the env @esgomezm . I'll patch the rdf.yaml here in the collection repo...

esgomezm commented 8 months ago

Hi @FynnBe! Ok, then I'll update also the export to specify the torch version. I thought that the library was automatically testing what's the installed version. Now I see that there's a different error :D

FynnBe commented 8 months ago

Hi @FynnBe! Ok, then I'll update also the export to specify the torch version. I thought that the library was automatically testing what's the installed version. Now I see that there's a different error :D

my bad, I messed up the YAML syntax of the patch...

esgomezm commented 8 months ago

I'm excited about seeing this one becoming green! I think this will never change xD Screenshot 2024-01-23 at 10 45 59

bioimageiobot commented 8 months ago

preview preview-collection-json

esgomezm commented 8 months ago

passed!!!