mahmoodlab / CLAM

Data-efficient and weakly supervised computational pathology on whole slide images - Nature Biomedical Engineering
http://clam.mahmoodlab.org
GNU General Public License v3.0
1.12k stars 364 forks source link

Issue creating env from yaml file #36

Closed surya-narayanan closed 3 years ago

surya-narayanan commented 3 years ago

Hi,

When I tried creating the env using

conda env create -n clam -f clam.yaml

I get the following error

json.decoder.JSONDecodeError: Unterminated string starting at: line 830302 column 14 (char 25006056)

Here is my full error report

Collecting package metadata (repodata.json): failed

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 263, in _load
        repodata_fn=self.repodata_fn)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 613, in fetch_repodata_remote_request
        raise Response304ContentUnchanged()
    conda.core.subdir_data.Response304ContentUnchanged

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.7/site-packages/conda/exceptions.py", line 1079, in __call__
        return func(*args, **kwargs)
      File "/opt/conda/lib/python3.7/site-packages/conda_env/cli/main.py", line 80, in do_call
        exit_code = getattr(module, func_name)(args, parser)
      File "/opt/conda/lib/python3.7/site-packages/conda_env/cli/main_create.py", line 118, in execute
        result[installer_type] = installer.install(prefix, pkg_specs, args, env)
      File "/opt/conda/lib/python3.7/site-packages/conda_env/installers/conda.py", line 32, in install
        prune=getattr(args, 'prune', False), update_modifier=UpdateModifier.FREEZE_INSTALLED)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/solve.py", line 117, in solve_for_transaction
        should_retry_solve)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/solve.py", line 158, in solve_for_diff
        force_remove, should_retry_solve)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/solve.py", line 262, in solve_final_state
        ssc = self._collect_all_metadata(ssc)
      File "/opt/conda/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
        return f(*args, **kwds)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/solve.py", line 425, in _collect_all_metadata
        index, r = self._prepare(prepared_specs)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/solve.py", line 1021, in _prepare
        self.subdirs, prepared_specs, self._repodata_fn)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/index.py", line 277, in get_reduced_index
        repodata_fn=repodata_fn)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 120, in query_all
        result = tuple(concat(executor.map(subdir_query, channel_urls)))
      File "/opt/conda/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator
        yield fs.pop().result()
      File "/opt/conda/lib/python3.7/concurrent/futures/_base.py", line 428, in result
        return self.__get_result()
      File "/opt/conda/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
        raise self._exception
      File "/opt/conda/lib/python3.7/concurrent/futures/thread.py", line 57, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 113, in <lambda>
        package_ref_or_match_spec))
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 125, in query
        self.load()
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 189, in load
        _internal_state = self._load()
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 278, in _load
        mod_etag_headers.get('_mod'))
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 326, in _read_local_repdata
        _internal_state = self._process_raw_repodata_str(raw_repodata_str)
      File "/opt/conda/lib/python3.7/site-packages/conda/core/subdir_data.py", line 364, in _process_raw_repodata_str
        json_obj = json.loads(raw_repodata_str or '{}')
      File "/opt/conda/lib/python3.7/json/__init__.py", line 348, in loads
        return _default_decoder.decode(s)
      File "/opt/conda/lib/python3.7/json/decoder.py", line 337, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/opt/conda/lib/python3.7/json/decoder.py", line 353, in raw_decode
        obj, end = self.scan_once(s, idx)
    json.decoder.JSONDecodeError: Unterminated string starting at: line 830302 column 14 (char 25006056)

`$ /opt/conda/bin/conda-env create -n clam -f clam.yaml`

  environment variables:
            BINARIES_PATH=/opt/deeplearning/binaries
                 CIO_TEST=<not set>
  CONDA_AUTO_UPDATE_CONDA=false
        CONDA_DEFAULT_ENV=base
                CONDA_EXE=/opt/conda/bin/conda
             CONDA_PREFIX=/opt/conda
    CONDA_PROMPT_MODIFIER=(base)
         CONDA_PYTHON_EXE=/opt/conda/bin/python
               CONDA_ROOT=/opt/conda
              CONDA_SHLVL=1
           CURL_CA_BUNDLE=<not set>
              DL_BIN_PATH=/opt/deeplearning/bin
         DL_METADATA_PATH=/opt/deeplearning/metadata
                  DL_PATH=/opt/deeplearning
        ENV_URI_FILE_PATH=/opt/deeplearning/metadata/env_uri
    ENV_VERSION_FILE_PATH=/opt/deeplearning/metadata/env_version
      FRAMEWORK_FILE_PATH=/opt/deeplearning/metadata/framework
        JUPYTER_DEPS_PATH=/opt/deeplearning/jupyter
          LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPT
                          I/lib64
                     PATH=/opt/conda/bin:/usr/local/cuda/bin:/opt/conda/bin:/opt/conda/condabin:
                          /usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
       REQUESTS_CA_BUNDLE=<not set>
RESTRICTION_TYPE_FILE_PATH=/opt/deeplearning/restriction
                 SRC_PATH=/opt/deeplearning/src
            SSL_CERT_FILE=<not set>
          TITLE_FILE_PATH=/opt/deeplearning/metadata/title
           TUTORIALS_PATH=/opt/deeplearning/workspace/tutorials
        VERSION_FILE_PATH=/opt/deeplearning/metadata/version
           WORKSPACE_PATH=/opt/deeplearning/workspace

     active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /home/Surya/.condarc
 populated config files : /opt/conda/.condarc
          conda version : 4.9.2
    conda-build version : not installed
         python version : 3.7.8.final.0
       virtual packages : __glibc=2.28=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /opt/conda  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /home/Surya/.conda/pkgs
       envs directories : /opt/conda/envs
                          /home/Surya/.conda/envs
               platform : linux-64
             user-agent : conda/4.9.2 requests/2.25.1 CPython/3.7.8 Linux/4.19.0-11-cloud-amd64 debian/10 glibc/2.28
                UID:GID : 1006:1007
             netrc file : None
           offline mode : False

An unexpected error has occurred. Conda has prepared the above report.

If submitted, this report will be used by core maintainers to improve
future releases of conda.
Would you like conda to send this report to the core maintainers?

I was unable to reproduce this however, since on a different machine, it worked. So it might be a distribution specific thing. I upgraded my conda using conda update conda but still face this issue on my system. Can you provide a few configuration details such as which conda distribution, or a different way to unpack the yaml file?

fedshyvana commented 3 years ago

Hi, I will perhaps look into a more uniform, user friendly way to install conda packages in the future, but for now, maybe you can refer to the requirement.txt file I provided for another individual? Feel free to let me know if you run into any further issues. You could also choose to install individual packages mannually (as far as I know, the code-base supports the most up-to-date version of pytorch, numpy, opencv, etc.) requirements.txt