search

Project-MONAI / tutorials

MONAI Tutorials
https://monai.io/started.html
Apache License 2.0
1.85k stars 682 forks source link

FileNotFoundError in profiling_camelyon_pipeline.ipynb #1598

Closed KumoLiu closed 10 months ago

KumoLiu commented 10 months ago 
12:41:33  Running ./pathology/tumor_detection/ignite/profiling_camelyon_pipeline.ipynb
12:41:33  Checking PEP8 compliance...
12:41:34  Running notebook...
12:41:40  MONAI version: 1.3.0+51.g02973f0e
12:41:40  Numpy version: 1.22.2
12:41:40  Pytorch version: 2.1.0a0+29c30b1
12:41:40  MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
12:41:40  MONAI rev id: 02973f0e2bac4f799bb9f522de5bb7427f257ed4
12:41:40  MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
12:41:40  
12:41:40  Optional dependencies:
12:41:41  Pytorch Ignite version: 0.4.11
12:41:41  ITK version: 5.3.0
12:41:41  Nibabel version: 5.2.0
12:41:41  scikit-image version: 0.22.0
12:41:41  scipy version: 1.11.1
12:41:41  Pillow version: 9.2.0
12:41:41  Tensorboard version: 2.9.0
12:41:41  gdown version: 4.7.1
12:41:41  TorchVision version: 0.16.0a0
12:41:41  tqdm version: 4.65.0
12:41:41  lmdb version: 1.4.1
12:41:41  psutil version: 5.9.4
12:41:41  pandas version: 1.5.2
12:41:41  einops version: 0.6.1
12:41:41  transformers version: 4.36.2
12:41:41  mlflow version: 2.9.2
12:41:41  pynrrd version: 1.0.0
12:41:41  clearml version: 1.13.3rc0
12:41:41  
12:41:41  For details about installing the optional dependencies, please visit:
12:41:41      https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
12:41:41  
12:41:43  papermill  --progress-bar -k python3
12:41:43  /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
12:41:43    warnings.warn(
12:42:21  
Executing:   0%|          | 0/19 [00:00<?, ?cell/s]
Executing:   5%|▌         | 1/19 [00:01<00:22,  1.25s/cell]
Executing:  32%|███▏      | 6/19 [00:17<00:39,  3.02s/cell]
Executing:  42%|████▏     | 8/19 [00:24<00:34,  3.09s/cell]
Executing:  53%|█████▎    | 10/19 [00:25<00:21,  2.40s/cell]
Executing:  63%|██████▎   | 12/19 [00:34<00:21,  3.03s/cell]
Executing:  68%|██████▊   | 13/19 [00:35<00:15,  2.53s/cell]
Executing:  89%|████████▉ | 17/19 [00:35<00:02,  1.32s/cell]
Executing:  89%|████████▉ | 17/19 [00:37<00:04,  2.20s/cell]
12:42:21  /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
12:42:21    warnings.warn(
12:42:21  Traceback (most recent call last):
12:42:21    File "/usr/local/bin/papermill", line 8, in <module>
12:42:21      sys.exit(papermill())
12:42:21    File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
12:42:21      return self.main(*args, **kwargs)
12:42:21    File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
12:42:21      rv = self.invoke(ctx)
12:42:21    File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
12:42:21      return ctx.invoke(self.callback, **ctx.params)
12:42:21    File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
12:42:21      return __callback(*args, **kwargs)
12:42:21    File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
12:42:21      return f(get_current_context(), *args, **kwargs)
12:42:21    File "/usr/local/lib/python3.10/dist-packages/papermill/cli.py", line 254, in papermill
12:42:21      execute_notebook(
12:42:21    File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 134, in execute_notebook
12:42:21      raise_for_execution_errors(nb, output_path)
12:42:21    File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 241, in raise_for_execution_errors
12:42:21      raise error
12:42:21  papermill.exceptions.PapermillExecutionError: 
12:42:21  ---------------------------------------------------------------------------
12:42:21  Exception encountered at "In [7]":
12:42:21  ---------------------------------------------------------------------------
12:42:21  FileNotFoundError                         Traceback (most recent call last)
12:42:21  Cell In[7], line 2
12:42:21        1 # Load NVTX Push/Pop Range Summary
12:42:21  ----> 2 summary = pd.read_csv("profile_report_nvtxppsum.csv")
12:42:21        3 # display(summary)
12:42:21        4 
12:42:21        5 # Set the Range (which is the name of each range) as the index
12:42:21        6 summary.set_index("Range", inplace=True)
12:42:21  
12:42:21  File /usr/local/lib/python3.10/dist-packages/pandas/util/_decorators.py:211, in deprecate_kwarg.<locals>._deprecate_kwarg.<locals>.wrapper(*args, **kwargs)
12:42:21      209     else:
12:42:21      210         kwargs[new_arg_name] = new_arg_value
12:42:21  --> 211 return func(*args, **kwargs)
12:42:21  
12:42:21  File /usr/local/lib/python3.10/dist-packages/pandas/util/_decorators.py:331, in deprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
12:42:21      325 if len(args) > num_allow_args:
12:42:21      326     warnings.warn(
12:42:21      327         msg.format(arguments=_format_argument_list(allow_args)),
12:42:21      328         FutureWarning,
12:42:21      329         stacklevel=find_stack_level(),
12:42:21      330     )
12:42:21  --> 331 return func(*args, **kwargs)
12:42:21  
12:42:21  File /usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py:950, in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, error_bad_lines, warn_bad_lines, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)
12:42:21      935 kwds_defaults = _refine_defaults_read(
12:42:21      936     dialect,
12:42:21      937     delimiter,
12:42:21     (...)
12:42:21      946     defaults={"delimiter": ","},
12:42:21      947 )
12:42:21      948 kwds.update(kwds_defaults)
12:42:21  --> 950 return _read(filepath_or_buffer, kwds)
12:42:21  
12:42:21  File /usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py:605, in _read(filepath_or_buffer, kwds)
12:42:21      602 _validate_names(kwds.get("names", None))
12:42:21      604 # Create the parser.
12:42:21  --> 605 parser = TextFileReader(filepath_or_buffer, **kwds)
12:42:21      607 if chunksize or iterator:
12:42:21      608     return parser
12:42:21  
12:42:21  File /usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py:1442, in TextFileReader.__init__(self, f, engine, **kwds)
12:42:21     1439     self.options["has_index_names"] = kwds["has_index_names"]
12:42:21     1441 self.handles: IOHandles | None = None
12:42:21  -> 1442 self._engine = self._make_engine(f, self.engine)
12:42:21  
12:42:21  File /usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py:1735, in TextFileReader._make_engine(self, f, engine)
12:42:21     1733     if "b" not in mode:
12:42:21     1734         mode += "b"
12:42:21  -> 1735 self.handles = get_handle(
12:42:21     1736     f,
12:42:21     1737     mode,
12:42:21     1738     encoding=self.options.get("encoding", None),
12:42:21     1739     compression=self.options.get("compression", None),
12:42:21     1740     memory_map=self.options.get("memory_map", False),
12:42:21     1741     is_text=is_text,
12:42:21     1742     errors=self.options.get("encoding_errors", "strict"),
12:42:21     1743     storage_options=self.options.get("storage_options", None),
12:42:21     1744 )
12:42:21     1745 assert self.handles is not None
12:42:21     1746 f = self.handles.handle
12:42:21  
12:42:21  File /usr/local/lib/python3.10/dist-packages/pandas/io/common.py:856, in get_handle(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)
12:42:21      851 elif isinstance(handle, str):
12:42:21      852     # Check whether the filename is to be opened in binary mode.
12:42:21      853     # Binary mode does not support 'encoding' and 'newline'.
12:42:21      854     if ioargs.encoding and "b" not in ioargs.mode:
12:42:21      855         # Encoding
12:42:21  --> 856         handle = open(
12:42:21      857             handle,
12:42:21      858             ioargs.mode,
12:42:21      859             encoding=ioargs.encoding,
12:42:21      860             errors=errors,
12:42:21      861             newline="",
12:42:21      862         )
12:42:21      863     else:
12:42:21      864         # Binary mode
12:42:21      865         handle = open(handle, ioargs.mode)
12:42:21  
12:42:21  FileNotFoundError: [Errno 2] No such file or directory: 'profile_report_nvtxppsum.csv'
KumoLiu commented 10 months ago

The root cause is the permissions for Google Drive used in tutorials have changed.

Access denied with the following error:

    Cannot retrieve the public link of the file. You may need to change
    the permission to 'Anyone with the link', or have had many accesses. 

You may still be able to access the file from the browser:

     https://drive.google.com/uc?id=1uWS4CXKD-NP_6-SgiQbQfhFMzbs0UJIr 

Access denied with the following error:

    Cannot retrieve the public link of the file. You may need to change
    the permission to 'Anyone with the link', or have had many accesses. 

You may still be able to access the file from the browser:

     https://drive.google.com/uc?id=1OxAeCMVqH9FGpIWpAXSEJe6cLinEGQtF

@drbeh, could you please help check the permissions, thanks in advance!

drbeh commented 10 months ago

Hi @KumoLiu, thanks for catching this. It seems that the permission of these files and the rest of pathology data (and I believe radiology data too) has changed to restricted by @aylward. Stephen is there any reason you have done this or has it been done accidentally?

CC: @ericspod @wyli

aylward commented 10 months ago

Sorry, that was an accident. I had to move our shared google drive to a new account.

It should be fixed now, and there should be no need for any future account changes. These files now have a "google-permanent" home. There may be other permission errors, but I tried to be thorough.