search

OCR-D / core

Collection of OCR-related python tools and wrappers from @OCR-D
https://ocr-d.de/core/
Apache License 2.0
118 stars 31 forks source link

FileExistsError in add_file in ocrd/workspace.py #1203

Closed MehmedGIT closed 1 month ago

MehmedGIT commented 6 months ago

A scarce error that could happen is when two different instances of the same OCR-D processor try to create the same folder:

  Traceback (most recent call last):
    File "/usr/local/sub-venv/headless-tf1/bin/ocrd-segment-repair", line 33, in <module>
      sys.exit(load_entry_point('ocrd-segment', 'console_scripts', 'ocrd-segment-repair')())
    File "/usr/local/sub-venv/headless-tf1/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
      return self.main(*args, **kwargs)
    File "/usr/local/sub-venv/headless-tf1/lib/python3.8/site-packages/click/core.py", line 1078, in main
      rv = self.invoke(ctx)
    File "/usr/local/sub-venv/headless-tf1/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/usr/local/sub-venv/headless-tf1/lib/python3.8/site-packages/click/core.py", line 783, in invoke
      return __callback(*args, **kwargs)
    File "/build/ocrd_segment/ocrd_segment/cli.py", line 21, in ocrd_segment_repair
      return ocrd_cli_wrap_processor(RepairSegmentation, *args, **kwargs)
    File "/build/core/src/ocrd/decorators/__init__.py", line 133, in ocrd_cli_wrap_processor
      run_processor(processorClass, mets_url=mets, workspace=workspace, **kwargs)
    File "/build/core/src/ocrd/processor/helpers.py", line 133, in run_processor
      raise err
    File "/build/core/src/ocrd/processor/helpers.py", line 130, in run_processor
      processor.process()
    File "/build/ocrd_segment/ocrd_segment/repair.py", line 192, in process
      self.workspace.add_file(
    File "/build/core/src/ocrd_utils/deprecate.py", line 15, in wrapper
      return f(*args, **kwargs)
    File "/build/core/src/ocrd_utils/deprecate.py", line 15, in wrapper
      return f(*args, **kwargs)
    File "/build/core/src/ocrd/workspace.py", line 422, in add_file
      makedirs(local_filename_dir)
    File "/usr/lib/python3.8/os.py", line 223, in makedirs
      mkdir(name, mode)
  FileExistsError: [Errno 17] File exists: 'OCR-D-SEGMENT-REPAIR'

An easy solution would be to ignore exists errors. Not sure though if this is good without the force flag?

Check the full logs of the slurm job here: slurm-job-19700959.txt

bertsky commented 6 months ago

An easy solution would be to ignore exists errors. Not sure though if this is good without the force flag?

I agree – mkdir should be mkdir -p behaviour, regardless of --force.