Getting protocol folder size can be problematic

delarosatrevin commented 7 months ago

In running protocols where some tmp folders/files are deleted, it can raise the following exception:

Exception in Tkinter callback
Traceback (most recent call last):
  File "/software/scipion/conda/lib/python3.8/tkinter/__init__.py", line 1892, in __call__
    return self.func(*args)
  File "/software/scipion/conda/lib/python3.8/tkinter/__init__.py", line 814, in callit
    func(*args)
  File "/software/scipion/source/core/scipion-pyworkflow/pyworkflow/gui/canvas.py", line 290, in <lambda>
    lambda: self._showTooltip(event.x_root,
  File "/software/scipion/source/core/scipion-pyworkflow/pyworkflow/gui/canvas.py", line 122, in _showTooltip
    self._tooltipCallback(tw, item)
  File "/software/scipion/source/core/scipion-pyworkflow/pyworkflow/gui/project/viewprotocols.py", line 1340, in _runItemTooltip
    tm += 'Folder size: %s\n' % pwutils.prettySize(prot.getSize())
  File "/software/scipion/source/core/scipion-pyworkflow/pyworkflow/protocol/protocol.py", line 2330, in getSize
    self._size = getFileSize(self.getPath())
  File "/software/scipion/source/core/scipion-pyworkflow/pyworkflow/utils/path.py", line 460, in getFileSize
    total_size += os.path.getsize(fp)
  File "/software/scipion/conda/lib/python3.8/genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: 'Runs/008023_ProtRelionExtractParticles/tmp/mic_009214.mrcs'

Moreover, for some large datasets (e.g. tens of thousands of movies) this can take a long time to compute and freeze the whole GUI.

I propose to either avoid that computation or have some sort of cache value that prevents from running that many IO queries every time one hover over a protocol box.

pconesa commented 7 months ago

I think it is cached . But I. This case it can't even cache it. We shoul tolerate errors there.

El jue., 7 dic. 2023 21:13, Jose Miguel de la Rosa Trevin < @.***> escribió:

Assigned #486 https://github.com/scipion-em/scipion-pyworkflow/issues/486 to @pconesa https://github.com/pconesa.

— Reply to this email directly, view it on GitHub https://github.com/scipion-em/scipion-pyworkflow/issues/486#event-11186420436, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF7ZYJABZTSQ5PJ2H6O4T3YIIPOBAVCNFSM6AAAAABALUETGSVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJRGE4DMNBSGA2DGNQ . You are receiving this because you were assigned.Message ID: @.*** com>

delarosatrevin commented 7 months ago

Apart from being cached, might be a good idea to do the updated in a background thread. This will prevent blocking the GUI, even in the case where the size needs to be re-calculated.

scipion-em / scipion-pyworkflow

Getting protocol folder size can be problematic #486