wrye-bash / wrye-bash

A swiss army knife for modding Bethesda games.
https://wrye-bash.github.io
GNU General Public License v3.0
456 stars 79 forks source link

FileInfo(s) APIs refactoring #336

Open Utumno opened 7 years ago

Utumno commented 7 years ago

It is not a terrible API per se but:

File operations - we need to mark_unchanged if using the python methods but SHFileOperation will preserve c/mtime - all this part of the API needs rewrite, it's system's programming after all, generally a hard subject. See Slight optimizations in rename:


Rename/Hide/Backup and co fall into two categories: copying and moving files. We should rethink our internal APIs for copying/moving and then rewrite those operations as thin wrappers (passing the right paths in) around a copy/move API. Those operations should be as atomic as the filesystem permits. Moreover, when renaming/hiding/backing up/moving in save profiles etc multiple files we should construct all the source/paths at once and issue one system call - see for instance _try_rename that does this per file...


I created an AFile class to represent a file with a cache check for mod times and size but it's difficult to plug it in. Opening this to document/coordinate the refactoring.

Infernio commented 3 months ago

I know this is probably fixed in your 336 work already(?), but I just got this with a corrupted save (or rather, save that WB does not yet know how to read) in Starfield:

Traceback (most recent call last):
  File "/home/infernio/Programming/wrye-bash/Mopy/bash/gui/events.py", line 181, in _post
    result = listener(*listener_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/infernio/Programming/wrye-bash/Mopy/bash/balt.py", line 515, in _conversation_wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/infernio/Programming/wrye-bash/Mopy/bash/basher/__init__.py", line 4006, in RefreshData
    store.unique_store_key: not booting and bool(store.refresh()) # todo add the refr object here
                                                 ^^^^^^^^^^^^^^^
  File "/home/infernio/Programming/wrye-bash/Mopy/bash/bosh/__init__.py", line 3412, in refresh
    return super().refresh(booting=booting) if refresh_infos else \
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/infernio/Programming/wrye-bash/Mopy/bash/bosh/__init__.py", line 1589, in refresh
    new_or_present, del_infos = self._list_store_dir()
                                ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/infernio/Programming/wrye-bash/Mopy/bash/bosh/__init__.py", line 1643, in _list_store_dir
    return self._diff_dir(FNDict(((x, {}) for x in inodes)))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/infernio/Programming/wrye-bash/Mopy/bash/bosh/__init__.py", line 1651, in _diff_dir
    if inf.fn_key not in inodes}
       ^^^^^^^^^^
AttributeError: 'Corrupted' object has no attribute 'fn_key'
Utumno commented 3 months ago

Thanks Inf and thanks for merging that ton of commits and fo4 refactoring and for investigating the heisenbugs - let's see if this one is really fixed - edit: fixed in 5a802fd297f2013ce1d31860fd24016e514fc692 - fastest way of fixing it