database corruption

[alextremblay]

ran into an issue my first time using tinfoil.

steps I took to reproduce:

• install tinfoil
• point it at a folder with a handful of titles
• trigger library scan
• confirm titles show up in the library tab 🎉
• add more games to library
• trigger new scan
• suddenly library tab is empty, no titles at all
• see this in the logs:

  [2024-08-23 14:00:50.677] ERROR (app) Exception on /api/library/scan [POST]
  Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1473, in wsgi_app
  response = self.full_dispatch_request()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 882, in full_dispatch_request
  rv = self.handle_user_exception(e)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 880, in full_dispatch_request
  rv = self.dispatch_request()
       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 865, in dispatch_request
  return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/auth.py", line 39, in decorated_view
  return f(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^
  File "/app/app.py", line 335, in scan_library_api
  scan_library()
  File "/app/app.py", line 292, in scan_library
  titles_library = generate_library()
                   ^^^^^^^^^^^^^^^^^^
  File "/app/library.py", line 104, in generate_library
  if int(title['version']) == int(app_id_version_from_versions_txt):
     ^^^^^^^^^^^^^^^^^^^^^
  TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
  [2024-08-23 14:00:50.681] INFO (_internal) 192.168.2.138 - "POST /api/library/scan HTTP/1.1" 500 -

i re-ran ownfoil through a debugger, and i think i see the issue: during parsing of a file with a malformed filename, ownfoil encountered a file ending in [010097F018539004][USA]v0].nsp there's a typo, a missing [

i suspect the bug lies here: https://github.com/a1ex4/ownfoil/blob/31bf87f08fb3822983a62ee1c9b665998c6f0b66/app/titles.py#L197 perhaps before committing an entry into the DB, it would be best to check that versions is not None and either default to 0 or skip this title?

The problem with the current approach is that even if a user catches the issue and fixes the typo in the filename, the erroneous entry has already been added to the DB, and the user then needs to delete the db, reconfigure all their app settings, and THEN rescan

[qwerty-the-lost]

I wonder if this is the issue I'm having.

There is quite a large library on a NAS and all was well. It was adding titles and I could see them populating into the library.

At one point there was a 500 error, and the container ground to a halt. After a restart; it continued however now no matter what I try the library is blank, even though the system is confirming all the titles have been scanned.

I've tried stopping the service as well as restarting it. It goes through the motions, tells me it's generating the library but nothing.

[alextremblay]

I wonder if this is the issue I'm having.

There is quite a large library on a NAS and all was well. It was adding titles and I could see them populating into the library.

At one point there was a 500 error, and the container ground to a halt. After a restart; it continued however now no matter what I try the library is blank, even though the system is confirming all the titles have been scanned.

I've tried stopping the service as well as restarting it. It goes through the motions, tells me it's generating the library but nothing.

to see if it's the same issue, you'd have to go through the logs and look at all the errors. if you figure out which file caused issues in the first place and then fixed/removed the file, you can delete config/ownfoil.db and restart the container

[a1ex4]

Thanks for the investigations and detailed reports, I'll make this part of the code more robust, and maybe display files with problems on the UI for identification as well.

[a1ex4]

I've pushed a quick fix to the develop branch to at least handle the case where a value is set to None, you should try the develop tab of the image to test it.

[a1ex4]

Fix has been pushed to latest version of the image.