search

manyfold3d / manyfold

A self-hosted digital asset manager for 3d print files.
https://manyfold.app
MIT License
814 stars 49 forks source link

Errors caused by MariaDB JSON serialization #3188

Closed AevumDecessus closed 3 days ago

AevumDecessus commented 3 days ago

Describe the bug image

Server Environment

To Reproduce Steps to reproduce the behavior:

  1. Go to any of the following:\
    • /models
    • /collections
    • /admin/files
  2. Immediately receive error message

Expected behavior Expected to view standard pages as normal

Error Logs

Floppy commented 3 days ago

Interesting, that's a strange one. It's actually due to the file attachment data in the model, switch_locale is just the outermost method for the request.

Is this for a few specific files, or for everything?

Could you go to the advanced admin area and look at the file list? The path is /admin/model_files. If you scroll that list to the right, you should see a column called "attachment_data". Could you tell me what it shows? Mine look like this:

Screenshot 2024-11-18 at 16 29 49

If yours look different, that will be a clue. I wonder, as you're using MariaDB, whether this is related to JSON fields on MySQL/MariaDB being a bit... different, somehow.

AevumDecessus commented 3 days ago

unfortunately /admin/model_files is one of the pages showing this issue

Floppy commented 3 days ago

Ah. Good point.

Floppy commented 3 days ago

If you can docker exec -it manyfold bundle exec rails console -s, and then puts ModelFile.first.attachment_data.inspect that might give us the same view (albeit with a bit more faff). Or use ModelFile.find_param("<id_from_url>").attachment_data.inspect if it's with a specific file. `

AevumDecessus commented 3 days ago

from the rails console

irb(main):001> puts ModelFile.first.attachment_data.inspect
"{\"id\":\"multibin/multipoint/0-5h/mp9/bin-to-bin-connector/0-5h-bin-to-bin-rail-click-connector#1081/0.5H - Bin-To-Bin Rail Click Connector.stl\",\"storage\":\"library_1\",\"metadata\":{\"filename\":\"0.5H - Bin-To-Bin Rail Click Connector.stl\",\"size\":88184,\"mime_type\":\"application/octet-stream\"}}"
=> nil

from the DB select attachment_data from model_files limit 5;

| "{\"id\":\"ender5plus/enclosure/3dup-ender-5-plus-enclosure#2/3dup-corner-L.stl\",\"storage\":\"library_1\",\"metadata\":{\"filename\":\"3dup-corner-L.stl\",\"size\":716184,\"mime_type\":\"application/octet-stream\"}}"                        |
| "{\"id\":\"assortment-collection/assortment-boxes-40/1x1-square-40#3/assortment box 40 1x1.STL\",\"storage\":\"library_1\",\"metadata\":{\"filename\":\"assortment box 40 1x1.STL\",\"size\":742084,\"mime_type\":\"application/octet-stream\"}}" |
| "{\"id\":\"ender5plus/enclosure/3dup-ender-5-plus-enclosure#2/3dup-corner-R.stl\",\"storage\":\"library_1\",\"metadata\":{\"filename\":\"3dup-corner-R.stl\",\"size\":751484,\"mime_type\":\"application/octet-stream\"}}"                        |
| "{\"id\":\"ender5plus/enclosure/3dup-ender-5-plus-enclosure#2/3dup-ender5plus-tray-lid.stl\",\"storage\":\"library_1\",\"metadata\":{\"filename\":\"3dup-ender5plus-tray-lid.stl\",\"size\":20084,\"mime_type\":\"application/octet-stream\"}}"   |
| "{\"id\":\"ender5plus/enclosure/3dup-ender-5-plus-enclosure#2/3dup-ender5plus-tray.stl\",\"storage\":\"library_1\",\"metadata\":{\"filename\":\"3dup-ender5plus-tray.stl\",\"size\":50784,\"mime_type\":\"application/octet-stream\"}}"           |
Floppy commented 3 days ago

OK, great, it looks like for some reason it's not parsing the attachment data JSON field into a sensible format before trying to use it. This might be to do with the fact that we have to prompt MariaDB to do that explicitly; perhaps that's not working. What version of MariaDB are you using?

AevumDecessus commented 3 days ago 
CONTAINER           REPOSITORY                    TAG                 IMAGE ID            SIZE
mariadb             lscr.io/linuxserver/mariadb   latest              97a0f37586f2        335MB
Floppy commented 3 days ago

OK, that's version 10.11.10 of the server, by the look of it. Thanks. I'll take a look at what might be going on and get back to you!

AevumDecessus commented 3 days ago

yeah confirmed version (had to go look up the command again)

MariaDB [(none)]> select VERSION();
+----------------------+
| VERSION()            |
+----------------------+
| 10.11.10-MariaDB-log |
+----------------------+
1 row in set (0.000 sec)
Floppy commented 3 days ago

One more thing - has Manyfold worked at all for you, or has this error hit you straight away?

AevumDecessus commented 3 days ago

Manyfold has been working for ages, this only happened when I upgrade to 0.88 from iirc 0.83

Floppy commented 3 days ago

OK, thanks - that's odd. I've got another report of the same issue, so there's definitely something gone wrong somewhere.

Floppy commented 3 days ago

@AevumDecessus any chance you could try to run some different versions instead of latest and see when the error appears? Since 0.83 (which was only a month ago) there haven't been any major changes to the database that will cause incompatibilities, so you should be safe enough to try the ones in between.

AevumDecessus commented 3 days ago

Looks like the problem was included in the 0.87.0 release

Floppy commented 3 days ago

fantastic, thanks, that'll really narrow it down!

Floppy commented 3 days ago

Good news, I've got it replicated locally...

Floppy commented 3 days ago

OK, it broke in 78c59b83d7a3bd61b11388fc89903f281340d88c

Floppy commented 3 days ago

Right, I think this should be fixed! If you want, give the edge image a try, or wait for the next release, probably tomorrow.

AevumDecessus commented 2 days ago

Can confirm, everything is good in the 0.89.0 release, thanks for the quick deep dive and resolution!