search

snipe / snipe-it

A free open source IT asset/license management system
https://snipeitapp.com
GNU Affero General Public License v3.0
10.84k stars 3.14k forks source link

[7.0.11] Wrong character encoding when restoring backup via SnipeIT GUI #15449

Open swift2512 opened 1 week ago

swift2512 commented 1 week ago

Debug mode

Describe the bug

When restoring backup (generated in 'Admin > Backups', lithuanian letters are encoded incorrectly. Instead of ĄČĘĖĮŠŲŪŽ i get something like ĄČĘĖĮŠŲŪŽ image This doesn't happen when restoring same backup file through SSH (php artisan snipeit:restore /var/www/snipeit/storage/app/backups/snipe-it-2024-08-26-15-42-46.zip) Can't figure out what's wrong. Tried Edge and Firefox browsers for backup restore - same result.

Reproduction steps

  1. Use Lithuanian characters (ĄČĘĖĮŠŲŪ) in names or other fields
  2. Generate backup.
  3. Restore backup in Admin > Backups

Expected behavior

Characters encoded correctly.

Screenshots

image image image

Snipe-IT Version

Version v7.0.11 - build 15044 (master)

Operating System

ubuntu

Web Server

apache

PHP Version

8.3

Operating System

Windows 11

Browser

Edge, Firefox

Version

Latest

Device

No response

Operating System

No response

Browser

No response

Version

No response

Error messages

No errors in log files with debug enabled.

Additional context

No

snipe commented 1 week ago

Well what makes less sense there is that the GUI tool just calls the cli tool, so it doesn't really have opinions about encoding.

snipe commented 1 week ago

When you do the gui restore, do you see any options (checkboxes)? We recently added a “clean” option where we do some regexes which could potentially be mangling the Unicode.

swift2512 commented 1 week ago

Yes, there's this checkbox for database cleaning. I checked that.

swift2512 commented 1 week ago

Leaving this unchecked restores backup with correct characters. If checked- garbage characters everywhere. image

BartFridayski commented 1 week ago

I have exactly the same problem. In my case it is a problem with Polish letters but now when I add new records containing Polish characters it is OK. The problem is only with the imported data.

I assume there is a problem with the export, then the encoding is lost.

swift2512 commented 1 week ago

I have exactly the same problem. In my case it is a problem with Polish letters but now when I add new records containing Polish characters it is OK. The problem is only with the imported data.

I assume there is a problem with the export, then the encoding is lost.

Exported backup is ok, it has correct encoding information and characters. Backup restore only fails when "Clean backed-up database before restore' is checked. Then this is unchecked or restore through ssh is done, are letters are correct.

While dbdumb file is using utf8 format, looks like data parser uses ANSI when reading this file and then forwards these wrongly decoded characters to the database.

snipe commented 1 week ago

You generally don't need to clean the database unless you've uploaded it from somewhere else. What we were finding is that restores would break, if for example you were running as a mysql user with special grants on one server, but the new server doesn't have those grants. The process would basically be that the tables get dropped and then the import fails because the mysql user on the current system didn't have enough rights granted (or sometimes there were settings included in the export from one machine that the new machine can't handle.)

Thanks for helping us pin this down, @swift2512 - we'll get it sorted shortly.