Request failed with status code 500: Malformed UTF-8 characters, possibly incorrectly encoded

[beard7]

Hi. I've just replaced log-viewer 1.7 with 3.1 and re-published the vendor assets, but now I can't view the Laravel log files which were working fine previously. I get an error message just under the search box saying: Request failed with status code 500: Malformed UTF-8 characters, possibly incorrectly encoded

Any suggestions?

[arukompas]

@beard7 Most likely something went wrong with the extraction of JSON context. Could you please post the log file that cannot be opened, and I'll try and debug it locally.

[beard7]

Hi,

It's the same result on every log I try to open, including one with a single entry:

[2023-08-31 11:28:13] local.WARNING: str_split(): Passing null to parameter #1 ($string) of type string is deprecated in C:\Code\insight\storage\framework\views\15dc66de37f3339832bf7117ec8a0120.php on line 80  

Not sure is this will help with debugging.

[arukompas]

That is weird @beard7 , can't see anything that would break Log Viewer... I assume this is a Windows log file, which might have a different line ending encoding. Could you upload the file as-is, as an attachment?

[beard7]

Sure. Here's an example. laravel-2023-09-21.log

[beard7]

Hi,

After a bit more digging, this error is only happening when using the 'database' cache driver. My databaseis MS SQL and this error occures when writing to the cache:

SQLSTATE[IMSSP]: An error occurred translating string for input param 1 to UCS-2: No mapping for the Unicode character exists in the target multi-byte code page.
 (Connection: sqlsrv, SQL: update [cache] set [value] = s:69:"x
K?2??δ24?4?40422?N?2??.?2?R
w???sW?eZX?pm-\???1?@?F??@
?wF?";, [expiration] = 1696848458 where [key] = laravellv:v3.1.4:file:0ae1da0b-laravel-2023-09-28.log:ecf8427e:chunk:0) {"userId":1,"exception":"[object] (Illuminate\\Database\\QueryException(code: IMSSP): SQLSTATE[IMSSP]: An error occurred translating string for input param 1 to UCS-2: No mapping for the Unicode character exists in the target multi-byte code page.

 (Connection: sqlsrv, SQL: update [cache] set [value] = s:69:\"x\u0001K�2��δ24�4�40422�N�2��.�2�R
w\f���sW�\beZ\u0019X�pm-\\���1\u001e�@�F��@
�\u0000wF\u001b�\";, [expiration] = 1696848458 where [key] = laravellv:v3.1.4:file:0ae1da0b-laravel-2023-09-28.log:ecf8427e:chunk:0) at C:\\[path]\\vendor\\laravel\\framework\\src\\Illuminate\\Database\\Connection.php:795)

After switching to the 'file' cache driver, all is good.

[arukompas]

Hey @beard7 , great find!

Yes, the cached value is first compressed with gzip, hence the use of multi-byte characters. I guess with SQL databases it does make a difference, in case the selected database charset does not support some of the symbols used by gzip. There was also another problem with SQL as cache driver, from another user.

I'll have a look at SQL as the cache driver for Log Viewer this weekend.

[ale1981]

Sorry to bump an existing post but this is also an issue when logging mail to the log file, especially if the mail has attachments.

[1YalcinSahin1]

I have the same problem often

[arukompas]

hey @1YalcinSahin1 , @beard7 , your issue has been fixed in latest update, v3.1.7. You can now use database driver for cache and the contents should be UTF8-safe.

Still working on mail logs with attachments

[arukompas]

hey @ale1981 , mail previews have been fixed in v3.1.8 :) hope this helps!

that's all from this bug report, I think. Let me know if anyone's still experiencing "Malformed UTF8 characters" error after v3.1.8

[1YalcinSahin1]

Thanks @arukompas You solved our problem quickly :)

[ale1981]

hey @ale1981 , mail previews have been fixed in v3.1.8 :) hope this helps!

that's all from this bug report, I think. Let me know if anyone's still experiencing "Malformed UTF8 characters" error after v3.1.8

Thanks, working now!

[1YalcinSahin1]

hey @ale1981. I experienced the same problem again. I am sending the images.

LOG_VIEWER_CACHE_DRIVER=database

Database character format: utf8mb4_turkish_ci

[arukompas]

hey @1YalcinSahin1 , could you share the error from your second screenshot? Preferably the entry from your laravel log, not browser console.

[1YalcinSahin1]

@arukompas I expect to repeat the mistake. I am reporting another situation that occurred at that time.I will write the error as soon as I get it.

https://github.com/opcodesio/log-viewer/assets/53931084/ea589751-383a-4585-a898-264f45a2ff4f

[arukompas]

@1YalcinSahin1 I just tried the utf8mb4_turkish_ci collation and it still works as expected.

What does your cache table structure look like?

[1YalcinSahin1]

Our database table structure is the same.

500 error came. I am attaching the file.

laravel.log

[1YalcinSahin1]

@arukompas I found the string expression causing the error.

[2023-12-18 16:15:19] production.ERROR: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'barcode' at row 1 (Connection: mysql, SQL: updatecustomer_1setproductID= 5881,productPrice= 9.975,productName= TL 084CN,productSalePrice= 9.975,productStock= 0,productStatus= 0,productInfoMD5= 25b39f0b71c310706d10ba4aaf486bb5,productStockPriceMD5= deb54e44f5ddaf4879440b8b24ce98a4,productImageMD5= d2253e3f51a945bdbfe113888f6456cf,imageErrorStatus= 0,infoUpdateDate= 2023-12-18 16:15:18,stockPriceUpdateDate= 2023-12-18 16:15:18,statusUpdateDate= 2023-12-18 16:15:18,imageUpdateDate= 2023-12-18 16:15:18,barcode= 33583 126 2100000114665 1424340 TL 084CN 126 Transistör Diğ,brand= Customer1,categories= "Elektronik Malzeme > Elektronik Devre Elemanlar\u0131 > Entegre, Soketleri > ",productText= <p><strong>Ürün Adı: </strong>TL 084CN</p><p></p><p><strong>Ürün Bilgisi</strong></p><p>Ürün AdıTL084CN</p><p>Ürün KategorisiAmplifikatör ve Komparatörler</p><p>Ürün Alt KategorisiOp-Amp</p><p>Paket TipiDIP-14</p><p>Pin Sayısı14</p><p>Giriş Gerilimi7 - 36</p><p>Bant Genişliği3MHz</p><p>Slew Rate (Tepki Hızı)13V/µs</p><p>Amplifikatör Sayısı4</p><p> </p>,productImages= ["https:\/\/domain.com\/Images\/org\/69057-tl-084cn-tl-084cn-14421.jpg"] whereid= 10321) {"exception":"[object] (Illuminate\\Database\\QueryException(code: 22001): SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'barcode' at row 1 (Connection: mysql, SQL: updatecustomer_1setproductID= 5881,productPrice= 9.975,productName= TL 084CN,productSalePrice= 9.975,productStock= 0,productStatus= 0,productInfoMD5= 25b39f0b71c310706d10ba4aaf486bb5,productStockPriceMD5= deb54e44f5ddaf4879440b8b24ce98a4,productImageMD5= d2253e3f51a945bdbfe113888f6456cf,imageErrorStatus= 0,infoUpdateDate= 2023-12-18 16:15:18,stockPriceUpdateDate= 2023-12-18 16:15:18,statusUpdateDate= 2023-12-18 16:15:18,imageUpdateDate= 2023-12-18 16:15:18,barcode= 33583\t126\t2100000114665\t1424340\tTL 084CN\t\t\t\t126\t\tTransistör\t\tDiğ,brand= Customer1,categories= \"Elektronik Malzeme > Elektronik Devre Elemanlar\\u0131 > Entegre, Soketleri > \",productText= <p><strong>Ürün Adı: </strong>TL 084CN</p><p></p><p><strong>Ürün Bilgisi</strong></p><p>Ürün AdıTL084CN</p><p>Ürün KategorisiAmplifikatör ve Komparatörler</p><p>Ürün Alt KategorisiOp-Amp</p><p>Paket TipiDIP-14</p><p>Pin Sayısı14</p><p>Giriş Gerilimi7 - 36</p><p>Bant Genişliği3MHz</p><p>Slew Rate (Tepki Hızı)13V/µs</p><p>Amplifikatör Sayısı4</p><p> </p>,productImages= [\"https:\\/\\/domain\\/UserFiles\\/Images\\/org\\/69057-tl-084cn-tl-084cn-14421.jpg\"] whereid= 10321) at /var/www/vhosts/domain/httpdocs/Project/vendor/laravel/framework/src/Illuminate/Database/Connection.php:801)

productImages = [\"https:\/\/domain\/UserFiles\/Images\/org\/69057-tl-084cn-tl-084cn-14421.jpg\"]

When I delete this line, the error goes away.

[arukompas]

hey @1YalcinSahin1 , another user recently pointed out a bug with multi-byte characters in logs not being processed correctly. A fix was made recently in v3.1.12

Could you try updating the Log Viewer to see if it worked for you?

Thanks 🙏