Importing of itemInfo.lua with UTF-8 character encoding results in an error.

[douglasparker]

Environment

- PHP: 8.2
- MySQL: mariadb:11.0

Current Behavior

An error occurs when importing itemInfo.lua from the translation project.

Repository: https://github.com/llchrisll/ROenglishRE File: https://github.com/llchrisll/ROenglishRE/blob/master/Pre-Renewal/System/itemInfo_EN.lua

It might have something to do with UTF-8 character set-

Originally my IDE wasn't setup correctly and it worked fine (UTF-8 charset seemed to have been discarded and replaced with random symbols).

Issue goes away when not using UTF-8 character encoding, but obviously I need to use UTF-8.

Once I fixed my IDE and ensured UTF-8 was correctly set up I started running into issues.

The issue appears when using the UTF-8 as seen below:

Most items still import just fine, but it seems that somewhere at the very end causes problems.

Any help on this would be greatly appreciated!

Error:

Critical Error
An error was encountered during the lifetime of the application.

This could be due to a variety of problems, such as a bug in the application.

However, normally it is caused by misconfiguration.

Exception Details
Error: PDOException

Message: SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xBE\xF3\xB1\xBC \xC7...' for column `midgard`.`cp_itemdesc`.`itemdesc` at row 1

File: /sites/example.com/lib/Flux/Connection/Statement.php:50

File    Line    Function/Method
/sites/example.com/lib/Flux/Connection/Statement.php    50  PDOStatement::execute()
/sites/example.com/modules/item/iteminfo.php    95  Flux_Connection_Statement::execute()
/sites/example.com/lib/Flux/Template.php    375 include()
/sites/example.com/lib/Flux/Dispatcher.php  170 Flux_Template::render()
/sites/example.com/index.php    154 Flux_Dispatcher::dispatch()
Exception Trace As String
#0 /sites/example.com/lib/Flux/Connection/Statement.php(50): PDOStatement->execute()
#1 /sites/example.com/modules/item/iteminfo.php(95): Flux_Connection_Statement->execute()
#2 /sites/example.com/lib/Flux/Template.php(375): include('...')
#3 /sites/example.com/lib/Flux/Dispatcher.php(170): Flux_Template->render()
#4 /sites/example.com/index.php(154): Flux_Dispatcher->dispatch()
#5 {main}

Expected Behavior

Import of itemInfo.lua should be free of any errors.

Steps To Reproduce

1. Download the file at https://github.com/llchrisll/ROenglishRE/blob/master/Pre-Renewal/System/itemInfo_EN.lua
2. Ensure UTF-8 characters are present in the file. Viewing raw in the browser doesn't seem to support UTF-8. If you download it in this format it will work fine, but the reality is we need it to work with UTF-8.
3. Import into FluxCP via the import button in the control panel.

Anything else?

No response

[secretdataz]

You have to use EUC-KR instead of UTF-8.

[douglasparker]

You have to use EUC-KR instead of UTF-8.

@secretdataz

I'm currently using EUC-KR with my IDE, but the import error persists. Does my database need to use it too?

Edit: I truncated cp_itemdesc and reimported with the correct character set, and I am still receiving the same error.

Edit: Could this be related to my PHP or MySQL version?

[Daraen1]

Also having problems with this, I installed Fluxcp locally and it works, after that, export your cp_itemdesc table and import it to your online Fluxcp. This is just an alternative solution

[sanasol]

I'm currently using EUC-KR with my IDE

Why should IDE help with php/mysql error :)

Change DB (at least for _cpitemdesc table) encoding and/or connection encoding

It should work with utf8 actually, so check that connection, database and table actually uses utf8.

[douglasparker]

I'm currently using EUC-KR with my IDE

Why should IDE help with php/mysql error :)

I am confirming that the contents of the itemInfo.lua file hasn't changed by using the wrong character encoding... :)

Change DB (at least for _cpitemdesc table) encoding and/or connection encoding

It should work with utf8 actually, so check that connection, database and table actually uses utf8.

The database is already using utf8. Same with the connection.

I'm pretty sure rAthena and FluxCP ships with the database and connection string with UTF8, so I am not sure how that is helpful.

[secretdataz]

Ah, my mistake. I thought the file was worked on with the wrong encoding.

[douglasparker]

Ah, my mistake. I thought the file was worked on with the wrong encoding.

All good. If this doesn't get fixed, I will probably create my own import tool.

I think I have seen previous bugs where the importer broke. It seems kinda flimsy.

[douglasparker]

It seems the error goes away when I downgraded from PHP 8.x to PHP 7.2.

@Akkarinage the README says the project uses PHP 5.2... It seems the documentation for FluxCP in regards to dependencies is severely lacking and major bugs go ignored.

Can you confirm the supported version of PHP and MySQL? At this point I plan to write my own control panel because the FluxCP codebase is a mess and sorting through everything seems like more pain than it's worth.

I also noticed that the mapdb and mobdb were removed without a replacement as well...

Unsure if there are any security vulnerabilities as well. It doesn't help that PHP 5.2 is being pushed still.