Incorrect string value for debug logs

[euantorano]

Whilst debugging #264, I've occasionally come across the following error:

1366 - Incorrect string value: '\x80g',\x0A ...' for column `mybb`.`mybb_debuglogs`.`message` at row 1

This is during the Users stage, the issue appears to be the IP address for a user: 'UpdateIPAddress\' => \'?�g\'. An extract of the query is below:

INSERT INTO mybb_debuglogs (`type`,`message`,`timestamp`) VALUES (8,'File: /merge/resources/modules/users.php Line: 176 Function: Log->datatrace -> \r\nFile: /merge/boards/vanilla/users.php Line: 45 Function: Converter_Module_Users->insert -> \r\nFile: /merge/index.php Line: 948 Function: VANILLA_Converter_Module_Users->import -> \r\n$data: array (\n \'UserID\' => \'8\',\n \'Name\' => \'redacted\',\n \'Password\' => \'redacted\',\n \'HashMethod\' => \'Vanilla\',\n \'Photo\' => \'redacted\',\n \'Title\' => NULL,\n \'Location\' => \'Perth, WA Australia\',\n \'About\' => \'\',\n \'Email\' => \'redacted\',\n \'ShowEmail\' => \'0\',\n \'Gender\' => \'f\',\n \'CountVisits\' => \'1447\',\n \'CountInvitations\' => \'0\',\n \'CountNotifications\' => \'56\',\n \'InviteUserID\' => NULL,\n \'DiscoveryText\' => NULL,\n \'Preferences\' => \'{\"Email.WallComment\":false,\"Email.ActivityComment\":false,\"Popup.WallComment\":false,\"Popup.ActivityComment\":false,\"Email.Applicant\":\"1\",\"Email.DiscussionComment\":false,\"Email.BookmarkComment\":false,\"Email.Mention\":false,\"Popup.DiscussionComment\":false,\"Popup.BookmarkComment\":false,\"Popup.Mention\":false,\"Email.ConversationMessage\":\"1\",\"Popup.ConversationMessage\":false,\"Email.Flag\":false,\"Email.ParticipateComment\":false,\"Popup.ParticipateComment\":false,\"DashboardNav.SectionLandingPages\":[],\"DashboardNav.DashboardLandingPage\":\"\",\"DismissedMessages\":[\"118\",\"63\"],\"Email.Groups\":\"1\",\"Popup.Groups\":\"1\",\"Email.Events\":\"1\",\"Popup.Events\":\"1\"}\',\n \'Permissions\' => \'\',\n \'Attributes\' => \'{\"TransientKey\":\"EWC24K3LBEQ7\",\"CountCommentSpamCheck\":1,\"DateCommentSpamCheck\":\"2017-03-25 00:53:13\",\"CheckedDiscussions\":[],\"CountDiscussionSpamCheck\":1,\"DateDiscussionSpamCheck\":\"2017-02-04 10:00:17\",\"CustomProfileFields\":[],\"CheckedComments\":[],\"CountConversationMessageSpamCheck\":1,\"TimeConversationMessageSpamCheck\":1486252154,\"CountConversationSpamCheck\":1,\"TimeConversationSpamCheck\":1443943087,\"TimeZone\":\"Australia/Perth\"}\',\n \'DateSetInvitations\' => NULL,\n \'DateOfBirth\' => NULL,\n \'DateFirstVisit\' => \'2012-01-02 20:48:37\',\n \'DateLastActive\' => \'2020-11-03 22:21:09\',\n \'LastIPAddress\' => \'r??,\n \'AllIPAddresses\' => NULL,\n \'DateInserted\' => \'2012-01-02 20:48:37\',\n \'InsertIPAddress\' => NULL,\n \'DateUpdated\' => \'2020-11-10 23:44:21\',\n \'UpdateIPAddress\' => \'?�g\',\n \'HourOffset\' => \'8\',\n \'Score\' => NULL,\n \'Admin\' => \'0\',\n \'Confirmed\' => \'1\',\n \'Verified\' => \'0\',\n \'Banned\' => \'0\',\n \'Deleted\' => \'0\',\n \'CountDiscussions\' => \'114\',\n \'CountUnreadDiscussions\' => NULL,\n \'CountComments\' => \'1071\',\n \'CountDrafts\' => \'66\',\n \'CountBookmarks\' => \'4\',\n \'CountUnreadConversations\' => \'19\',\n \'DateAllViewed\' => NULL,\n \'Troll\' => \'0\',\n \'Fingerprint\' => \'4f1237e32f049\',\n \'CountBadges\' => \'0\',\n \'Points\' => \'5427\',\n \'Punished\' => \'0\',\n \'usergroups\' => \'33\',\n)',1611505159)

[euantorano]

Note: This also occurs when merging forums. The issue is related to how binary data is handled, but I'm not 100% sure of the best approach to solve this.

[yuliu]

What kind of db system are you using? MySQL?

Both #235 and #241 are relevant to this issue. https://github.com/mybb/merge-system/issues/235#issuecomment-541418687 provides some ideas that may be helpful but may not be ideal, and not sure if they're applicable to db systems other than MySQL.

[euantorano]

Yeah, I'm on MariaDB 10.5, using UTF8 mb4. I'm using the standard mariadb:10.5 Docker image.

[luisalvarado]

Yes, also having this issue and @euantorano has been super helpful in regards to tackling multiple vanilla issues.

[euantorano]

Note: This should be resolved by#267.