bigpresh / Dancer-Plugin-Database

Dancer::Plugin::Database - easy database support for Dancer applications
http://search.cpan.org/dist/Dancer-Plugin-Database
37 stars 36 forks source link

UTF-8 support in mysql requires mysql_enable_utf8mb4, not mysql_enable_utf8 #101

Open yahermann opened 2 years ago

yahermann commented 2 years ago

According to: https://metacpan.org/pod/DBD::mysql#mysql_enable_utf8mb4 DBD::mysql requires setting mysql_enable_utf8mb4 to support 4-byte UTF-8.

MySQL is a bit quirky in that utf8 is not real UTF-8. To get real UTF-8, you need utf8mb4: https://mathiasbynens.be/notes/mysql-utf8mb4

Shared/lib/Dancer/Plugin/Database/Core.pm, appears to be setting mysql_enable_utf8 which only provides 3-byte UTF-8:

# If the app is configured to use UTF-8, the user will want text from the
# database in UTF-8 to Just Work, so if we know how to make that happen, do
# so, unless they've set the auto_utf8 plugin setting to a false value.
my $app_charset = $settings->{charset} || "";
my $auto_utf8 = exists $settings->{auto_utf8} ?  $settings->{auto_utf8} : 1;

if (lc $app_charset eq 'utf-8' && $auto_utf8) {
    # The option to pass to the DBI->connect call depends on the driver:
    my %param_for_driver = (
        SQLite => 'sqlite_unicode',
        mysql  => 'mysql_enable_utf8',   #################################################
        Pg     => 'pg_enable_utf8',
    );

    my $param = $param_for_driver{$driver};

    if ($param && !$settings->{dbi_params}{$param}) {
        $logger->(
            debug => "Adding $param to DBI connection params"
                . " to enable UTF-8 support"
        );
        $settings->{dbi_params}{$param} = 1;
    }
}

and to add insult to injury, adds a warning if we override to mysql_enable_utf8mb4 without disabling auto_utf8, and on top of that, it doesn't Just Work as advertised, it Just Works but not Always 😅