HaxeFoundation / record-macros

Macro-based ORM (object-relational mapping)
MIT License
49 stars 24 forks source link

Unicode. Would we need a `@:unicode` meta? #53

Closed grepsuzette closed 1 year ago

grepsuzette commented 4 years ago

Had prematurely proposed a simple PR https://github.com/HaxeFoundation/record-macros/pull/52 to add a test about unicode, since I was experiencing problems and no documentation about it.

Current target is php+mysql. Turns out I find why characters were shown as "????", it's because I needed to set the Table and the Column to utf8mb4.

Since I don't think this commands a pull request after all, I will share it here, and maybe it can open discussion about whether there is any better solution (something not requesting an ALTER request):

   // this was supposed to go in test/CommonDatabaseTest.hx
    public function testUnicode() {
        // without these two lines the test passes,
        // but content is garbage like "?????????" 
        Manager.cnx.request("ALTER TABLE NullableSpodClass CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci");
        Manager.cnx.request("ALTER TABLE NullableSpodClass MODIFY string VARCHAR(250) CHARACTER SET utf8mb4");

        for (sUnicode in [ 
            "ﺎﻠﻋﺮﺒﻳﺓ", // Arabic
            "汉语",    // Chinese
        ]) { 
            var n1 = getDefaultNull();
            n1.string = sUnicode;
            n1.insert();

            var n2 = NullableSpodClass.manager.select($theId == n1.theId);
            Assert.isTrue( n2 != null );
            Assert.equals( n2.string, sUnicode );
        } 
    } 

I am imagining 'maybe' a @:unicode markup could be used so that the ALTER lines are unneeded. But this is probably a long topic. Meanwhile at least this is searchable if anybody has an issue.

jonasmalacofilho commented 4 years ago

It's been a while, but I think you can (and will probably prefer to) set the default encoding when the MySQL database is created. That said, someone needs to check that all targets are consistently sending UTF-8.