BUG: error constant too long

[mingodad]

Testing Squirrel with the sample script bellow give this error message error constant too long when it shouldn't.

print('\t')
//print('\031')
print('\u002a')
print('\u0236')

Ouput:

./sq test2.nut 
test2.nut line = (4) column = (15) : error constant too long
Error [constant too long]

[mingodad]

Possible fix that I'm applying in https://github.com/mingodad/squilu :

@@ -349,10 +349,11 @@ SQInteger SQLexer::ProcessStringHexEscape(SQChar *dest, SQInteger maxdigits)
     return n;
 }

 SQInteger SQLexer::ReadString(SQInteger ndelim,bool verbatim)
 {
+    SQInteger utf_len = 0; SQUnsignedInteger utf_value = 0;
     INIT_TEMP_STRING();
     NEXT();
     if(IS_EOB()) return -1;
     for(;;) {
         while(CUR_CHAR != ndelim) {
@@ -387,16 +388,18 @@ SQInteger SQLexer::ReadString(SQInteger ndelim,bool verbatim)
                         SQChar temp[8 + 1];
                         ProcessStringHexEscape(temp, maxdigits);
                         SQChar *stemp;
 #ifdef SQUNICODE
 #if WCHAR_SIZE == 2
-                        AddUTF16(scstrtoul(temp, &stemp, 16));
+                        utf_value = scstrtoul(temp, &stemp, 16);
+                        utf_len += AddUTF16(utf_value);
 #else
                         APPEND_CHAR((SQChar)scstrtoul(temp, &stemp, 16));
 #endif
 #else
-                        AddUTF8(scstrtoul(temp, &stemp, 16));
+                        utf_value = scstrtoul(temp, &stemp, 16);
+                        utf_len += AddUTF8(utf_value);
 #endif
                     }
                     break;
                     case _SC('t'): APPEND_CHAR(_SC('\t')); NEXT(); break;
                     case _SC('a'): APPEND_CHAR(_SC('\a')); NEXT(); break;
@@ -431,12 +434,12 @@ SQInteger SQLexer::ReadString(SQInteger ndelim,bool verbatim)
     }
     TERMINATE_BUFFER();
     SQInteger len = _longstr.size()-1;
     if(ndelim == _SC('\'')) {
         if(len == 0) Error(_SC("empty constant"));
-        if(len > 1) Error(_SC("constant too long"));
-        _nvalue = _longstr[0];
+        if(len > 1 && len != utf_len) Error(_SC("constant too long"));
+        _nvalue = utf_len ? utf_value : _longstr[0];
         return TK_INTEGER;
     }
     _svalue = &_longstr[0];
     return TK_STRING_LITERAL;
 }

[mingodad]

Commit on SquiLu is here https://github.com/mingodad/squilu/commit/b34e9469fa5e8c5977cb97b12301c2340afbabd1 any feedback is welcome.

[mingodad]

After pushing the above commit I realize it would not work in this case:

print('\u1234\u1234')

So here is a fix for this case https://github.com/mingodad/squilu/commit/10cc021e470564e540f0aa9fb90fe1c6949d0a31

[mingodad]

Still there is more to it not solved, like:

print('á'):