[Termux] TypeError: __init__() expects list, not 'list'

[anzhi0708]

~/downloads/kuroko $ ./kuroko
Kuroko 1.3.1 (Oct 18 2022 at 19:47:41) with clang 15.0.2
Type `help` for guidance, `paste()` to toggle automatic indentation, `license` for copyright information.
>>> list()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() expects list, not 'list'
>>> [1,2,3]
 => [1, 2, 3]
>>> range(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() expects range, not 'range'
>>> list(range(5))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() expects range, not 'range'
>>>

~/.../kuroko/test $ ../kuroko testThreading.krk
Starting 10 threads.
Traceback (most recent call last):
  File "testThreading.krk", line 33, in <module>
    let numbers    = list(range(totalCount))
TypeError: __init__() expects range, not 'range'

[klange]

Can you tell me a bit about your environment - architecture, OS, etc.? Were there any unexpected warnings when building?

[anzhi0708]

I was using Termux (Android) aarch64; same 'range' error was printed after make, but executable file was successfully built at that point - I thought it was a cool way for the author to stop the make process ;-P

Then I saw your reply and grabbed my Macbook, compiled it again, this time everything works just fine. ¯_(ツ)_/¯

So I guess I'm changing the title...

[klange]

I do not support building for Android under the default configuration, primarily because I don't have any modern Android devices available for testing and also because Android is very different from a typical Linux environment. Android is known to have issues related to thread-local storage, in particular, which is a critical requirement for Kuroko's default Linux build setup. I suggest you try disabling thread support by passing KRK_DISABLE_THREADS=1 in your invocation to make - be sure to make clean first.

Could you also tell me what version of Android you are running? Some support for TLS has been available since Android 29, but I believe it still does not support the initial-exec model I attempt to use.

[anzhi0708]

I'm using Android 11 (OPPO Reno 2, ColorOS V11.1

~ $ neofetch
         -o          o-            u0_a478@localhost
          +hydNNNNdyh+             -----------------
        +mMMMMMMMMMMMMm+           OS: Android 11 aarch64
      `dMMm:NMMMMMMN:mMMd`         Host: OPPO PCKM00
      hMMMMMMMMMMMMMMMMMMh         Kernel: 4.14.180-perf+
  ..  yyyyyyyyyyyyyyyyyyyy  ..     Uptime: 11 days, 11 hours, 41
.mMMm`MMMMMMMMMMMMMMMMMMMM`mMMm.   Packages: 163 (dpkg), 1 (pkg)
:MMMM-MMMMMMMMMMMMMMMMMMMM-MMMM:   Shell: bash 5.2.2
:MMMM-MMMMMMMMMMMMMMMMMMMM-MMMM:   CPU: Qualcomm SDM730G AIE (8)
:MMMM-MMMMMMMMMMMMMMMMMMMM-MMMM:   Memory: 4499MiB / 7551MiB
:MMMM-MMMMMMMMMMMMMMMMMMMM-MMMM:
-MMMM-MMMMMMMMMMMMMMMMMMMM-MMMM-
 +yy+ MMMMMMMMMMMMMMMMMMMM +yy+
      mMMMMMMMMMMMMMMMMMMm
      `/++MMMMh++hMMMM++/`
          MMMMo  oMMMM
          MMMMo  oMMMM
          oNMm-  -mMNs

[klange]

I believe that's API 30, so it should have at least basic support for thread-local storage...

You can either try to build without thread support, which is your best bet, or you can also try removing line 37 of src/vm.c (__attribute__((tls_model("initial-exec")))), and see if either of those help.

[anzhi0708]

Removing line 37 of src/vm.c (__attribute__((tls_model("initial-exec")))) got the same error, it did not help :-( Anyway I'm already happy to use it on macOS, thanks again for your amazing work Closing issue.

[klange]

I'm still curious if disabling threads in general resolved the issue? Otherwise, I might set up an Android VM and explore this further, as it is a very bizarre issue for equality checks on classes to fail, but for the mechanism of printing a type name in the error message to function.

[klange]

I was not able to set up an aarch64 Android emulator through Android Studio on an x86-64 host - I will try again later on an M1 Mac. I did try on an x86-64 Android VM using Termux's latest package for clang, and did not encounter this issue even with the existing TLS model. I suspect the issue is specific to aarch64 Android. For reference, I have not encountered this issue on aarch64 Linux (with glibc), nor on other aarch64 ELF platforms (namely, my own), or on macOS.

[anzhi0708]

I just woke up, sorry for the late reply, I tried to export KRK_DISABLE_THREADS=1 and then make clean then make and still got the same 'expects range' error. (File "tools/codectools/gen_sbencs.krk", line 152, in <module>)

[klange]

Looks like MTE is to blame here - you can see the 0xb4 in the top byte of the address in the last debug line here, which is the arbitrarily-chosen tag byte Android's libc allocator uses. This demonstrates a broken aspect of MTE - the untagged pointer points to the same thing, but doesn't evaluate as equal.

We can disable heap pointer tagging early with mallopt(M_BIONIC_SET_HEAP_TAGGING_LEVEL, M_HEAP_TAGGING_LEVEL_NONE) and now everything works:

[klange]

Another option for us is to recognize that all Kuroko objects are going to be heap-allocated and to bake this into how heap object pointers are extracted from boxed values:

--- a/src/kuroko/value.h
+++ b/src/kuroko/value.h
@@ -209,1 +209,1 @@
- #define AS_OBJECT(value)    ((KrkObj*)(uintptr_t)((value) & KRK_VAL_MASK_LOW))
+ #define AS_OBJECT(value)    ((KrkObj*)(uintptr_t)((value) & KRK_VAL_MASK_LOW | 0xB400000000000000))

This seems to work, though it relies on knowing what the libc will tag heap pointers with - something we could figure out at runtime, but at significant cost compared to hardcoding it here.

[anzhi0708]

Yes - it's working now. VERY COOL!

[klange]

I have pushed the latter fix behind a build flag. Compile with make KRK_HEAP_TAG_BYTE=0xB4. I tried to get this behind a suitable set of #ifdefs but __ANDROID_API__ is unhelpfully set to a lower number than what would indicate MTE is going to be enabled. I may extend the Makefile to detect this more dynamically, but for now a bit of manual massaging should work.

[anzhi0708]

Very cool - this language deserves more attention, so does your OS ;-) thanks for your work