Introduce Python-specific standard library

[krzema12]

https://kotlinlang.slack.com/archives/C0289CS37AS/p1636875290016700

We were advised by JetBrains to look at how kotlin-stdlib-wasm is implemented.

[krzema12]

Experimenting on https://github.com/krzema12/kotlin-python/tree/python-stdlib

Trying with

dist/kotlinc/bin/kotlinc-py -libraries libraries/stdlib/python/build/libs/kotlin-stdlib-python-js-1.6.255-SNAPSHOT.klib -Xir-produce-js -output out_ir.py python/experiments/python.kt

but getting

exception: java.lang.IllegalStateException: Internal function 'jsEqeq' not found
    at org.jetbrains.kotlin.ir.backend.py.JsIrBackendContext.getJsInternalFunction$backend_py(JsIrBackendContext.kt:384)
    at org.jetbrains.kotlin.ir.backend.py.JsIntrinsics.getInternalFunction(JsIntrinsics.kt:367)
    at org.jetbrains.kotlin.ir.backend.py.JsIntrinsics.<init>(JsIntrinsics.kt:31)
    at org.jetbrains.kotlin.ir.backend.py.JsIrBackendContext.<init>(JsIrBackendContext.kt:175)
    at org.jetbrains.kotlin.ir.backend.py.JsIrBackendContext.<init>(JsIrBackendContext.kt:48)
    at org.jetbrains.kotlin.ir.backend.py.CompilerKt.compile(compiler.kt:57)
    at org.jetbrains.kotlin.ir.backend.py.CompilerKt.compile$default(compiler.kt:32)
    at org.jetbrains.kotlin.cli.py.K2JsIrCompiler.doExecute(K2JsIrCompiler.kt:249)
    at org.jetbrains.kotlin.cli.py.K2JSCompiler.doExecute(K2JSCompiler.java:182)
    at org.jetbrains.kotlin.cli.py.K2JSCompiler.doExecute(K2JSCompiler.java:75)
    at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:92)
    at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:44)
    at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:98)
    at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:76)
    at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:45)
    at org.jetbrains.kotlin.cli.common.CLITool$Companion.doMainNoExit(CLITool.kt:227)
    at org.jetbrains.kotlin.cli.common.CLITool$Companion.doMainNoExit$default(CLITool.kt:225)
    at org.jetbrains.kotlin.cli.common.CLITool$Companion.doMain(CLITool.kt:214)
    at org.jetbrains.kotlin.cli.common.CLITool.doMain(CLITool.kt)
    at org.jetbrains.kotlin.cli.py.K2JSCompiler.main(K2JSCompiler.java:103)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.jetbrains.kotlin.preloading.Preloader.run(Preloader.java:87)
    at org.jetbrains.kotlin.preloading.Preloader.main(Preloader.java:44)

It fails here: https://github.com/krzema12/kotlin-python/blob/52ad7cb65633b68387038e39b0ca1bdbe9be83f4/compiler/ir/backend.py/src/org/jetbrains/kotlin/ir/backend/py/JsIntrinsics.kt#L31

because the WASM stdlib (what I copied to create our stdlib from) doesn't have jsEqeq function.

[krzema12]

I'm unblocked and continuing with integrating the new stdlib. Current status can be found on python-stdlib branch.

[krzema12]

Iterating with:

./gradlew dist && ./gradlew :kotlin-stdlib-python:build && dist/kotlinc/bin/kotlinc-py -libraries libraries/stdlib/python/build/libs/kotlin-stdlib-python-js-1.6.255-SNAPSHOT.klib -Xir-produce-js -output out_ir.py python/experiments/python.kt

[krzema12]

I'm pausing it to focus on project cleanup. It now doesn't fail at runtime for some simple example, but I had to remove a lot of Python backend pieces, which results in incorrect output anyway. Will get back to it once cleanup is done.

[SerVB]

Looks like I have some progress here, will open a PR.

[krzema12]

Right now, after explicitly building Python stdlib (./gradlew :kotlin-stdlib-py:build), the KLib is accessible under libraries/stdlib/py/build/libs/kotlin-stdlib-py-js-1.6.255-SNAPSHOT.klib. The goal now is to build it within ./gradlew dist and copy it over to the dist/kotlinc/lib directory.

Another observation and a thing to be changed: the Python stdlib mimics the build logic after JS library, while we should copy WASM's build logic.

[SerVB]

we should copy WASM's build logic

Could you elaborate please? From what I've briefly seen, the WASM build logic is quite specific because it's bound to JS. However, I can easily be wrong because I haven't spent much time on it.

The problem I see is that we compile stdlib-py with the KJS compiler, so we get not only KLIB, but also JS. I guess it's not a big problem for now.

[krzema12]

I refer to https://kotlinlang.slack.com/archives/C0289CS37AS/p1637160755034700?thread_ts=1636875290.016700&cid=C0289CS37AS which is pretty vague, but I understand it as e.g. using maven-publish plugin for publishing instead of some custom Gradle configurations. Pragmatically, we can leave things as they are for now, and follow up once needed.

[krzema12]

Left to do: currently Python stdlib relies on copying some JS stdlib. It shouldn't happen - both should be independent. It's a matter of copying some files over to Python stdlib and adjusting build logic.