search

gogins / csound-webserver-opcodes

The "webserver" opcodes enable Csound to act as a local Web server that can host HTML5 GUIs and score generators.
GNU Lesser General Public License v2.1
0 stars 0 forks source link

Csound proxy in renderer process #1

Closed gogins closed 2 years ago

gogins commented 2 years ago

Sometimes I feel like I am getting stupider.

I have known for ages and ages that contemporary Web browsers actually run several processes, this is to keep code or anything else fetched from a server in a security sandbox. But I spent several days trying to do this as though everything were in one process.

The same is now true of WebKit and that is why the JavaScript engine and the renderer are so completely separated. Any native addons have to go into an extension that is loaded in the render process' JavaScript context. That, obviously, will not work with the WebKit opcodes.

In persisting in my illusion, I was encouraged by the behavior of the Chromium Embedded Framework in which this separation can (if I recall correctly) be avoided. The fact is, there needs to be an asynchronous, message-passing RPC mechanism to connect the various browser processes, and this explains the rationale for e.g. the QtWebChannel.

In order for the WebKit opcodes to inject a JavaScript proxy to the running instance of Csound into a Web page, there needs to be such an RPC connection.

I could try redoing what I have done for WebKitGTK using the QtWebEngine and QtWebChannel, as I already have done for CsoundQt, but the overhead of the Qt SDK is horrific.

gogins commented 2 years ago

This of course raises the question, why not just use CsoundQt which already does this stuff? Reasons why I have not been using CsoundQt so far:

However I will take another look.

Doesn't build with Qt6 SDK. Many many breaking changes in Qt6. I'm not pursuing this port. However I will perhaps fix a few things in the Qt5 implementation of CsoundQt.

gogins commented 2 years ago

Perhaps I can inject a WebRTC proxy for the plugin's instance of Csound into the Web page. I think this can be purely asynchronous.

gogins commented 2 years ago

The window.MessageChannel facility is no good for this, it is for renderer to worker or renderer to IFrame communications.

I don't see WebRTC in the GTK WebKit at all.

However it should be possible to use JSON-RPC with WebSockets, there are some libraries for that. For now, we only need asynchronous methods in the renderer to send Csound channel messages, diagnostic messages, scores, orchestras, and such.

gogins commented 2 years ago

I am going to work with CsoundQt and the Clang opcodes for a while, and see whether I really need to do the webkit-opcodes, or redo them with the QtWebEngine, or pursue the renderer proxy idea.

The clang_examples.csd piece crashes CsoundQt (although not, of course, when running in the terminal, but what would be the point of that if the point is to come HTML with Clang?).

gogins commented 2 years ago

This may be a clue, two LLVM libraries have been loaded, perhaps one is from qtcreator:

backtrace() returned 58 addresses
/usr/local/lib/libcsound64.so.6.0(+0x4c747) [0x7fbd3e273747]
/lib/x86_64-linux-gnu/libc.so.6(+0x46210) [0x7fbd3ae0a210]
/usr/lib/x86_64-linux-gnu/libLLVM-12.so.1(_ZN4llvm15LLVMContextImplD1Ev+0xa0) [0x7fbd0b53ebe0]
/usr/lib/x86_64-linux-gnu/libLLVM-12.so.1(_ZN4llvm11LLVMContextD1Ev+0x11) [0x7fbd0b53d131]
/usr/lib/x86_64-linux-gnu/libLLVM-12.so.1(+0x21b0378) [0x7fbd0c9fa378]
/usr/lib/x86_64-linux-gnu/libLLVM-13.so.1(+0x229b68c) [0x7fbc6c71668c]
/usr/lib/x86_64-linux-gnu/libLLVM-13.so.1(_ZN4llvm3orc14IRCompileLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE+0x136) [0x7fbc6c716216]
/usr/lib/x86_64-linux-gnu/libLLVM-13.so.1(_ZN4llvm3orc31BasicIRLayerMaterializationUnit11materializeESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EE+0x112) [0x7fbc6c71caa2]
/usr/lib/x86_64-linux-gnu/libLLVM-13.so.1(_ZN4llvm3orc19MaterializationTask3runEv+0x21) [0x7fbc6c6e47f1]
/usr/lib/x86_64-linux-gnu/libLLVM-13.so.1(+0x2274207) [0x7fbc6c6ef207]
/usr/lib/x86_64-linux-gnu/libLLVM-13.so.1(_ZN4llvm3orc16ExecutionSession22dispatchOutstandingMUsEv+0x17a) [0x7fbc6c6e5eba]
/usr/lib/x86_64-linux-gnu/libLLVM-13.so

No, not from qtcreator.

I have LLVM versions 5 through 13 installed. I can't remove the actual libLLVM-12.so, it's used by too many other installed packages.

gogins commented 2 years ago

Perhaps it's this:

/tmp/clang_opcode_qIELqs.cpp:274:64: warning: illegal character encoding in string literal [-Winvalid-source-encoding]
        m->declare("envelopes.lib/ar:author", "Yann Orlarey, St<E9>phane Letz");

No help.

gogins commented 2 years ago

Debugging in QtCreator

Got through loading plugins, no problem.

Got through csoundStart and down to calling musmon.

Crashes in init0, on the 17th init.

I see that I need to do this: https://stackoverflow.com/questions/44896217/how-to-save-set-solib-search-path.,

That worked, now I have localized the crash to the ExecuteAction call here in clang_opcodes.cpp:

        std::unique_ptr<CodeGenAction> emit_llvm_action(new EmitLLVMOnlyAction());
        if(!compiler_instance.ExecuteAction(*emit_llvm_action)) {
            return 1;
        }

What the hell is this?

Thread 1 "CsoundQt-d-html" hit Catchpoint 1
  Inferior loaded /usr/lib/x86_64-linux-gnu/dri/iris_dri.so
    /usr/lib/x86_64-linux-gnu/libLLVM-12.so.1
    /usr/lib/x86_64-linux-gnu/libzstd.so.1
    /usr/lib/x86_64-linux-gnu/libsensors.so.5
    /usr/lib/x86_64-linux-gnu/libdrm_radeon.so.1
    /usr/lib/x86_64-linux-gnu/libelf.so.1
    /usr/lib/x86_64-linux-gnu/libdrm_amdgpu.so.1
    /usr/lib/x86_64-linux-gnu/libdrm_nouveau.so.2
    /usr/lib/x86_64-linux-gnu/libvulkan.so.1
    /usr/lib/x86_64-linux-gnu/libedit.so.2
    /lib/x86_64-linux-gnu/libtinfo.so.6
gogins commented 2 years ago

This is probably it, libLLVM-12.so.1 is not loaded by plain Csound.

Also this: https://askubuntu.com/questions/1299067/ubuntu-20-04-no-driver-loaded-for-intel-iris-xe-graphics. Not sure if there is a fix.

I will try to update the Iris driver. The problem could be in that, or in the Qt library, or even in the Clang/LLVM infrastructure.

I have proved that the webkit_opcodes.csd file with clang_hello.csd added in works fine, so the LLVM crash is most likely due to a problem in the linkage of the Qt5 SDK.

I will install and build with LLVM-12. But there are breaking changes.

clang_opcodes.cpp:139:38: error: use of undeclared identifier 'SelfExecutorProcessControl'
        : execution_session(cantFail(SelfExecutorProcessControl::Create())), target_machine(std::move(target_machine)), data_layout(std::move(data_layout))
                                     ^
clang_opcodes.cpp:128:30: warning: field 'execution_session' is uninitialized when used here [-Wuninitialized]
    JITDylib &main_jit_dylib{execution_session.createBareJITDylib(name)};
                             ^
clang_opcodes.cpp:136:5: note: during field initialization in this constructor
    JITCompiler(

https://github.com/llvm/llvm-project/blob/llvmorg-12.0.1/clang/examples/clang-interpreter/main.cpp

gogins commented 2 years ago

I will try statically linking to LLVM. No, there is no single static LLVM library, there are a whole huge bunch of LLVM libraries, bt it must be that libLLVM-13.so links statically to all of its LLVM specific dependencies.

gogins commented 2 years ago

My video controller: 00:02.0 VGA compatible controller: Intel Corporation UHD Graphics 620 (rev 07)

gogins commented 2 years ago

mkg@xenakis:~/clang-opcodes$ glxinfo -B name of display: :0 display: :0 screen: 0 direct rendering: Yes Extended renderer info (GLX_MESA_query_renderer): Vendor: Intel (0x8086) Device: Mesa Intel(R) UHD Graphics 620 (KBL GT2) (0x5917) Version: 21.0.3 Accelerated: yes Video memory: 3072MB Unified memory: yes Preferred profile: core (0x1) Max core profile version: 4.6 Max compat profile version: 4.6 Max GLES1 profile version: 1.1 Max GLES[23] profile version: 3.2 OpenGL vendor string: Intel OpenGL renderer string: Mesa Intel(R) UHD Graphics 620 (KBL GT2) OpenGL core profile version string: 4.6 (Core Profile) Mesa 21.0.3 OpenGL core profile shading language version string: 4.60 OpenGL core profile context flags: (none) OpenGL core profile profile mask: core profile

OpenGL version string: 4.6 (Compatibility Profile) Mesa 21.0.3 OpenGL shading language version string: 4.60 OpenGL context flags: (none) OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.2 Mesa 21.0.3 OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

gogins commented 2 years ago

I am not pursuing coexistence in CsoundQt even though CsoundQt itself is much improved.

I am not moving my LLVM version back to 12.

I will proceed with the RPC proxy.

gogins commented 2 years ago

I will categorize and reorder this by descending preference pending a final decision.

JSON-RPC

JavaScript side

C++ side

WebSockets

JavaScript side

C++ side

Too many of these are basically just for Node.js.

gogins commented 2 years ago

I know I need Websockets, but I do not know if I need JSON-RPC. Wrong, it looks smarter to use JSON-RPC with whatever network protocol will carry it in this use case.

Wrong Wrong!! WebSockets are the only fit for this use. XMLHttpRequest obviously works but would be too slow. The only other network protocol usable in WebKit is WebSockets, and only it is conceivably fast enough for this use.

gogins commented 2 years ago
gogins commented 2 years ago

This obviously is going to work. I see the following issues right away:

gogins commented 2 years ago

Actually I think this is working well enough for now with Ajax. I think that for many applications bandwidth will be negligible and performance should not be an issue. I am leaving the WebSockets implementation aside until I get everything else worked out.

gogins commented 2 years ago

Add these functions from the Csound API to the Csound proxy: