Copy Paste in emscripten

frink commented 3 years ago

Tested the webapp on ChromeOS everything worked great. Resize is clean anti-alias is great. However, copy/paste did not seem to work at all. I don't know if this is a feature of your wrapper or something on my side. But Thought it would be good to report.

frink commented 3 years ago

Reading more into this it seems to be related to the security model of WASM in the browser. Not easily fixed...

pthom commented 3 years ago

Hello, Thanks for reporting this! You are right, javascript acces to the clipboard is limited for security reasons. There is a potential solution in order to export the clipboard from javascript to the PC. The other way around is more tricky, and less resistant to browser / platform variations.

I will study this in the next weeks.

frink commented 3 years ago

If you can post some info I'll be happy to jump down that rabbit hole with you...

frink commented 3 years ago

Think I found a workaround... The following code works!

<style>
        .emscripten {
            /*...*/
            z-index: 1000;
        }
</style>
<canvas class="emscripten" id="canvas" oncontextmenu="event.preventDefault()"></canvas>
<textarea id="clipping" style="width:0;height:0;border:0"  aria-hidden="true"></textarea>
<script>
async function copy(text) {
    document.getElementById("clipping").focus();
    const rtn = await navigator.clipboard.writeText(text);
    document.getElementById("canvas").focus(); 
}

async function paste() {
    document.getElementById("clipping").focus();
    const rtn = await navigator.clipboard.readText();
    document.getElementById("canvas").focus();
    return rtn;
}

async function test() {
    document.body.focus()
    await copy(Math.random())
    alert(await paste())
}

test();
</script>

The key is that you have to textarea element focusible which means it cannot be hidden or invisible. However, other elements can completely obscure the view so it is essentially hidden. I've also made the textarea zero width, height and border so that you don't even see this element.

I've also added aria-hidden="true" to avoid screen readers barking when you do something. I know Imgui isn't going to be accessible on the web without a lot of tinkering. But I figured you could at least start down that path...

In C++ I think it would look something like this:

EM_JS(void, copy, (const char* str), {
  Asyncify.handleAsync(async () => {
    document.getElementById("clipping").focus();
    const rtn = navigator.clipboard.writeText(str);
    document.getElementById("canvas").focus();
  });
});

EM_JS(char*, paste, (), {
  Asyncify.handleAsync(async () => {
    document.getElementById("clipping").focus();
    const str = navigator.clipboard.readText();
    document.getElementById("canvas").focus();
    const size = (Module.lengthBytesUTF8(str) + 1);
    const rtn = Module._malloc(size);
    Module.stringToUTF8(str, rtn, size);
    return rtn;
  });
});

I don't have a build environment to test this out at present.

But at least this should get you closer to the mark...

pthom commented 3 years ago

Many many thanks for your work! I'm back from holiday and I was away from keyboard for a while, so that I am sorry to answer late.

Copy and paste is not that easy under emscripten as I wrote you before.

I had a lengthy discussion about this with Andre Weissflog, who had already encountered this problem in his quite nice sokol libraries.

We discussed this in the context of "imgui manual" here

Basically I was able to partially solve this inside imgui manual:

Exporting to the system clipboard ("copy") works
Importing from the system clipboard ("paste") kind of works, but as far as I remember, your mileage will vary depending on the browser: It worked on half of the browser / platforms. I suspect this is were the browser will try harder to avoid potential security issues. So, it is disabled in the code for now but you could reenable it in the code and test it on your side (it is probably buggy since I did not put it into production)

I did not (yet) find a way to do it generically from inside hello_imgui, so that the application code (i.e the ImGui Manual window) needs to catch the "Ctrl-C" events manually.

Anyhow, here are some hints on how this is partially done inside imgui manual and sokol:

JsClipboardTricks.h and JsClipboardTricks.cpp provide the plumbing functions. Only the export is validated for now
WindowWithEditor.cpp handles the keyboard shortcuts for copying

I did not have time to work on porting this back to hello_imgui. If you are willing to help, I would appreciate it very much.

Thanks

pthom commented 3 years ago

On a side note, I saw that you are trying to use emscripten asyncify options. Those require sepcific compiler flags. A possible quick and dirty solution for the testing is to hack the file hello_imgui_cmake/emscripten/hello_imgui_emscripten_global_options.cmake(later we would need to find a more robust solution)

For example, line 8 could become:

set(CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS} "-s USE_SDL=2 -s USE_WEBGL2=1 -s WASM=1 -s FULL_ES3=1 -s ALLOW_MEMORY_GROWTH=1 -s ASYNCIFY -s 'ASYNCIFY_IMPORTS=[\"copy,paste\"]'")

frink commented 3 years ago

The Sokol approach is deprecated in most browsers. (Line 17 has a note on that...) The reason for the deprecation was tat the security concerns are so massive for the old approach as you mentioned above...

I don't think adding Asyncify to the build is at all a bad thing. Many new JS APIs are asynchronous and return promises. (webcams and fetch are two examples that come to mind...) Thus, many serious apps will need Asyncify anyway.

The code above calls the permissions dialog the first time you try to paste something. (copy always works fine...) My thought is that if the permission is denied you can fail gracefully but optionally return something to let you know you don't have proper access to the system clipboard so that the app can respond appropriately.

I'll look into the genericizing thing, but I think you may be right that the event needs to be caught by the imgui_impl_xxx.cpp. We should probably ask a questions upstream to make sure we are tracking with the general vision of Dear Imgui. I'll leave this open and get back to you...

sschoener commented 3 years ago

Dear future visitor from Google, I'm sure you're stumbling over this because it is as of writing the most useful result for copy/paste in Emscripten when using dear imgui. Let me help you out and slightly improve upon @frink 's excellent answer from above, because if you are here you might (like me) have no clue what you are doing and are new to this WebASM thing :)

The functions posted above for C++ are almost correct, they are a great starting point. I've tested them and fixed them up:

EM_JS(void, copy, (const char* str), {
    Asyncify.handleAsync(async () => {
        document.getElementById("clipping").focus();
        const rtn = await navigator.clipboard.writeText(UTF8ToString(str));
        document.getElementById("canvas").focus();
    });
});

EM_JS(char*, paste, (), {
    return Asyncify.handleAsync(async () => {
        document.getElementById("clipping").focus();
        const str = await navigator.clipboard.readText();
        document.getElementById("canvas").focus();
        const size = lengthBytesUTF8(str) + 1;
        const rtn = _malloc(size);
        stringToUTF8(str, rtn, size);
        return rtn;
    });
});

I'd also recommend putting a position: fixed on the textarea, otherwise the call to focus might scroll your view around.

slowriot commented 1 year ago

There's now a simple header-only library to achieve copy and paste from Emscripten in the browser: https://github.com/Armchair-Software/emscripten-browser-clipboard

This doesn't require you to modify the HTML of your page, add any text elements, or do anything other than include a single header.

Here's an example of putting it to use with ImGui:

#include <emscripten_browser_clipboard.h>
#include <imgui/imgui.h>
#include <iostream>

std::string content;  // this stores the content for our internal clipboard

char const *get_content_for_imgui(void *user_data [[maybe_unused]]) {
  /// Callback for imgui, to return clipboard content
  std::cout << "ImGui requested clipboard content, returning " << std::quoted(content) << std::endl;
  return content.c_str();
}

void set_content_from_imgui(void *user_data [[maybe_unused]], char const *text) {
  /// Callback for imgui, to set clipboard content
  content = text;
  std::cout << "ImGui setting clipboard content to " << std::quoted(content) << std::endl;
  emscripten_browser_clipboard::copy(content);  // send clipboard data to the browser
}

// ...

  emscripten_browser_clipboard::paste([](std::string const &paste_data, void *callback_data [[maybe_unused]]){
    /// Callback to handle clipboard paste from browser
    std::cout << "Clipboard updated from paste data: " << std::quoted(paste_data) << std::endl;
    content = std::move(paste_data);
  });

  // set ImGui callbacks for clipboard access:
  ImGuiIO &imgui_io = ImGui::GetIO();
  imgui_io.GetClipboardTextFn = get_content_for_imgui;
  imgui_io.SetClipboardTextFn = set_content_from_imgui;

Karm commented 1 year ago

Hello @pthom, @floooh, @slowriot,

Thanks for the awesome libraries. I love that I could fit visual6502remix on a 3.5" floppy.

I have been tinkering with ImGui/ImPlot and most recently adopted @slowriot's solution to carry out Copy and Paste. It works for me with the weird caveat that I have to do paste with a mouse middle button click, not with Ctrl+V. Hitting Ctrl+V doesn't fire JS "paste" event, although it does call ImGuiIO's GetClipboardTextFn. Not being sure I haven't messed up something in my Emscripten I took a look at @pthom's reference apps online and on @flooth's Visual6502 remix, opening Firefox dev console and injecting this code:

document.addEventListener(
"paste", (event) => {
console.log('paste event');
});

I can see that Ctrl+V does not fire the "paste" event on none of imgui_manual.html, implot_demo.html, while middle mouse button does. Just like my app that uses HelloImgui and Implot.

On the other hand imgui_manual.html with Sokol backend and visual6502remix log the "paste" event both when you hit Ctrl+V and mouse middle button.

Could it be that SDL backend in the aforementioned reference apps eats the keyboard events so there is no "paste" event for the JavaScript listener when you hit Ctrl+V? ...while Sokol backed apps don't have this problem?

I am eyeballing the SDL Emscripten config in HelloImGui and I am none the wiser. Naive imgui_io.WantCaptureKeyboard = false; did not help. I can see keyEventHandler functions in the Emscripten generated js, including event.preventDefault();, not sure if that's pertinent.

Thx for hints and also greetings to all who stumble on this thread in their Internet search engine of choice.

Edit:

As usual. Writing it all down helped a bit.

Adding

document.addEventListener('keydown', function(event){
    event.stopImmediatePropagation();
}, true);

document.addEventListener('keyup', function(event){
    event.stopImmediatePropagation();
}, true);

helps, you can fire "paste" events with Ctrl+V now, but there is this nasty side effect of Enter and Backspace not working any more. You can see it on e.g. on implot_demo.html, where you inject:

document.addEventListener('keydown', function(event){
    event.stopImmediatePropagation();
}, true);
document.addEventListener('keyup', function(event){
    event.stopImmediatePropagation();
}, true);
document.addEventListener(
"paste", (event) => {
console.log('paste event');
});

and the navigate to Code tab and try to type something.

Credit for the .stopImmediatePropagation(); goes to samuelnj.

pthom commented 1 year ago

Hi @Karm

Let's continue on this thread in the hope of helping future visitors that try to have a working copy-paste with emscripten.

You are definitely onto something! What you describe is strange, indeed. I could reproduce your issue using Firefox and Chrome under Linux and Windows. However, it does work under MacOS (with Command-V instead of Ctrl-V).

As far as your workaround is concerned, there is a way to let it only handle Ctrl-V (and thus to preserve Enter + Backspace), like this:

// Only stop propagation for Ctrl-V
window.addEventListener('keydown', function(event){
    if (event.ctrlKey && event.key == 'v')    
        event.stopImmediatePropagation();
}, true);

// Log paste events
document.addEventListener(
"paste", (event) => {
console.log('paste event');
});

Now, for more explanation; I suspect that this part of the javascript code emitted by emscripten might be part of the reason of these issues:

// code emitted by emscripten
function registerKeyEventCallback(target, userData, useCapture, callbackfunc, eventTypeId, eventTypeString, targetThread) {
    if (!JSEvents.keyEvent)
        JSEvents.keyEvent = _malloc(176);
    var keyEventHandlerFunc = function(e) {
        assert(e);
        var keyEventData = JSEvents.keyEvent;
        HEAPF64[keyEventData >> 3] = e.timeStamp;
        var idx = keyEventData >> 2;
        HEAP32[idx + 2] = e.location;
        HEAP32[idx + 3] = e.ctrlKey;
        HEAP32[idx + 4] = e.shiftKey;
        HEAP32[idx + 5] = e.altKey;
        HEAP32[idx + 6] = e.metaKey;
        HEAP32[idx + 7] = e.repeat;
        HEAP32[idx + 8] = e.charCode;
        HEAP32[idx + 9] = e.keyCode;
        HEAP32[idx + 10] = e.which;
        stringToUTF8(e.key || "", keyEventData + 44, 32);
        stringToUTF8(e.code || "", keyEventData + 76, 32);
        stringToUTF8(e.char || "", keyEventData + 108, 32);
        stringToUTF8(e.locale || "", keyEventData + 140, 32);
        if (getWasmTableEntry(callbackfunc)(eventTypeId, keyEventData, userData))
            e.preventDefault() // is this the reason why Ctrl-V is not handled by the browser?
    };
    var eventHandler = {
        target: findEventTarget(target),
        allowsDeferredCalls: true,
        eventTypeString: eventTypeString,
        callbackfunc: callbackfunc,
        handlerFunc: keyEventHandlerFunc,
        useCapture: useCapture
    };
    JSEvents.registerOrRemoveHandler(eventHandler)
}

Karm commented 1 year ago

@pthom Thanks for the reply. I can confirm that if I do this to my Emscripten SDK installation:

karm@localhost:~/Tools/emsdk/upstream/emscripten/src (main *)$ pwd
/home/karm/Tools/emsdk/upstream/emscripten/src
karm@localhost:~/Tools/emsdk/upstream/emscripten/src (main *)$ git diff
diff --git a/library_html5.js b/library_html5.js
index 55b1067..aae5b00 100644
--- a/library_html5.js
+++ b/library_html5.js
@@ -282,7 +282,11 @@ var LibraryHTML5 = {
       if (targetThread) JSEvents.queueEventHandlerOnThread_iiii(targetThread, callbackfunc, eventTypeId, keyEventData, userData);
       else
 #endif
-      if ({{{ makeDynCall('iiii', 'callbackfunc') }}}(eventTypeId, keyEventData, userData)) e.preventDefault();
+      if ({{{ makeDynCall('iiii', 'callbackfunc') }}}(eventTypeId, keyEventData, userData)) {
+          if(!(event.ctrlKey && event.key == 'v')) {
+              e.preventDefault();
+          }
+      }
     };

     var eventHandler = {

...and re-run my CMake based CLion build, It Works :tm: just fine now :smile:

I have no JS experience and I have a hunch that opening a PR to Emscripten GitHub with this patch would just propagate a hack that is omitting a variety of corner cases (Cmd on Mac? 'v' with Capslock on?).

What do you think would be the best course of action for Copy-Paste capability to finally cease to be an issue now?

pthom commented 1 year ago

@Karm : Congrats, this is quite a thorough analysis!

You may be onto something inside emscripten! May be you should open an issue in their repository, and mention the solution you have. If someone stumbles upon your issue and reacts/analyzes it, this would be nice.

Xadiant commented 11 months ago

I have been following this thread for a long while as a valuable resource on getting copy/paste working in my app. So I wanted to shared how I connected it up to ImGui for the time being:

It started out just like the @slowriot 's library suggests by connecting to the ImGui clipboard callback functions, and adding @Karm 's javascript event handler for the keydown event:

    emscripten_browser_clipboard::paste([](std::string const &paste_data, void *callback_data [[maybe_unused]]){
        std::cout << "Copied clipboard data: " << paste_data << std::endl;
        clipboardContent = std::move(paste_data);
        ImGui::GetIO().AddKeyEvent(ImGuiKey_ModCtrl, true);
        ImGui::GetIO().AddKeyEvent(ImGuiKey_V, true);
        simulatedImguiPaste = true;
    });

    ImGui::GetIO().GetClipboardTextFn = get_content_for_imgui;
    ImGui::GetIO().SetClipboardTextFn = set_content_from_imgui;

    EM_ASM({
        window.addEventListener('keydown', function(event){
            if (event.ctrlKey && event.key == 'v')    
                event.stopImmediatePropagation();
        }, true);
    });

Copying now works, but then ImGui does not receive the required key events to paste. Unfortunately there isn't a great way to trigger the paste as each widget checks the io for the paste key combo, so I had to manually add the event with "AddKeyEvent". Luckily the browser delivers this event between ImGui frames, but beware this may not always be the case. (Will also need to check for mac and use command key instead)

This works, but the paste will then be repeating, so after a render cycle we check if we manually triggered CTRL + V and then undo it:

    if (simulatedImguiPaste) {
        simulatedImguiPaste = false;
        ImGui::GetIO().AddKeyEvent(ImGuiKey_ModCtrl, false);
        ImGui::GetIO().AddKeyEvent(ImGuiKey_V, false);
    }

I really do not like how fragile this way of faking inputs then cleaning up after feels, but it seems to work for now.

Karm commented 6 months ago

...and I came back to the project, with a new system, set it all up again, built it and it still doesn't work without that workaround in https://github.com/emscripten-core/emscripten/pull/19510
Even with the aforementioned workaround, it still doesn't work in Safari, despite I modified it to:

      if ({{{ makeDynCall('iipp', 'callbackfunc') }}}(eventTypeId, keyEventData, userData)) {
        if(!((event.ctrlKey || event.metaKey) && event.key == 'v')) {
          e.preventDefault();
        }
      }

Karm commented 6 months ago

And I open Sokol app like https://floooh.github.io/visual6502remix/, hit Alt+A, assembly edit window opens, and you can copy-paste to and from that window on Linux, Windows, Firefox, Chrome and also in Safari on Mac.

digitalsignalperson commented 2 months ago

This doesn't require you to modify the HTML of your page, add any text elements

Does anyone know of a solution that takes the approach of mirroring all rendered text to hidden html? It seems like it would be ideal for accessibility, and retains the familiar experience of interacting with html text.

Technically I think it would be possible to get the coordinates of every string rendered by imgui, and then each frame update the html so invisible text is displayed at the exact same positions. Thoughts?

vertexi commented 2 months ago

This doesn't require you to modify the HTML of your page, add any text elements

Does anyone know of a solution that takes the approach of mirroring all rendered text to hidden html? It seems like it would be ideal for accessibility, and retains the familiar experience of interacting with html text.

Technically I think it would be possible to get the coordinates of every string rendered by imgui, and then each frame update the html so invisible text is displayed at the exact same positions. Thoughts?

You can check this repo https://github.com/zhobo63/imgui-ts which use a overlay html element to overlay the Imgui::input. In my opinion, you can use text html element to overlay the Imgui::text, so then the text is selectable in browser.

pthom / hello_imgui

Copy Paste in emscripten #3

Edit: