Native clipboard handling

[sadiuk]

Description

This issue covers native clipboard support on several operating systems, including Windows, Linux, macOS, IOS and Android. It also contains suggestions for possible lightweight cross-platform libraries which can be used in Nabla to simplify this process.

Native Implementations

Windows

The clipboard support on Windows is very straightforward. A clipboard is a global buffer that can be used between applications to store the data of various formats. In order to use the clipboard you must first open it:

#include <windows.h>

int32_t res = OpenClipboard(0);
assert(res != 0);

The only parameter is a handle to the window to be associated with the open clipboard. If this parameter is NULL, the open clipboard is associated with the current task.

To copy data into the clipboard, you first must clear the data, currently stored in it and then fill in the clipboard buffer:

ClearClipboard();
const char* data = "Data to be stored in clipboard";
const size_t data_size = strlen(data) + 1;
HGLOBAL h = GlobalAlloc(GMEM_MOVEABLE, data_size);
strcpy(GlobalLock(h), LPCSTR(data));
GlobalUnlock(h);
SetClipboardData(CF_TEXT, h);

The first input parameter for SetClipboardData is the data format. The documentation on supported clipboard formats on Windows can be seen here.

To get the data, currently stored in the clipboard, the process is even more simple:

const char* data = (const char*)GetClipboardData(CF_TEXT);

When you're done with clipboard, you must close it:

CloseClipboard();

Linux

General

The clipboard on Linux is supported via the X11 library. The clipboard managing process is a bit different then on Windows and a little more cumbersome.

The X11 library is a multi-buffer library, but the only two buffers we're gonna focus on are: 1) Clipboard - the common copy-paste buffer. 2) Primary - the implicit mouse selection feature buffer. Each time the user selects the text, it is being copied to the Primary buffer.

Unlike Windows, Linux clipboard buffers are not global objects, but application-side buffers. To change the buffer data, an application just changes its local data and notifies the server than it is the new owner of the buffer. To get the data, currently stored in the clipboard, an application must request it from the current owner.

In code

The first two things you need to use the clipboard features are X11 server connection (display) and the window.

#include <X11/Xlib.h>

Display * display = XOpenDisplay(0);
Window window = XCreateSimpleWindow(display, RootWindow(display, N), 0, 0, 1, 1, 0, BlackPixel(display, N), WhitePixel(display, N));

We also need to create property atoms which are unique identifiers, associated with the properties:

Atom targets_atom = XInternAtom(display, "TARGETS", 0);
Atom text_atom = XInternAtom(display, "TEXT", 0);
Atom utf8 = XInternAtom(display, "UTF8_STRING", 1);
Atom selection = XInternAtom(display, "CLIPBOARD", 0); // Can also be "PRIMARY"

Then we set this application to be the owner of the buffer:

XSetSelectionOwner (display, selection, window, 0);
if (XGetSelectionOwner (display, selection) != window) return;

Some event handling:

XEvent event;
XNextEvent(display, &event);
if(event.type == SelectionRequest)
{
    XSelectionRequestEvent * xsr = &event.xselectionrequest;
    XSelectionEvent ev = {0};
    int R = 0;
    ev.type = SelectionNotify;
    ev.display = xsr->display; 
    ev.requestor = xsr->requestor;
    ev.selection = xsr->selection; 
    ev.time = xsr->time;
    ev.target = xsr->target;
    ev.property = xsr->property;
    if (ev.target == utf8)
        R = XChangeProperty(ev.display, ev.requestor, ev.property, utf8, 8, PropModeReplace, text, size);
    else ev.property = None;
    if ((R & 2) == 0) XSendEvent (display, ev.requestor, 0, 0, (XEvent *)&ev); // this actually copies the data to clipboard
}

The pasting process is a bit more simple:

const char* data_to_get = nullptr;
Atom utf8 = XInternAtom(display, "UTF8_STRING", True);
XEvent event;
int format;
unsigned long N, size;
char * data, * paste_data = 0;
Atom target;
Atom CLIPBOARD = XInternAtom(display, "CLIPBOARD", 0); // Can also be "PRIMARY"
Atom XSEL_DATA = XInternAtom(display, "XSEL_DATA", 0);

First request a buffer:

XConvertSelection(display, CLIPBOARD, utf8, XSEL_DATA, window, CurrentTime);
XSync(display, 0);
XNextEvent(display, &event);

Then wait until the owner prepares a buffer.

while (event.type != SelectionNotify)
{
    XNextEvent(display, &event);
}

And finally read buffer content from window property:

XGetWindowProperty(event.xselection.display, event.xselection.requestor, event.xselection.property, 0L,(~0L), 0, AnyPropertyType, &target, &format, &size, &N,(unsigned char**)&data);
if(target == utf8) {
    paste_data = strndup(data, size);
    XFree(data);
}

In the end you must delete the property:

XDeleteProperty(event.xselection.display, event.xselection.requestor, event.xselection.property);

Good resources about X11 clipboard copy-pasting

SO (The second answer), and The Minimal X11 Copy-Pasting Functionality Implementation

MacOS, IOS

For Mac OS(macOS 10.0+) Apple provides a nice interface of clipboard management - NSPasteboard. Also there is a clipboard version, available on both MacOS and IOS - UIPasteboard

To use the pasteboard (it's called pasteboard on macOS/IOS, so I'll use that term further on) you need to get the global pasteboard object:

MacOS-only option

NSPasteboard *pasteBoard = [NSPasteboard generalPasteboard];

Option available on both MacOS and IOS

UIPasteboard *pasteBoard = [UIPasteboard generalPasteboard];

Then, to copy text into the pasteboard, use these two lines:

NSString* string_to_write = @"Data to be stored in the pasteboard";
BOOl pasted = [pasteBoard setString:string_to_write forType:NSStringPboardType];

You can also store data of different format, using setData(_:forType:) method.

To get the text, currently stored in the pasteboard, do this:

NSString* data_from_pasteboard = [pasteBoard stringForType:NSStringPboardType];

Android

For android there is a good class that helps you manage clipboard - ClipboardManager

First create an instance of the ClipboardManager class:

ClipboardManager clipboard = (ClipboardManager) getSystemService(Context.CLIPBOARD_SERVICE); 

Then create a ClipData class instance and provide it with some data and push it to the clipboard:

String text = "Text to be stored in the stored in the clipboard";
ClipData clip = ClipData.newPlainText(label, text);
clipboard.setPrimaryClip(clip);

It is also possible to set the data of different formats, using the static ClipData methods.

To get the data, stored in the clipboard:

ClipData data = getPrimaryClip();
CharSequence text = data.getPrimaryClip().getText();

Cross-platform Libraries for clipboard management

Apparently there are not many open-source cross-platform libraries for clipboard management. 1) Clip (MIT) - A lightweight library that works with Windows, Linux and Mac OS. This one is probably the closest to ideal. No IOS/Android implementation though. 2) Qt (GPL3, LGPL3) - Obviously not a good choice, the only advantage is that it supposts all 5 platforms.

IMO, the Clip library is the one to go with. I think it's not that big of a problem to make it work on IOS (even though it uses the way of clipboard management, not available in IOS), and the Android support is something that should be added on our own.

[devshgraphicsprogramming]

@sadiuk what about Linux without X11 (Wayland session) ?

[devshgraphicsprogramming]

https://github.com/dacap/clip looks like a good codebase to cannibalise but not necessarily include as a 3rdparty dependency (missing Wayland and Android support).

P.S. It seems that Windows and X11 care about the window/display used, but Android and Apple has a global clipboard? or do they need any information about the window?

[sadiuk]

P.S. It seems that Windows and X11 care about the window/display used, but Android and Apple has a global clipboard? or do they need any information about the window?

With Apple the global clipboard is just a pointer that you don't even need to call any functions to get access to, and no function calls require any arguments like window/display. Apple actually has two clipboards - global and local (in-application). The one, written in the issue is global, so in case of apple it's quite easy. With Android the situation is similar, you need like 3 lines of code to copy data to clipboard and i haven't seen any call that requires any info about the window.

Btw, windows doesn't require any window data either. It's just X11.

[devshgraphicsprogramming]

The only parameter is a handle to the window to be associated with the open clipboard. If this parameter is NULL, the open clipboard is associated with the current task.

The point is that windows can take info about the window into account.

Regarding the info with X11, great, but I also need to know about Wayland.

[sadiuk]

@sadiuk what about Linux without X11 (Wayland session) ?

Haven't thought of it. I'll check it out later, cause there is not that much info about that stuff, so i'll have to dig deeper. The only thing i see now is this SO stuff: https://stackoverflow.com/questions/39616066/wayland-clipboard-api/40027443

[sadiuk]

The only parameter is a handle to the window to be associated with the open clipboard. If this parameter is NULL, the open clipboard is associated with the current task.

The point is that windows can take info about the window into account.

Regarding the info with X11, great, but I also need to know about Wayland.

Oh, i get it. Then i also have to check whether Apple and Android API can take that stuff.

[devshgraphicsprogramming]

@sadiuk what about Linux without X11 (Wayland session) ?

Haven't thought of it. I'll check it out later, cause there is not that much info about that stuff, so i'll have to dig deeper. The only thing i see now is this SO stuff: https://stackoverflow.com/questions/39616066/wayland-clipboard-api/40027443

ok this seems nice to cannibalise then https://github.com/bugaevc/wl-clipboard

[sadiuk]

@sadiuk what about Linux without X11 (Wayland session) ?

Haven't thought of it. I'll check it out later, cause there is not that much info about that stuff, so i'll have to dig deeper. The only thing i see now is this SO stuff: https://stackoverflow.com/questions/39616066/wayland-clipboard-api/40027443

ok this seems nice to cannibalise then https://github.com/bugaevc/wl-clipboard

yeah pretty good

[devshgraphicsprogramming]

Anyway it seems that because of the weirdness of the Linux desktop (nobody seemed to have thought that a non-GUI server might benefit from a clipboard as an OS-user-level thing, but I digress) the way that one should be able to get to a clipboard should be via a window XD

seems a ui::IClipboard* IWindow::getClipboard(); is the only way

then if we have a window agnostic clipboard like Apple/Android, then the pointer returned would be to a singular object static smart_refctd_ptr<ui::CClipboardPLATFORM> ui::CWindowPLATFORM::g_clipboard.

P.S. @Crisspl thoughts?

[Crisspl]

https://stackoverflow.com/a/15691001/5538150 Seems like "PNG" will be best for clipboard format for images on Windows. It's recognized by popular softwares. Handling transparency is a mess, no standard, everyone does it differently. No idea what about floating point images (EXR). Gimp converts them to rgb8 while copying and ignores alpha.

[devshgraphicsprogramming]

https://stackoverflow.com/a/15691001/5538150 Seems like "PNG" will be best for clipboard format for images on Windows. It's recognized by popular softwares. Handling transparency is a mess, no standard, everyone does it differently. No idea what about floating point images (EXR). Gimp converts them to rgb8 while copying and ignores alpha.

ok so for images we just always convert them to PNG and put the PNG "file" in the clipboard P.S. I wonder if for images of "fatter" bit depths we could just put an EXR in the clipboard XD

[devshgraphicsprogramming]

P.S. @sadiuk close this issue after implementing Linux and Android stuff in the ui and system namespaces.