search

cablehead / stacks

an experimental "tool for thought"
https://stacks.cross.stream
MIT License
95 stars 5 forks source link

feat: Windows support #49

Open cablehead opened 9 months ago

cablehead commented 9 months ago

Do you use windows as your daily driver, and have a passion for clipboard managers? We'd love to hear from you!

What's needed:

See: https://github.com/cablehead/stacks/tree/main/src-tauri/bin https://github.com/cablehead/workspace/tree/x-macos-pasteboard

https://github.com/cablehead/stacks/blob/main/src-tauri/src/spotlight.rs

Likely a few more things??

fdncred commented 8 months ago

I figure there are some crates that do this type of thing already. Here's one. https://github.com/DoumanAsh/clipboard-master

This looks interesting too https://github.com/CrossCopy/tauri-plugin-clipboard

cablehead commented 8 months ago

I'd like to be able dig into the raw capabilities of the clipboard on each platform.

Two examples for macOS:

When you copy text on a Safari web page, this is what ends up in the macOS clipboard:

{
  "types": {
    "com.apple.webarchive": "YnBsaXN0MDDRAQJfEA9XZWJNYWluUmVzb3VyY2XVAwQFBgcICQoLDF5XZWJSZXNvdXJjZVVSTF8QFFdlYlJlc291cmNlRnJhbWVOYW1lXxAPV2ViUmVzb3VyY2VEYXRhXxATV2ViUmVzb3VyY2VNSU1FVHlwZV8QG1dlYlJlc291cmNlVGV4dEVuY29kaW5nTmFtZV8Qnmh0dHBzOi8vYWxleGFuZGVyb2JlbmF1ZXIuY29tL29sbG9zLyM6fjp0ZXh0PVdoaWxlJTIwYnVpbGRpbmclMjBhbmQlMjBsaXZpbmclMjBpbiUyME9MTE9TJTJDJTIwSSUyMGxlYXJuZWQlMjBhJTIwZmV3JTIwdGhpbmdzJTIwYWJvdXQlMjBwZXJzb25hbCUyMHNvZnR3YXJlJTNBUE8RAks8IURPQ1RZUEUgaHRtbD48c3BhbiBzdHlsZT0iY2FyZXQtY29sb3I6IHJnYigxNywgMTcsIDE3KTsgY29sb3I6IHJnYigxNywgMTcsIDE3KTsgZm9udC1mYW1pbHk6ICZxdW90O0NyaW1zb24gUHJvJnF1b3Q7LCBzZXJpZjsgZm9udC1zaXplOiAyMi40cHg7IGZvbnQtc3R5bGU6IG5vcm1hbDsgZm9udC12YXJpYW50LWNhcHM6IG5vcm1hbDsgZm9udC13ZWlnaHQ6IDMwMDsgbGV0dGVyLXNwYWNpbmc6IG5vcm1hbDsgb3JwaGFuczogYXV0bzsgdGV4dC1hbGlnbjogbGVmdDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBub3JtYWw7IHdpZG93czogYXV0bzsgd29yZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgYmFja2dyb3VuZC1jb2xvcjogcmdiKDI0MiwgMjQwLCAyMzcpOyB0ZXh0LWRlY29yYXRpb246IG5vbmU7IGRpc3BsYXk6IGlubGluZSAhaW1wb3J0YW50OyBmbG9hdDogbm9uZTsiPldoaWxlIGJ1aWxkaW5nIGFuZCBsaXZpbmcgaW4gT0xMT1MsIEkgbGVhcm5lZCBhIGZldyB0aGluZ3MgYWJvdXQgcGVyc29uYWwgc29mdHdhcmU6PC9zcGFuPll0ZXh0L2h0bWxVVVRGLTgACAALAB0AKAA3AE4AYAB2AJQBNQE2A4UDjwAAAAAAAAIBAAAAAAAAAA0AAAAAAAAAAAAAAAAAAAOV",
    "public.html": "PHNwYW4gc3R5bGU9ImNhcmV0LWNvbG9yOiByZ2IoMTcsIDE3LCAxNyk7IGNvbG9yOiByZ2IoMTcsIDE3LCAxNyk7IGZvbnQtZmFtaWx5OiAmcXVvdDtDcmltc29uIFBybyZxdW90Oywgc2VyaWY7IGZvbnQtc2l6ZTogMjIuNHB4OyBmb250LXN0eWxlOiBub3JtYWw7IGZvbnQtdmFyaWFudC1jYXBzOiBub3JtYWw7IGZvbnQtd2VpZ2h0OiAzMDA7IGxldHRlci1zcGFjaW5nOiBub3JtYWw7IG9ycGhhbnM6IGF1dG87IHRleHQtYWxpZ246IGxlZnQ7IHRleHQtaW5kZW50OiAwcHg7IHRleHQtdHJhbnNmb3JtOiBub25lOyB3aGl0ZS1zcGFjZTogbm9ybWFsOyB3aWRvd3M6IGF1dG87IHdvcmQtc3BhY2luZzogMHB4OyAtd2Via2l0LXRleHQtc3Ryb2tlLXdpZHRoOiAwcHg7IGJhY2tncm91bmQtY29sb3I6IHJnYigyNDIsIDI0MCwgMjM3KTsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyBkaXNwbGF5OiBpbmxpbmUgIWltcG9ydGFudDsgZmxvYXQ6IG5vbmU7Ij5XaGlsZSBidWlsZGluZyBhbmQgbGl2aW5nIGluIE9MTE9TLCBJIGxlYXJuZWQgYSBmZXcgdGhpbmdzIGFib3V0IHBlcnNvbmFsIHNvZnR3YXJlOjwvc3Bhbj4=",
    "com.apple.WebKit.custom-pasteboard-data": "AQAAAB0AAAABaHR0cHM6Ly9hbGV4YW5kZXJvYmVuYXVlci5jb20AAAAAAAAAAAAAAAAAAAAA",
    "public.rtf": "e1xydGYxXGFuc2lcYW5zaWNwZzEyNTJcY29jb2FydGYyNzU3Clxjb2NvYXRleHRzY2FsaW5nMFxjb2NvYXBsYXRmb3JtMHtcZm9udHRibFxmMFxmbmlsXGZjaGFyc2V0MCBDcmltc29uUHJvUm9tYW4tTGlnaHQ7fQp7XGNvbG9ydGJsO1xyZWQyNTVcZ3JlZW4yNTVcYmx1ZTI1NTtccmVkMTRcZ3JlZW4xNFxibHVlMTQ7fQp7XCpcZXhwYW5kZWRjb2xvcnRibDs7XGNzc3JnYlxjNjY2N1xjNjY2N1xjNjY2Nzt9ClxkZWZ0YWI3MjAKXHBhcmRccGFyZGVmdGFiNzIwXHNhNDQ4XHBhcnRpZ2h0ZW5mYWN0b3IwCgpcZjBcZnM0NFxmc21pbGxpMjI0MDAgXGNmMiBcZXhwbmQwXGV4cG5kdHcwXGtlcm5pbmcwClxvdXRsMFxzdHJva2V3aWR0aDAgXHN0cm9rZWMyIFdoaWxlIGJ1aWxkaW5nIGFuZCBsaXZpbmcgaW4gT0xMT1MsIEkgbGVhcm5lZCBhIGZldyB0aGluZ3MgYWJvdXQgcGVyc29uYWwgc29mdHdhcmU6fQ==",
    "public.utf16-external-plain-text": "//5XAGgAaQBsAGUAIABiAHUAaQBsAGQAaQBuAGcAIABhAG4AZAAgAGwAaQB2AGkAbgBnACAAaQBuACAATwBMAEwATwBTACwAIABJACAAbABlAGEAcgBuAGUAZAAgAGEAIABmAGUAdwAgAHQAaABpAG4AZwBzACAAYQBiAG8AdQB0ACAAcABlAHIAcwBvAG4AYQBsACAAcwBvAGYAdAB3AGEAcgBlADoA",
    "public.utf8-plain-text": "V2hpbGUgYnVpbGRpbmcgYW5kIGxpdmluZyBpbiBPTExPUywgSSBsZWFybmVkIGEgZmV3IHRoaW5ncyBhYm91dCBwZXJzb25hbCBzb2Z0d2FyZTo="
  },
  "change": 6507,
  "source": "com.apple.Safari"
}

  1. stacks isn't using it yet, but that source field would be interesting to surface.

  2. currently stacks only pulls out the basic string that was copied from "public.utf8-plain-text"

% bp | jq -r '.types["public.utf8-plain-text"]' | base64 -D
While building and living in OLLOS, I learned a few things about personal software:

But, there's an amazing nugget:

% bp | jq -r '.types["com.apple.webarchive"]' | base64 -D > /tmp/o.webarchive && webarchive inspect /tmp/o.webarchive
WebArchive of "https://alexanderobenauer.com/ollos/#:~:text=While%20building%20and%20living%20in%20OLLOS%2C%20I%20learned%20a%20few%20things%20about%20personal%20software%3A" ("text/html", 587 bytes): 0 subresources, 0 subframe archives totalling 587 bytes

the "com.apple.webarchive" stashed on the clipboard gives you site url, PLUS the #:~:text=, which links you directly to the text on page.

The goal is to eventually surface this data too, so just copying text on web pages gives you the ability to get back to the site, and location you copied from.

I'm curious what a kitchen sink dump of the clipboard looks like on Windows, and Linux?

devyn commented 8 months ago

I think probably the most portable advanced clipboard spec on Linux would be the freedesktop interface available over D-Bus, which allows an application to advertise that it put content on the clipboard and what mime types it supports, and then for other applications to request that clipboard data be written somewhere in its chosen mime type. However, in order for this to be available, the user must be running some kind of application that provides a clipboard manager - an X11 server won't provide this on its own. Desktop environments in practice always do, but someone using a custom setup with a plain window manager might not.

That said, the situation on Linux is a bit complicated, as there are multiple layers of clipboards - desktop environments like KDE and GNOME might implement even more behavior on top of that, such as KDE having a clipboard manager which keeps history of things copied to it rather than just the most recent one. You might be able to access this more advanced functionality with APIs specific to those environments.

Under that, you also have the X11 or Wayland clipboard, which CLI apps often use via xclip, or wl-copy/wl-paste. Both of these also work by having an application service requests for clipboard data, and the CLI tools I just mentioned usually daemonize to handle those requests.

X11 has three clipboards: SELECTION, PRIMARY, and SECONDARY. I don't think SECONDARY is often used, but SELECTION essentially is expected to automatically copy any text you select on the screen, which you can usually paste with the middle mouse button. PRIMARY is used for Ctrl-C/Ctrl-V explicit copy/paste. There is no support for different MIME types, so you don't really know what kind of data is on the clipboard. Applications that are using the X11 clipboard might do magic number checks like file to try to figure that out. Of course, X has a very long history (almost 40 years!), and basically any kind of graphical environment would be expected to support this - even Wayland pretty much always has XWayland running, which would support this. But it's very basic.

Wayland doesn't have SECONDARY, as far as I can tell, and works more similarly to the freedesktop spec, which makes sense as it's not a legacy thing. It also supports mime types. Otherwise though it still also has a selection clipboard and a primary clipboard, just like X11.

Then, there's also the case of a non-graphical environment, like an SSH session or VC - some CLI applications will try to talk to screen or tmux to use their clipboard functionality if they're present. This isn't universal, but definitely some text editors do.

All of this of course also applies to BSD using X or Wayland, it's not Linux-specific.

cablehead commented 8 months ago

Currently recording clipboard activity looks like this:

Spawn a platform specific CLI tool that emits JSON that includes everything the platform has to offer on stdout when there's a clipboard update.

Reading this CLI's stdout, for each new item, is currently hardcoded to pull out the interesting data, for a macOS specific payload. Currently only plain text and PNG images are recognized. These are then added as a new clips to the event stream.

Spawning a platform specific subprocess is ganky, but serviceable. It'd be better if this was either compiled in using FFI or more interestingly, I've been chatting with @nilslice from the extism project about how to make this a WASM plugin.

More urgently though, the payload from the platform clipboard monitor should be put as-is into the event store as a new clip.

I'm planning to add the ability to register modules (potentially WASM plugins, or nushell snippets!) which can monitor the event stream and take action on payloads. The current hardcoded mac specific code would be one of these modules. It'd see the macOS shape clipboard data, extract the interesting parts and immediately issue an update event to update clip in place, so to the end user it would work as it does now.

You should be able to navigate the history of clips. A long standing issue is if you right click an image. on a web page for instance, and copy, you don't get that image on your clipboard. With the above in place, you could review the raw payload, notice how the raw image is stored, and then update the module to extract it correctly.

Supporting all the different flavors of Linux clipboards could then be incrementally fleshed. Step 1., for a given setup, write a module that converts the available information to a JSON payload: you'll see these raw payloads in Stacks whenever you copy something. Step 2., write a module which looks for these payloads and issues an update event for how you'd like to work with that clip type.

ChurchTao commented 8 months ago

Let me self-recommend my crate. Clipboard API (text | image | rich text | html | files | monitoring changes) https://github.com/ChurchTao/clipboard-rs

cablehead commented 5 months ago

Bookmarking: Stacks' api currently only listens on a local unix domain socket. We should add the option to listen on a TCP port, and / or potentially a named pipe, for Windows support (not sure if a named pipe is a sane option?)