search

w3c / clipboard-apis

Clipboard API and events
https://w3c.github.io/clipboard-apis/
Other
143 stars 41 forks source link

Support clipboard write() for audio formats (midi, wav) #218

Open chr15m opened 2 weeks ago

chr15m commented 2 weeks ago

Hello! I am a developer of web based audio tools like https://dopeloop.ai/melody-generator and https://pro.sfxr.me. I am working on more tools at the moment for generating samples as well as MIDI.

Something users have asked me for is the ability to copy MIDI data directly from the page so that they can paste into their DAW such as Abelton, Garage Band, etc. At the moment I can't do this because the clipboard API allowed mimetypes don't include MIDI data.

I am currently working on a drum machine and it would likewise be great to let users copy drum loops and individual drum samples (as WAV formatted blobs) into their buffer to paste into their DAWs.

At the moment both of these use cases require a round trip through downloading the data to a local file (with .mid or .wav extension respectively) and then the user must import that file into their DAW or music app. The feedback I get is this is cumbersome. It would be a great usability enhancement to let them copy audio data directly from web apps so they can paste them into their DAWs, just like they can currently do with PNG files and image editors.

Thank you for your time!

P.S. Would love to hear @cwilso's opinion on this.

tomayac commented 2 weeks ago

I love the user case. This is more of a browser support request than a spec issue, though. For Chrome, the Async Clipboard API is being worked on in the context of Project Fugu 🐡. You can request new Fugu features through our bug tracker. For orientation, this is a similar feature request where support for a new image format is requested. If you send me the bug URL, I'm happy to help triage it. It's pretty much the same flow for the other browsers as well.

(In Chromium-based browsers, there's web custom formats that allow you to copy anything, but it depends on support for the web custom format on the receiving end, that is, Abelton and friends.)

saschanaz commented 2 weeks ago

It would be nice to speak based on the specification rather than Chromium implementation here, for the interop sake.

Custom formats are not really suitable if the purpose is for interoperability with other (native) apps, right?

tomayac commented 2 weeks ago

It would be nice to speak based on the specification rather than Chromium implementation here, for the interop sake.

The spec makes no assumptions as to the supported formats, it's all just abstract data. Different browsers may support different MIME types.

Custom formats are not really suitable if the purpose is for interoperability with other (native) apps, right?

Correct, the native app needs to be aware of the web custom format.

saschanaz commented 2 weeks ago

There are mandatory/optional types so I see at least some level of assumption. This issue can be read as a request to add a new item in the list.

chr15m commented 2 weeks ago

Yes it is a request to add audio mimetypes to the mandatory list. At least audio/wav and audio/midi.

I personally don't use Chrome as my daily browser, but obviously many users of my software do.

tomayac commented 2 weeks ago

There are mandatory/optional types so I see at least some level of assumption. This issue can be read as a request to add a new item in the list.

That's fair, albeit there's ClipboardItem.supports(), so there's a baseline assumption that you can't know until you know—outside of the mandatory types. Since the list has shrunk in the past, I think there's limited interest in growing it again.

saschanaz commented 2 weeks ago

The new item indeed would only make sense when we get enough implementor interest.

chr15m commented 2 weeks ago

get enough implementor interest

Cool. What does this mean in practice? What's the best way for me to make that happen? File issues on each browser as suggested previously in the issue?