search

taoensso / sente

Realtime web comms library for Clojure/Script
https://www.taoensso.com/sente
Eclipse Public License 1.0
1.73k stars 193 forks source link

Transferring blobs / binary data #381

Closed nanomonkey closed 2 years ago

nanomonkey commented 3 years ago

Many thanks for Sente, I've been happily using it for some time.

Are there any examples of sending and receiving binary files, such as images or database blobs? I'm unsure if there is a more appropriate route than base64 encoding/decoding. Is it possible to open up a binary websocket for such messages?

Thanks again!

ptaoussanis commented 3 years ago

Hey there, apologies for the slow response!

Sente unfortunately doesn't currently provide anything specifically for binary messages out-the-box. Haven't given it much thought, would be open to seeing ideas/PRs 👍

Otherwise base64 would be one option in the meantime.

Cheers!

awb99 commented 2 years ago

Binary blobs would be cool. Especially when sending larger payload data there is a significant overhead with the encoding. Now that say Apache aeriw can be sent as blobs via http get it would be cool to be able to send them via sente websocket as well.

rosejn commented 2 years ago

Hi, so I've implemented binary support, and unlike some of the previous discussions around what this would take, it only required minor modifications of about 10 lines of code, not including the packers. I'm using this with msgpack in both Clojure and Clojurescript, and I've extended it to work with typed arrays on both ends, while also supporting all of the Clojure data structures and base value types in binary form. Binary blobs, files, or anything else would be trivial to add. For now though, I'd like to try to get these modifications accepted, and then I'm happy to also contribute the msgpack implementation if you'd like to include that in sente. Those require 2 external libraries, although if I get the time I'd like to unify them and make a new msgpack.cljc library that cleanly implements it once and for all. We needed this ASAP for a product release though, so for now here we are.

Here's the whole diff: https://github.com/rosejn/sente/commit/3de8274f7451d402d342f327b174672eaf0724e3

In short, the current code wraps the string to transmit over the wire with an optional + or - and a callback UUID in case the user hopes for an RPC style reply. That was the only code which relied on the msg being a string. I changed that to instead wrap the msg in a clojure vector which does the same thing, except this way the packers are totally responsible for serialization. If you have thoughts on another approach I'm happy to modify this. Thanks for Sente,

ptaoussanis commented 2 years ago

@rosejn Thanks Jeff, that sounds promising! Would be very happy to see a PR. Just a heads-up that I'd only expect to review+merge next time I'm doing some batched work on Sente.

Cheers!

ptaoussanis commented 2 years ago

Closing since https://github.com/ptaoussanis/sente/pull/398 is now merged and available on Clojars as [com.taoensso/sente "1.17.0-RC2"] 👍