cc-tweaked / CC-Tweaked

Just another ComputerCraft fork
https://tweaked.cc
940 stars 211 forks source link

[Suggestion] Add base64 built in #1960

Open Soapy7261 opened 2 months ago

Soapy7261 commented 2 months ago

Useful information to include:

SquidDev commented 2 months ago

Things like web socket sending require encoding binary data as Base64

Not sure if you're aware but websockets should support sending binary messages.

I have considered using textutils.urlEncode(), but it has a bug(?) where if the string is too big, it never yields. [...] There are LUA packages that do this, but they are quite slow for large amounts of binary data, it would be nice if it was included (using a java implementation ideally so it could be done faster)

textutils.urlEncode is probably not the right tool here, but I am curious how much data you're trying to send here? Looking at the implementation, there's some obvious optimisations there which I'll make, but it should be fine for anything less than a megabyte.

I generally prefer putting this stuff in Lua where possible, so would like to understand what performance expectations you have.

Soapy7261 commented 2 months ago

Not sure if you're aware but websockets should support sending binary messages.

Well, I guess there goes 3 hours of my life I'll never get back.

textutils.urlEncode is probably not the right tool here, but I am curious how much data you're trying to send here? Looking at the implementation, there's some obvious optimisations there which I'll make, but it should be fine for anything less than a megabyte.

I'll admit I was trying to encode a 150MB file, but the limit before that happens is probably quite a bit lower, I'll do more testing when I have more time

I generally prefer putting this stuff in Lua where possible, so would like to understand what performance expectations you have.

That's understandable, even if it was in LUA, it would still be nice to have included

Soapy7261 commented 2 months ago

image Also, just looking at the documentation 'Whether this message should be treated as a', then it cuts off, i know its probably 'Whether this message should be treated as a binary message' but still

Soapy7261 commented 2 months ago

Also, kind of unrelated but textutils.serialiseJSON() does not seem to like binary data, the chunk_data is exactly 131036 big image Yet somehow it grows to 535823? Didn't do this when i gave it base64 data image

Soapy7261 commented 2 months ago

image Seems to be converting the binary data to string, although that actually makes sense now that i think about it, i should probably not use JSON, although using base64 is probably easier (and faster) then re-designing my entire system