Feature Request: Message Encoding 'FILE'

[ISibboI]

It would be really cool if there were a message encoding type to transfer files, in simple binary format.

[Scopelabz]

What about doing it just like in emails? Something like this: This is a message! ---FILE=image/png,name=aninterstingpicture.png base64xbase64xxxxx== ---END I think this should be a bit easier to implement while not breaking the basic text messaging code.

[ISibboI]

It's no problem to use a binary format, since the message content is preceded with its length. And since we have to do proof of work, I guess the extra 1/3 of processing power should be saved. You idea isn't bad, but I think it is better to work on the protocol itself right now, while there's only one client. I'd suggest to add the type FILE as Message encoding type (https://bitmessage.org/wiki/Protocol_specification#Message_Encodings). It should look like the other types, but it should have a var_str named "header", containing information about the file type and name, and a char[] with variable length (var_int followed by uchar[]) for the content. That would fit to the protocol more I guess.

[Scopelabz]

Ah, I see now what you meant, thanks for explaining in more detail =) If I understand correctly the filemsg payload should look like this:

1+          header       var_str   File type and name
1+          content_len  var_int   The length of the file content
content_len content      uchar[]   The file contents in simple binary format

And would be inserted into the message field in both person-to-person and broadcast messages?

[Atheros1]

@ISibboI Yes, that would be more efficient. I have thought about doing exactly this. Perhaps I will next week; it would be a good reason to go to 0.2.8, people would like it and it would be easy to implement. I plan on having it optionally auto-save to a folder which would make it very useful for certain applications. I can't decide on how or if to handle multipart messages- messages that contain messages and 1 or more files. Maybe I'll just leave that for MIME later and support simple file transfer for now.

thinks for several minutes

Ok. So when signing up for a subscription, you will be given a checkbox: "If a file is broadcast by this address, save it to a particular folder.." and then a "Browse" button to select the folder. For person-to-person messages I suppose I have to decide on an interface to show it instead of an inbox message and let the user optionally save it.

[ISibboI]

@Scopelabz It would replace the message length and message, yes ;)

@Atheros1 Great! Multiple files or a message with appendix could be implemented using a message list. Just use a var_int containing the amount of following "message_type ..." patterns.

What I just thought about: Not everyone has a good internet connection. So obviously not every node is suitable to handle larger objects. It could be useful if we add a max_object_size field as var_int to the version message. This way, nodes can request not to be DOSed ;)

[ghost]

I am quite wary about adding images or other large content payloads to messages. If every client receives every message then surely that's going to severely reduce scalability. I'd rather not get into a bitcoin situation where messages.dat becomes a gigantic multi-gigabyte monster just so that I can occasionally send out a few tweet-like messages.

A possible compromise solution is that there are various behavior bits in the protocol. These might be configured to indicate whether the node will relay messages larger than a certain size. That way I would be happy to relay small messages but not have my bandwidth hogged by users sending giant files.

[ISibboI]

@fuzzgun The addition of a max_object_size field would be the way to go. I honestly don't like the idea of using the behavior bits for that, as that would limit the decision to certain fixed values. It would cost multiple behavior bits to give the node at least a little bit of choice.

[ghost]

Yes, a separate field would be better.

[Atheros1]

As originally envisioned, it is the POW that keeps users from sending large files and leaving everyone with a large messages.dat file. The target of the POW task (and thus the difficulty) changes basically linearly with the size of the message. This was done because the other solutions discussed thus far don't stop anyone from implementing their own client which sends large messages split between a bunch of smaller messages. The POW solution which is currently implemented Does discourage this because it doesn't matter whether you use one huge message or a bunch of smaller messages- the POW that you have to do is practically the same.

[ISibboI]

The POW solution is definitely a good idea, I just think about countries where people have a bad connection, 56k for example. They are not able to use bitmessage. Thus, specifying a maximum object size could allow these clients to operate by leaving out larger objects. Sure you can still attack them directly by ignoring that or by sending a bunch of smaller messages. But it might be a step in the right direction.

[r14c]

I think for the sake of symmetry it would be best to implement some kind of message chunking, that way you can send attachments without hurting clients with limited bandwidth. I could also see this improving the anonymity to some degree (if every bitmessage is the same size all traffic looks exactly the same)

[PeterSurda]

A more flexible encoding is planned for 0.8, it may not be necessarily exactly like the one described here but will allow the same goals. I will keep this open to compare when I'm implementing it.