Split long elements

[felipedau]

Once #57 and #58 are implemented, long elements will have to be split into multiple element packets. Most of this has already been done:

• Element packets have the ID of the element they are part of, the number of the part and the total number of parts
• When receiving a packet, a PartialElement is created if it did not exist already and the contents of that packet is added to its respective part in the PartialElement's dictionary
• PartialElement.is_complete tells if all parts were received, __str__() joins them and to_element() returns a complete and deserialized Element
• PartialElement.from_element() receives an element and its ID, serializes it and returns a PartialElement with a single part with the whole element
• PartialElement.to_packets() builds and returns ElementPackets containing each part
• PartialElement.from_element() splits the serialized element based on the max_len argument

[felipedau]

89 implements this for the current state of the packet format.

However, I am going to leave this open because I am not happy with the solution. We depend on #57 to define the exact sizes of each part of a packet and send only bytes. from_element() shouldn't expect the data to increase 34% due to the base64 encoding.

What has to be done:

• Once #57 is implemented, see if we can estimate the final size of elements and packets with the .*_LEN constants in peer.py or if we need to actually calculate

• Modify from_element() based on the decision above

Future work:

• Creating the parts of a partial element from an element is using a lot of CPU for long elements

• In general I think that keeping everything in memory is good, but then we use a lot of resources when dealing with long elements

• Assigning just the payloads to the partial element and then creating the element packets from them may not be doing a good use of memory - maybe the packets should be assigned to the partial element right away