search

algorealmInc / SwarmNL

SwarmNL is a simple configurable library designed for P2P networking in distributed systems.
https://algorealminc.github.io/SwarmNL/swarm_nl/index.html
Apache License 2.0
17 stars 2 forks source link

Feature request: File transfer and data streaming #38

Open kap2fox opened 4 months ago

kap2fox commented 4 months ago

We have a scenario with data transfer in local network. And we need two types of data transferring - the whole file with guarantee delivery; the data streaming when we don't care about some messages. We'd like to have functiron like _node.senddata(to: PeerId,data: bytes)->Result// send data to another node

and

_node.wait_for_data()->Result<DataFromAnotherNode{ // accept data from another nodes originfrom: PeerId, data: bytes}>

thewoodfish commented 4 months ago

Please can you elaborate on the "data streaming" part?

thewoodfish commented 4 months ago

For the file transfer, there is a maximum amount of bytes that can be sent via RPC or gossiped. If the file is larger, it will most likely be broken to fit and then reassembled on the receiving end. Also, there has to be some mechanism to ensure the file is correct, maybe after transfer of each byte chuck a hash can be sent back to confirm the authenticity and completeness of the data sent. That should work

kap2fox commented 3 months ago

Please can you elaborate on the "data streaming" part?

I'm talking about data sequences delivering without guaranteed of integrity. The part of the data may be lost but important to get it periodically for monitoring task.

thewoodfish commented 3 months ago

Yes, Swarm makes provisions for that too. You can even multicast if you desire.