search

transient-haskell / transient

A full stack, reactive architecture for general purpose programming. Algebraic and monadically composable primitives for concurrency, parallelism, event handling, transactions, multithreading, Web, and distributed computing with complete de-inversion of control (No callbacks, no blocking, pure state)
MIT License
630 stars 28 forks source link

Different underlying transport #74

Open CMCDragonkai opened 6 years ago

CMCDragonkai commented 6 years ago

We're working on porting libp2p to haskell under (github.com/MatrixAI). But I'm very intrigued by your project here. While libp2p is just a sort of communication system (although a very sophisticated one at that), it lacks any higher level of composability. I'm wondering how compatible is transient with different underlying communication primitives? I haven't grokked transient yet, but libp2p sort of exposes bytes-streams that are the transparently mapped to other nodes in the network.

There are notions of addressing services that supply bytestreams or receive bytestreams in libp2p however.

agocorona commented 6 years ago

Hi Roger:

transient now support sockets and websockets. protocols are not pluggable but almost:

https://github.com/transient-haskell/transient-universe/blob/master/src/Transient/Move/Internals.hs#L884-L889

 (conn,parseContext) <- checkSelf node

<|> timeout 1000000 (connectNode2Node host port) <|> timeout 1000000 (connectWebSockets host port) <|> checkRelay <|> (throwt $ ConnectionError "" node)

This is how two nodes connect: if it is not the same node, it tries sockets, then , websockets, and then looks for a node that may work as relay between the two.

So if you insert another protocol here, that will be the whole of it. "conn" is basically the socket connection and "parseContext" contains either the IO operation that read from the socket or stream or a lazy bytestring stream.

ParseContext is ok, but I'm afraid that "conn" is not well defined for allowing pluggable communications, since the details of the connection are exposed, but that can be improved.

Also the node concept, which is basically a host, a port and a map String String that specify attributes of the node may or may not be general enough for your case....

But anyway once that is integrated, you can use the higher level transient primitives over it.

I'm planning to port WebRT to perfomr direct transient communications between Web browsers but I had no time yet, so abstracting out and making the above code more pluggable goes also in my interest.

Are you developing MatrixAI for some special purpose?

2018-04-25 10:10 GMT+02:00 Roger Qiu notifications@github.com:

We're working on porting libp2p to haskell under (github.com/MatrixAI). But I'm very intrigued by your project here. While libp2p is just a sort of communication system (although a very sophisticated one at that), it lacks any higher level of composability. I'm wondering how compatible is transient with different underlying communication primitives? I haven't grokked transient yet, but libp2p sort of exposes bytes-streams that are the transparently mapped to other nodes in the network.

There are notions of addressing services that supply bytestreams or receive bytestreams in libp2p however.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/transient-haskell/transient/issues/74, or mute the thread https://github.com/notifications/unsubscribe-auth/AAf5isQ6dWdDs6Z6ou5NARu-0EikQGSvks5tsC-EgaJpZM4Ti9tt .

-- Alberto.

agocorona commented 6 years ago

Oh, I see that it is a protocol used by IPFS. I'm very interested in IPFS. Let me know if I can help you

2018-04-25 23:05 GMT+02:00 Alberto G. Corona agocorona@gmail.com:

Hi Roger:

transient now support sockets and websockets. protocols are not pluggable but almost:

https://github.com/transient-haskell/transient-universe/ blob/master/src/Transient/Move/Internals.hs#L884-L889

 (conn,parseContext) <- checkSelf node

<|> timeout 1000000 (connectNode2Node host port) <|> timeout 1000000 (connectWebSockets host port) <|> checkRelay <|> (throwt $ ConnectionError "" node)

This is how two nodes connect: if it is not the same node, it tries sockets, then , websockets, and then looks for a node that may work as relay between the two.

So if you insert another protocol here, that will be the whole of it. "conn" is basically the socket connection and "parseContext" contains either the IO operation that read from the socket or stream or a lazy bytestring stream.

ParseContext is ok, but I'm afraid that "conn" is not well defined for allowing pluggable communications, since the details of the connection are exposed, but that can be improved.

Also the node concept, which is basically a host, a port and a map String String that specify attributes of the node may or may not be general enough for your case....

But anyway once that is integrated, you can use the higher level transient primitives over it.

I'm planning to port WebRT to perfomr direct transient communications between Web browsers but I had no time yet, so abstracting out and making the above code more pluggable goes also in my interest.

Are you developing MatrixAI for some special purpose?

2018-04-25 10:10 GMT+02:00 Roger Qiu notifications@github.com:

We're working on porting libp2p to haskell under (github.com/MatrixAI). But I'm very intrigued by your project here. While libp2p is just a sort of communication system (although a very sophisticated one at that), it lacks any higher level of composability. I'm wondering how compatible is transient with different underlying communication primitives? I haven't grokked transient yet, but libp2p sort of exposes bytes-streams that are the transparently mapped to other nodes in the network.

There are notions of addressing services that supply bytestreams or receive bytestreams in libp2p however.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/transient-haskell/transient/issues/74, or mute the thread https://github.com/notifications/unsubscribe-auth/AAf5isQ6dWdDs6Z6ou5NARu-0EikQGSvks5tsC-EgaJpZM4Ti9tt .

-- Alberto.

-- Alberto.

CMCDragonkai commented 6 years ago

Nodes in libp2p are abstracted from the host+port concept. Since the goal of libp2p was to abstract across all possible underlying transports. And rathet than following OSI's layering model, they decompose network features horizontally. The only major user of libp2p is ipfs, but it can also be used for other services. In libp2p, there's peers (nodes), services and streams.

We want to see how to use libp2p and transient for the coordination system for Matrix AI agents.

agocorona commented 6 years ago

Roger

For the nodes, it is not strictly necessary to user host-port. For example, nodes that represent browsers connected, the nodes have "webnode" as the (fake) hostname in transient.

Ok, let me know If I can help you in some way.

2018-04-26 2:24 GMT+02:00 Roger Qiu notifications@github.com:

Nodes in libp2p are abstracted from the host+port concept. Since the goal of libp2p was to abstract across all possible underlying transports. And rathet than following OSI's layering model, they decompose network features horizontally. The only major user of libp2p is ipfs, but it can also be used for other services. In libp2p, there's peers (nodes), services and streams.

We want to see how to use libp2p and transient for the coordination system for Matrix AI agents.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/transient-haskell/transient/issues/74#issuecomment-384474032, or mute the thread https://github.com/notifications/unsubscribe-auth/AAf5ikyr1JQitSQzwSM6SI3ZPRKZUw7lks5tsRO5gaJpZM4Ti9tt .

-- Alberto.