ReubenBond
commented
3 years ago Hi @staviloglu, Orleans is indeed a good match for IoT and has been put to use in many IoT applications, including in the backend of Azure IoT Digital Twins. We typically do not provide much input into overall application architecture - maybe we should, but it's time-intensive and ambiguous, being more suitable for discussion than Q&A. Regardless, please accept my apology for not responding sooner, as unfulfilling as this response may be.
I would generally use an IHostedService (eg BackgroundService) over aGrainService`. Using gRPC between services is a fine approach. Orleans streams are fine, too, and give you some benefits such as increased decoupling over gRPC (given gRPC is point-to-point and temporally coupled). Interfacing with RabbitMQ directly gives you more control, so perhaps that's suitable if your scenario doesn't fit into the virtual streams / firehose approach that Orleans Streams takes.
If anyone has opinions here, please feel free to chime in. I'll convert this to a discussion, since I think it's more suited for that
Hi.
First of all thanks for the Orleans framework. I appreciate your efforts to make distributed application development a lot easier.
Orleans seems to be an exact match for IoT business where we need device twins on backend.
But unfortunately I can not seem to find a suitable example rather than the old GPS Tracker example which is working with an HTTP Frontend.
I was wondering if you could point me in the right direction and your thoughts on the following architectures that come to my mind.
I will use RabbitMQ and its MQTT Plugin to support MQTT connections from the devices on the field. They will periodically publish sensor data to given topics.
1- Each grain can create its own connection + channel and subscribe/publish to related queue on RabbitMQ. This does not seem to be a reasonable solution to me, because each device already makes connections if the device twins do that too we double the connection count which will lead to performance loss on RabbitMQ.
2- Each silo will have 2 connections (one for publish and one for subscribe) but each grain will create their own channels using the shared connections. This solution is ok for connection count but increase in channel count creates a lot less but almost the same effect on RabbitMQ. And how should I share the connections (1 stateless worker, IHostedService, GrainService?) Pooling connections come to my mind but first need to be sure.
3- I may create different processes for subscribing and publishing, each create their own connections and subscribe/publish to RabbitMQ. When a message arrived I can use GrainClient to send data to Orleans cluster. That seems totally fine to me, this way I can scale subscription process seperate from the silos. But I'm not sure how can I publish from grains to RabbitMQ which requires orleans to communicate with the publisher process. (gRPC etc? Orleans streams?)
4- I may have the subscriber process mentioned above but rather than a seperate publisher process, each grain can create their connection + channel and publish the data when required. And close the connection + cannel. Or share one connection per silo but seperate channel opened/closed in each grain.
5- Stream Providers? I have seen some documentation about pulling agents but I want to use RabbitMQ push based. I have seen Stream provider for RabbitMQ bu I think that does not solve the problem here. Am I wrong?
Tihs is might be too much to ask and time consuming to read. :(
Hope to see an answer from your side.
Thanks for your time.