search

ODM2 / ODM2DataSharingPortal

A Python-Django web application enabling users to upload, share, and display data from their environmental monitoring sites via the app's ODM2 database. Data can either be automatically streamed from Internet of Things (IoT) devices, manually uploaded via CSV files, or manually entered into forms.
BSD 3-Clause "New" or "Revised" License
30 stars 8 forks source link

Consider CoAP over UDP instead to reduce power/data use on cellular networks #95

Open SRGDamia1 opened 6 years ago

SRGDamia1 commented 6 years ago

CoAP has similar functionality to HTTP, but is over UDP which has much lower overhead than TCP/IP.

https://en.wikipedia.org/wiki/Constrained_Application_Protocol

horsburgh commented 6 years ago

@SRGDamia1 - I'm going to backburner this one. We have a working prototype for the data posting service. See my comments on #11. We can revisit later if we have time/resources.

Also - my University tends to watch UDP traffic on the network pretty closely, so there might be some implications for hosting a service that uses UDP. Don't know much about CoAP and so there would be a lot of learning/overhead there.

aufdenkampe commented 6 years ago

When we revive this issue, we may want to consider Message Queuing Telemetry Transport (MQTT), which "works on top of the TCP/IP protocol", and is "designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited".

This approach is being adopted by some IoT systems, and wouldn't conflict with USU's policies regarding UDP.

SRGDamia1 commented 5 years ago

@aufdenkampe - are you planning on adding both MQTT and CoAP?

aufdenkampe commented 5 years ago

No. Probably just MQTT. We still need to evaluate these and move some or all out of milestone 0.11.

What are your thoughts?

SRGDamia1 commented 5 years ago

From what I've read CoAP has lower data consumption and is essentially designed to be a drop-in replacement for a HTTP REST service but implemented over UDP. So it's best for posting data and not looking for a response. MQTT seems to be preferred when there's any two way communication, but we're not doing that.

This is one of the best tests I've seen comparing MQTT and CoAP: https://www.researchgate.net/publication/267636202_Performance_evaluation_of_MQTT_and_CoAP_via_a_common_middleware

Aside from the data use, though, doing MQTT would require almost no Arduino coding. ModularSensors can already successfully send MQTT messages to ThingSpeak. If we send a similar (or identical) json object over MQTT that we're currently using over REST, it would be pretty easy to switch over.

TinyGSM doesn't support UDP, though, so that would have to be written to use CoAP. I don't think that's an enormous hurdle; most of the modems I've looked at only need to change one or two lines in the connection process to switch to UDP. But that's still a set of code that we'd need to write and test before we could use CoAP and not for MQTT.

I have no idea how much coding would have to be written on the server side to use either one. For MQTT I think you'd have to either write your own broker or piggy back on a public one. I don't think you'd need to do anything like that for CoAP.

aufdenkampe commented 5 years ago

@SRGDamia1, thanks for those additional considerations! That will all be very helpful for us as we evaluate options moving forward.