[IDEA] Module for storing generic IOT event / message

[diggy128]

I think we should create a module to handle generic input FROM IOT devices and allow to store them to a separated model.

Rough Analysis: Up to now installing an IOT module gives you the possibility to handle IOT devices and allow methods to update data in Odoo. The thing here is there is so much data that can come from an IOT device that does not correlate to any model in Odoo or it does not make sense to, but useful conclusions can come from consolidating and analyzing that data. A very rough example is a simple magnetic switch in a door for a controlled environment room. Creating a new module to hold data from that single sensor is an overkill but having an analysis on when / how often the door is opened / closed can give useful insights.

So my proposal is to create a module eg iot.event that will have:

• datetimestamp
• date (calculated from datetimestamp)
• time (calculated from datetimestamp)
• device_id
• value

The separate date and time fields are to allow grouping and pivoting by date and time respectively. For the value field we can copy the definitions from base_custom_info in server-tools to handle all data types.

There will be a check box in device input to allow logging in the event table as well as a data type selection. A separate menu will be created along with tree, pivot and graph views to allow for some analysis.

As a side-note, using that data with mis_builder can create a multitude of KPIs Disclaimer: I am about to implement a module around the same concept (in-house) so if there is interest I would be happy to work with the community and push it to OCA.

[etobella]

Hi, I made a small module with the intention to handle the storage of a themperature sensor. Maybe it is useful to you https://github.com/tegin/iot/tree/11.0-add-iot_input_sensor . This is just an idea, so it is not necessary to take it :wink:

From my side, the idea is welcome and we can review it if you want to propose a module

[diggy128]

So, I wrote a module that implements the logic I mentioned above and tested with 3 different platforms: -Arduino -Arduino using MIT App Inventor application to post data -ESP8266 with Micropython -Serial data on COM port (using pyserial)

But I keep bumping on the limitations of the current controller implementation in iot_input.

A few examples:

• Each input has its own url, an iot device cannot send data from multiple sensors in 1 HTTP POST.
• Each input has a separate passphrase.
• Sending different range of data is cumbersome

I believe a rewrite of the call_unauthorized_iot is needed. @etobella Would you consider merging breaking changes?

[etobella]

I cannot promise a merge, but I will review for sure!

[JuanDCG]

Hi @diggy128 and @etobella 👍🏽 How about this work ?

I was thinking about integrate IoT on Odoo for 6 years ago. But tools and markets were immature yet, so I only theorized.

I want to contribute to this PSC/repository and this [IDEA] proposal is very good to start with it.

First, a solution for IoT should be break in little parts:

Some thoughts about IoT Odoo based solution:

• Storage should be NoSQL solution like MongoDB, Prometheus, or Elasticsearch...

  • Odoo OCA have Elasticsearch connector .

  • At this point could be a good approach build internet of things with the Elastic stack, but entirely Open Source with Open Distro for Elasticsearch, I suggest it.

  • In Elasticsearch you have timestamps... and key ( devide_id ) and value ( data ) structure, for IoT event streams. And with kibana you have all Queries dashboards...

• Data input:

  • To split solution and be able to work together I think if @diggy128 could you please simulate your input requirements to build like a mock, stub or fake for unit testing or some else....

  • I think that the better approach to do it, could be build a python script that call to your micropython code on any device, but aligning with Odoo IoT Box, that is only a Raspberry Pi with Odoo custom image on SD, but extensible anyway...

• In this way, I be able to testing without my old raspberry pi or microarduino and try to work focusing on the integration with Elasticsearch and Odoo.

What are you thinking about all ?

[etobella]

Hi @JuanDCG , The proposal of @diggy128 was merged on #21

About your ideas:

• Using a NoSQL for storage might be complicated, as the connectors you were checking are not currently using elasticsearch data, just sending, It might be interesting to use other systems, but it has to be done from draft
• Data Input: I made a class to implement any kind of integration with Iot modules.
  • https://github.com/tegin/oot: Basic Class
  • https://github.com/tegin/oddoor: Implementation for a Door System
  • https://github.com/tegin/oddoor-launcher: System used on a Raspberry with an RFID reader, keyboard, buzzer and a Relay
  • https://github.com/tegin/ras/tree/ras_cb: Implementation of Forgeflow RAS using the oot package

I don't have a video right now of our door, but I can try to make one if you are interested :wink:

[diggy128]

Actually @etobella PR #21 just sets the base for the module I was thinking of. I've put together a module cleverly named iot_event 😃. I haven't pushed it yet because it lacks tests. It's been in production for more than 2 months now.

If you want I can create a PR and discuss over it's implementation

[etobella]

Sure!

[JuanDCG]

I agree,

You have done a hard work that cannot be read in any documentation so I would need more time to read the code ... Although to test it locally I need more space, resources and time ... Maybe I can document your work while I learn.

Thank you !

[JuanDCG]

At the moment, I have a lot to learn about what you have built, but if I can help stimulate conversations in the community and generate new ideas or approaches, great.

So, since I thought the project was less mature, I had thought about it from scratch.

[JuanDCG]

For example, a scalable case study, imagine that an alarm company in electronic door locks wants to automate its business flow by innovating in cutting-edge technologies in IoT but maintaining its supply and maintenance chain with Odoo CE.

They want to gradually implement electronic locks in a few clients while automating the entire business and technological flow... Then, to be able to scale to millions of automated doors

How would you scale the database? PostgreSQL? How would you ingest so much traffic? Apache Kafka? How would the security and events of the doors be analyzed? Apache Spark?

And so on and so for

[JuanDCG]

@OCA/iot-maintainers

For better discussion, sorry.

[diggy128]

@OCA/iot-maintainers @etobella @JuanDCG Created PR #24 with new module. It is still WIP and needs some polishing before tests but pushed it to continue discussion.

[github-actions[bot]]

There hasn't been any activity on this issue in the past 6 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days. If you want this issue to never become stale, please ask a PSC member to apply the "no stale" label.