search

timmbogner / Farm-Data-Relay-System

A system that uses ESP-NOW, LoRa, and other protocols to transport sensor data in remote areas without relying on WiFi.
MIT License
507 stars 114 forks source link

One GateWay To Rule Them All #20

Closed Devilbinder closed 2 years ago

Devilbinder commented 2 years ago

No idea if this will work since I do not have hardware. So test first.

Reworked all the Gateways into classes.

Added Universal_Gateway example that is compatible with this PR.

timmbogner commented 2 years ago

Cool cool cool! I'm going to have time to sit down and work with all of this next Mon-Wed. Again, I really appreciate all of the work you're doing.

Devilbinder commented 2 years ago

Shot if you got any bug please let me know.

timmbogner commented 2 years ago

I'm working with it now, but I think either some of the functionality is lost or I'm misunderstanding your methods. For example: the most complicated but essential function of a gateway is to work as a repeater. You can set it to listen for packets from a specific device address 'A', then forward them to another specific address 'B'. At the same time, it could be set to take any packets from 'B' and forward them to 'A'. At the same time as that it could be sending unknown ESP-NOW packets to device 'B' or even a completely different device 'C'. While all this is happening, LoRa might be set to be doing similar things to different addresses. Since LoRa is a bottleneck, instead of sending instantly it has to go into a buffer to be released periodically. Would you be able to accomplish all of this with the new design?

timmbogner commented 2 years ago

@Devilbinder Just realized I didn't tag you. Also, is this the proper place for this discussion?

Devilbinder commented 2 years ago

What functionality disappeared? For I am blind and can not see.

How the code is currently working in theory.

The incoming data is governed by the _data variable across all class types. https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L8

This functions adds the data. https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L74

Any incoming data is collected in there respective receive function and added to _data.

Receive ESP https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L15

Receive MQTT https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L250

Receive Serial https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L355

Receive LoRa https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L458

The collected data then gets send via the release function that calls that send function of the respective class. The send then loops though the _data variable and transforms the data into that classes send format.

Send ESP https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L209

Send MQTT https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L319

Send Serial https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L389

Send LoRa https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L504

All that then has to be deleted once it is sent via its respective protocol. Otherwise it will cause memory leak. This is done with the flush function. https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L71 This will delete all data in the buffer regardless of what class type calls it.

Part 2 incoming Yes this is the right place. You do not need to @ me. Since I am listed as a participant, Github sends me a mail. Don't know how true this is for other users but that is my settings.

Devilbinder commented 2 years ago

For the ESP code. I am adding in coming data from unknown EPSs

This checks if the peer is know or not. if it is not know, add it to the unknown peer list. https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L28

When release gets called The unknow peers get added to the peer list https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L215

The unknown peers then get removed after sending data. https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L228

The unknown peers then get deleted to free up some memory.

Devilbinder commented 2 years ago

Explaining the example

Witch ever class type you crate determines where is can send and receive data.

Let say you create a ESP and LoRa object. https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/Examples/Universal_Gateway/Universal_Gateway.ino#L51

https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/Examples/Universal_Gateway/Universal_Gateway.ino#L71

The gateway is now capable of send and receiving data on ESP or LoRa protocols. It is also capable of only sending via LoRa or ESP. Did not look at disabling receiving for the ESP and MQTT protocol.

Collection of the data is done for LoRa with the get function. (I am assuming this is where the issue is) https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/Examples/Universal_Gateway/Universal_Gateway.ino#L121

I can add a soft delay like this to the get. https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/Examples/3_ESPNOW_Gateway/3_ESPNOW_Gateway.ino#L111

The data can then be sent via its respective protocol calling the release function. Only send data via ESP https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/Examples/Universal_Gateway/Universal_Gateway.ino#L129

Only send data via LoRa. https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/Examples/Universal_Gateway/Universal_Gateway.ino#L141

The saved incoming data is then deleted via the flush function https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/Examples/Universal_Gateway/Universal_Gateway.ino#L147

Issues that I have seen with current version. The serial is block. If the serial line breaks it will just hang waiting for a \n https://github.com/timmbogner/Farm-Data-Relay-System/blob/6371f9447b84d02eec556152a5a0a4231875907a/fdrs_gateway.cpp#L358

When the data is received in a sterilized format there is no why to determine if it is the start or a end of frame. So you can get partial data in and corrupting the data and making it fall over like a tipsy elephant. The ESP and MQTT protocols cover it to a degree but not fully. Serial has it partially with with the '\n' but like I said it is using a blocking function call. LoRa only has the MAC at the start of the frame and I can tell you now it is not reliable.

Devilbinder commented 2 years ago

Hopefully this clarifies the logic behind the code.

timmbogner commented 2 years ago

Your patience is much appreciated. I'm still working on fitting my brain around a lot of new C++ concepts, so my code comprehension is not so great. How would you configure a gateway to only deliver packets from ESPNOW_PEER1 to ESPNOW_PEER2? Then adding to that, how would you tell it to route all other (unknown) packets to PEER1?

Devilbinder commented 2 years ago

Oh get you. That's not in at the moment. I seem to have misunderstood what the code was doing. You are manually setting up the broadcast route A -> B -> C. I am going to make a PR fixing it.