Closed maxchu765 closed 3 years ago
This is almost certainly due to the limits on packet size imposed by Radiohead's (the library we use for LoRa) RF95 implementation which maxes out at 251 bytes. Some possible solutions include:
For the time being, we are going to have to recommend that users do their best to check that the data they are shipping out is the minimum viable if packet size is proving to be the limiting factor while our team figures out the best path forward and works to provide a fix.
This problem will be addressed specifically in SitkaNet code by using an external process to compress the JSON into the minimum values needed before transmission. I created a pull request that allows the sketch to send/receive raw payloads for this purpose (https://github.com/OPEnSLab-OSU/Loom/pull/73), and will be working with @maxchu765 on SitkaNet individually. This will not address this problem in the general case, and a general solution is something Loom will need to consider in the future.
Currently, there is an example code called Transmit_Raw and Receive_Blocking_Raw in the LoRa example that is based on the JSON translation function for all users for different projects. You can find this code in the SerialSensor Branch for now. We need to test it before we close the issue. It should be good. There are many comments that guide what to do so that people won't get lost with it. If there any help, they are going to make an issue on the GitHub page.
Updates! The main issue has been solved. Therefore, you don't need to follow these code examples to do large package radio communication. However, these examples will not be deleted because it is a good alternative way to solve if there is an issue in this method that I created.
It will now able to send the larger package size over Radio communication in the Commplat branch. Currently, there are some parts that need to be added into the code to make the performance better such as two nodes into one hub sending with larger package size or with batch sending, but the main problem of sending a larger package will be available. The maximum size is now can send up to 2048 bytes rather than 251 bytes. This will be updated on master in the next coming month.
Updates for the changes that will happen for this issue and solve the main problem.
Current Situation
If you are planning to send a large size package over LoRa or any kind of radio communication, please use this branch rather than the master branch because this will solve this current issue.
Future plans
Developer Notes
mergeJSON
, but make more like sendJSON
and receiceJSON
so that it will remove waste. I hope this will work so that it will be simple to fix.If you have any ideas on how to answer these questions, please let me know! That would be great!
Updates In the last developer notes, I accomplish for finding the solution and fixing it properly for the first one. However, the second one will not be fixed for now. And here is why. While COM7 is sending, COM10 is receiving. You can see a comparison between the two that the message pack size is different when you send verses when you receive by 30 bytes even though they are the same message data. I believe that it happens it increases during radio communication. Because of that, I can't change that part at all. Therefore, I will let the user know that they need to reduce the size of the package, but it has increased from 251 bytes up to 2000 bytes of radio communication, so I believe that it is better in what way. This will be as it is for now, but once we find a new way, this issue will be reopened. You can find the code in Commplat until September 25th, then it will be in master.
Developer Notes If anyone has an idea where to fix it, please open the issue and send a comment here. I will try to look at it and discuss it together if that will work or not.
If you take a look at here, even with the general send and receive for the basic example of Loom shows a difference between JSON size that the receiving end gains more memory usage. We need to know what is going on between each radio communication.
The problem occurs when you measure the size of the received msgpack packet using this strlen call. Strlen is designed to iterate over a string until a null terminator (0x00
) is found—a msgpack payload is not a string, however, and as a result it is perfectly valid behavior for a msgpack payload to have 0x00
values all over the place. Because of this, attempting to measure the msgpack with strlen sometimes results in the correct value, but usually results in a value significantly less than the actual length. This problem is demonstrated below using msgpack.org:
To fix this issue, you'll need to find an alternate method of measuring the size of the received msgpack. I would suggest deserializing it and then measuring the newly deserialized payload, but I'm sure there are other options.
After disgusting with other team members, we decided to let the end-user to limit the size of the JSON package sending and receiving. However, as I said before, the size has improved than before this was integrated. This will be merged into master soon!
After disgusting with other team members, we decided to let the end-user to limit the size of the JSON package sending and receiving. However, as I said before, the size has improved than before this was integrated. This will be merged into master soon!
This is now in the master branch,
Describe the bug LoRa communication will fail when trying to transmit data from more than 3 sensors. The packet is too large for the receive to parse.
Hardware in Use Feather M0 LoRa Hypnos Board I2C Multiplexer
I2C and Teros sensors
To Reproduce Transmit data with 3 or more I2C sensors over LoRa. The receiver will show a "Failed to parse MsgPack" error.
Code https://github.com/OPEnSLab-OSU/SitkaNet-2/blob/master/Node/Node.ino
Config https://github.com/OPEnSLab-OSU/SitkaNet-2/blob/master/Node/config.h