Closed ClementBoesmier closed 4 months ago
timmbogner
commented
5 months ago Hey Clément, I'm thinking and hoping that you can fix this issue by updating your ArduinoJSON library to the most current version (v7.x) and trying again. Let me know how it goes!
ClementBoesmier
commented
5 months ago Okay, thanks for your help. I have updated the lib and the gateway work now !
but I have another problem: when I update the sensor, the gateway no longer receives the information.
i've updated the config file on the 2 devices, keeping my settings.
What's the purpose of the time part in the latest versions? Could the problem come from there?
my sensor config
#define READING_ID 10 //Unique ID for this sensor
#define GTWY_MAC 0x01 //Address of the nearest gateway
// #define USE_ESPNOW
#define USE_LORA
#define DEEP_SLEEP
//#define POWER_CTRL 14
#define FDRS_DEBUG
#define DBG_LEVEL 2 // 0 for minimal messaging, 1 for troubleshooting, 2 for development
// I2C - OLED or RTC
#define I2C_SDA 5
#define I2C_SCL 6
// OLED -- Displays console debugging messages on an SSD1306 I²C OLED
// #define USE_OLED
#define OLED_HEADER "FDRS"
#define OLED_PAGE_SECS 30
#define OLED_RST -1
// LoRa Configuration
#define RADIOLIB_MODULE SX1276
#define LORA_SS 18
#define LORA_RST 23
#define LORA_DIO 26
#define LORA_BUSY 33
//#define USE_SX126X
#define LORA_TXPWR 17 // LoRa TX power in dBm (: +2dBm - +17dBm (for SX1276-7) +20dBm (for SX1278))
//#define LORA_ACK // Request LoRa acknowledgment.
#define LORA_FREQUENCY 868
#define FDRS_LORA_SYNCWORD 0x12
#define CUSTOM_SPI
#define LORA_SPI_SCK 5
#define LORA_SPI_MISO 19
#define LORA_SPI_MOSI 27
// Time settings
// #define USDST
#define EUDST
#define STD_OFFSET (-6) // Local standard time offset in hours from UTC - if unsure, check https://time.is
#define DST_OFFSET (STD_OFFSET + 1) // Local savings time offset in hours from UTC - if unsure, check https://time.is
#define TIME_PRINTTIME 15 // Time, in minutes, between printing local time to debugOne initial discrepancy that I see is that your gateway config uses the default value for LORA_RST, while the sensor config shown has it changed to 23. Are the sensors also using the Heltec v2 board? If so, that value should be set to the default 14.
What does the serial output of your sensor look like when it attempts to send data via LoRa?
This shouldn't affect this issue, but you should not set FDRS_LORA_SYNCWORD in your config file, as this value (with the FDRS_ prefix) is only supposed to be used by the system. The correct way is for you to set either GLOBAL_LORA_SYNCWORD in the globals.h file, or LORA_SYNCWORD in your local config file. FDRS_LORA_SYNCWORD is then set behind-the-scenes to one of those two.
In your case though, it's not changed from 12, so it's not a problem. If you changed it, the system would end up overwriting it back to the global value of 12, fyi.
Don't try changing it though, anyway. The effects aren't well-documented and not understood (to my research) 😅
The timekeeping changes make it possible to share the current time between all devices that connect to an FDRS gateway. We'll have to exhaust a couple more troubleshooting steps before knowing if it's related. Serial logs for both the sensor and gateway will be the most helpful. I see you've already set your DBG_LEVEL to 2, which is perfect.
ClementBoesmier
commented
5 months ago Hello I apologize for my late reply, but I've got lots of other projects on the go at the same time.
I use a LILYGO lora card (with the battery charger) which explains the change. But, as I said, the code works with an older version of your library, so that's not the problem.
log from gateway :
12:06:02.770 > ets Jun 8 2016 00:22:57
12:06:02.771 >
12:06:02.771 > rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
12:06:02.771 > configsip: 0, SPIWP:0xee
12:06:02.771 > clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
12:06:02.771 > mode:DIO, clock div:2
12:06:02.771 > load:0x3fff0030,len:1184
12:06:02.771 > load:0x40078000,len:12776
12:06:02.774 > load:0x40080400,len:3032
12:06:02.774 > entry 0x400805e4
12:06:02.975 >
12:06:02.975 > Initializing FDRS Gateway!
12:06:02.975 > Address: 1
12:06:02.975 > [1] LORA_NEIGHBOR_1: 0
12:06:02.975 > [1] LORA_NEIGHBOR_2: 3
12:06:02.975 > Debugging verbosity level: 2
12:06:02.989 > RadioLib initialization successful!
12:06:02.989 > LoRa Initialized. Frequency: 868 Bandwidth: 125.00 SF: 7 CR: 5 SyncWord: 18 Tx Power: 17dBm
12:06:03.991 > [2] json parse err
12:06:03.991 > [2] ␀log from sensor :
12:20:50.175 > ets Jul 29 2019 12:21:46
12:20:50.187 >
12:20:50.187 > rst:0x5 (DEEPSLEEP_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
12:20:50.187 > configsip: 0, SPIWP:0xee
12:20:50.187 > clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
12:20:50.198 > mode:DIO, clock div:2
12:20:50.198 > load:0x3fff0030,len:1184
12:20:50.198 > load:0x40078000,len:12776
12:20:50.202 > load:0x40080400,len:3032
12:20:50.202 > entry 0x400805e4
12:20:50.364 >
12:20:50.375 > Initializing FDRS Node!
12:20:50.375 > Reading ID 10
12:20:50.375 > Gateway: 1
12:20:50.375 > Debugging verbosity level: 2
12:20:50.375 > [1] LoRa is enabled.
12:20:50.375 > [1] Deep sleep is enabled.
12:20:50.388 > RadioLib initialization successful!
12:20:50.399 > LoRa Initialized. Frequency: 868 Bandwidth: 125.00 SF: 7 CR: 5 SyncWord: 18 Tx Power: 17dBm
12:20:50.432 > LoRa node address is 0x789b
12:20:50.432 > DHTxx Sketch!
12:20:50.432 > Id: 10 - Type: 3 - Data loaded: 66.00
12:20:50.432 > Id: 10 - Type: 1 - Data loaded: 23.00
12:20:50.441 > Sending FDRS Packet!
12:20:50.441 > [2] DR added to LoRa buffer. start: 0 end: 2
12:20:50.441 > Deep sleeping.gateway config :
#define UNIT_MAC 0x01 // The address of this gateway
#define ESPNOW_NEIGHBOR_1 0x00 // Address of ESP-NOW neighbor #1
#define ESPNOW_NEIGHBOR_2 0x02 // Address of ESP-NOW neighbor #2
#define LORA_NEIGHBOR_1 0x00 // Address of LoRa neighbor #1
#define LORA_NEIGHBOR_2 0x03 // Address of LoRa neighbor #2
// Interfaces
// #define USE_ESPNOW
#define USE_LORA
//#define USE_WIFI // Will cause errors if used with ESP-NOW. Use a serial link instead!
//#define USE_ETHERNET
// Routing
// Options: sendESPNowNbr(1 or 2); sendESPNowPeers(); sendLoRaNbr(1 or 2); broadcastLoRa(); sendSerial(); sendMQTT();
#define ESPNOWG_ACT sendSerial();
#define LORAG_ACT sendSerial();
#define SERIAL_ACT sendESPNowNbr(2); sendESPNowPeers(); sendLoRaNbr(2); broadcastLoRa();
#define MQTT_ACT sendSerial();
#define INTERNAL_ACT sendSerial();
#define ESPNOW1_ACT sendSerial();
#define ESPNOW2_ACT sendSerial();
#define LORA1_ACT sendSerial();
#define LORA2_ACT sendSerial();
// LoRa Configuration
#define RADIOLIB_MODULE SX1276
#define LORA_SS 18
#define LORA_RST 14
#define LORA_DIO 26
// #define LORA_BUSY 33
//#define USE_SX126X
#define LORA_TXPWR 17 // LoRa TX power in dBm (: +2dBm - +17dBm (for SX1276-7) +20dBm (for SX1278))
#define LORA_FREQUENCY 868
#define CUSTOM_SPI
#define LORA_SPI_SCK 5
#define LORA_SPI_MISO 19
#define LORA_SPI_MOSI 27
#define FDRS_DEBUG // Enable USB-Serial debugging
#define DBG_LEVEL 2 // 0 for minimal messaging, 1 for troubleshooting, 2 for development
// I2C - OLED or RTC
#define I2C_SDA 4
#define I2C_SCL 15
// OLED -- Displays console debugging messages on an SSD1306 I²C OLED
///#define USE_OLED
#define OLED_HEADER "FDRS"
#define OLED_PAGE_SECS 30
#define OLED_RST 16
//#define USE_LR // Use ESP-NOW LR mode (ESP32 only)
// WiFi and MQTT Credentials -- These will override the global settings
#define WIFI_SSID "Clement4G"
#define WIFI_PASS "Clement123"
// NTP Time settings
// #define USDST
#define EUDST
#define TIME_SERVER "0.us.pool.ntp.org" // NTP time server to use. If FQDN at least one DNS server is required to resolve name
#define STD_OFFSET (-6) // Local standard time offset in hours from UTC - if unsure, check https://time.is
#define DST_OFFSET (STD_OFFSET + 1) // Local savings time offset in hours from UTC - if unsure, check https://time.is
#define TIME_FETCHNTP 60 // Time, in minutes, between fetching time from NTP server
#define TIME_PRINTTIME 15 // Time, in minutes, between printing local time to debug
#define TIME_SEND_INTERVAL 10 // Time, in minutes, between sending out time to remote devices
sensor config :
// FARM DATA RELAY SYSTEM
//
// Sensor Configuration
#define READING_ID 10 //Unique ID for this sensor
#define GTWY_MAC 0x01 //Address of the nearest gateway
// #define USE_ESPNOW
#define USE_LORA
#define DEEP_SLEEP
//#define POWER_CTRL 14
#define FDRS_DEBUG
#define DBG_LEVEL 2 // 0 for minimal messaging, 1 for troubleshooting, 2 for development
// I2C - OLED or RTC
#define I2C_SDA 5
#define I2C_SCL 6
// OLED -- Displays console debugging messages on an SSD1306 I²C OLED
// #define USE_OLED
#define OLED_HEADER "FDRS"
#define OLED_PAGE_SECS 30
#define OLED_RST -1
// LoRa Configuration
#define RADIOLIB_MODULE SX1276
#define LORA_SS 18
#define LORA_RST 23
#define LORA_DIO 26
#define LORA_BUSY 33
//#define USE_SX126X
#define LORA_TXPWR 17 // LoRa TX power in dBm (: +2dBm - +17dBm (for SX1276-7) +20dBm (for SX1278))
#define LORA_ACK // Request LoRa acknowledgment.
#define LORA_FREQUENCY 868
#define CUSTOM_SPI
#define LORA_SPI_SCK 5
#define LORA_SPI_MISO 19
#define LORA_SPI_MOSI 27
// Time settings
// #define USDST
#define EUDST
#define STD_OFFSET (-6) // Local standard time offset in hours from UTC - if unsure, check https://time.is
#define DST_OFFSET (STD_OFFSET + 1) // Local savings time offset in hours from UTC - if unsure, check https://time.is
#define TIME_PRINTTIME 15 // Time, in minutes, between printing local time to debugThanks, I think I see the problem now.
@aviateur17 It looks like the node is going to sleep before the async LoRa code has a chance to send the packet. Do you have a solution in mind? I'll have a chance to follow up more by this weekend.
timmbogner
commented
5 months ago @aviateur17 #212 may be related
aviateur17
commented
5 months ago In sleepFDRS() we can spin and check until LoRa buffer is empty or a timeout has expired and then go to sleep.
I'll do some testing tonight.
timmbogner
commented
5 months ago @aviateur17 It would seem that the actual transmission routines aren't being invoked for some reason. When deep sleep is turned off, the transmission still never occurs and the buffer just keeps growing with each iteration. I managed to recreate the overflow from #212 this way.
aviateur17
commented
5 months ago is loopFDRS(); called in loop()? I can replicate without loopFDRS() being called but seems to work when it is called - seems to send every 2nd or 3rd time boot when using a delay. The LoRa_Sensor example sketch does not call loopFDRS() in the loop() function.
Can you paste the entire sensor sketch if it is different than the LoRa_sensor sketch?
I'm hoping adding the loopFDRS() will fix this one up with the addition of making sure our LoRa buffer is empty before going to sleep/delay.
aviateur17
commented
5 months ago Added code below to top of sleepFDRS() in fdrs_node.h:
unsigned long timeout = millis() + 1000;
while(millis() < timeout && (!ISBUFFEMPTY(drBuff) || !ISBUFFEMPTY(spBuff))) {
handleLoRa();
yield();
}Will clean up tomorrow as this will not compile if LoRa is not enabled. But this seems to send each time.
timmbogner
commented
5 months ago I see the issue there. I forgot or didn't realize that sensors needed loopFDRS() now. My general thought is that it would be best not to need loopFDRS() in single-loop sketches. At first, the solution of tying everything up in sleepFDRS() seemed sufficient, given the issue at hand.
However the more I think about it, I think it's a bit confusing for the user if I say "You send data using sendFDRS(), but it won't actually be sent unless you do these other things...". Would there be an easy way to make sendFDRS() block execution until it finishes its radio transmissions? We can definitely still keep the async version for looping sketches that run loopFDRS().
aviateur17
commented
5 months ago Without adding blocking code the only way I can think of doing this is to just synchronously send instead of using async. This may be okay for a sensor that just sends a couple of readings and then delays or turns off. I'd have to look at the code again and see how that would work with ACKs. I'll look into the code and see how difficult that would be based on what's there now and get back.
aviateur17
commented
5 months ago I guess my preference would be to for some checks in sleepFDRS() as that is the most concise way of handling this.
Option 1 (my preference): Check for asynch completion in sleepFDRS() and not require loopFDRS() to be called. I believe this is more understandable and better for the long term. Option 2: For sensors use only synchronous function calls. I think this is more complex and makes the code more obscure.
More information on Option 2: To change to only synchronous calls for sensors we first need to determine that it is a sensor. We may be able to do this by checking for existing #define FDRS_NODE in fdrs_node.h and then checking for "is_controller" variable to be false but this is not good for the long run. If we decide to go this route it would be better to explicitly have a variable or define for FDRS_SENSOR, FDRS_CONTROLLER, FDRS_GATEWAY or similar, in my opinion. Once we know that we are a sensor then we can send synchronously but ACKs will be broken as now we need to figure out how to wait for some time to listen for the ACKs and respond accordingly. That functionality is built into handleLoRa() which is normally called by loopFDRS().
So, to summarize, in order to go synch we would need to determine that the node is a sensor and then ACKs would not be able to be used without additional coding complexity.
I think we can get away with not having the loopFDRS() call in the loop() if we call handleLoRa() (not directly called but via another function) inside the sleepFDRS() function before we go to sleep/delay. This is the route I'd prefer to go as this would keep all of the ACK and async functionality and only require a few lines of code and be the most straightforward. I tested this last night and it seems to work.
Let me know if you have questions, concerns, other ideas, want to discuss further, or which route you'd prefer to go.
timmbogner
commented
5 months ago I think I'm being confusing with my terminology. What I was hoping for was essentially to take this code that you just wrote for sleepFDRS():
unsigned long timeout = millis() + 1000;
while(millis() < timeout && (!ISBUFFEMPTY(drBuff) || !ISBUFFEMPTY(spBuff))) {
handleLoRa();
yield();
}...and add it at the end of sendFDRS(), so that it handles all of the transmissions before moving on.
I'll mull it over, but I don't think option 2 will be the solution.
aviateur17
commented
5 months ago Then we would still only need to run that code on sensors and wouldn't want to run it on controllers and gateways. So would need to have some additional code to determine that. sleepFDRS is only run for sensors so it seems to me to make the most sense to put it there as that is what requires it to be called and where it's needed to make sure stuff is sent.
timmbogner
commented
5 months ago sleepFDRS is only run on sensors, but it isn't (necessarily) always run. There's a chance the user will be handling sleep with their own code, or that they will delay after running sendFDRS(), take some more readings, and then run sendFDRS() a second time. Obviously there are ways that the user could be doing this better, but we still need to make sure it will work. I think the user should be able to run sendFDRS() and know their data has been sent by the time it finishes. That's why I'd propose that sendFDRS() be changed with the above suggestion, then add a new asyncSendFDRS() that will be used when there is a loop. Or maybe sendFDRS() with an extra argument to differentiate it... I'm flexible on the terminology.
aviateur17
commented
5 months ago Running SendFDRS() delaying and running it again is not an issue. The only issue is when the controller is commanded to shut down at a specific point - sleep - before any async processes are given a chance to complete.
The whole point of async process is to not spend time idly waiting for stuff to complete while other processes can happen in cases where things take a "long time" (in micro controller time) to happen.
When you call sendFDRS how do you differentiate between when a sync or async call happens? What needs to be known to differentiate between them? We can go back to everything happening synchronously but that degrades the performance of the controller but has the benefit of making code more simple.
think the user should be able to run sendFDRS() and know their data has been sent by the time it finishes.
That is not possible with async code. If that is the requirement then only synchronous code needs to be used.
That's why I'd propose that sendFDRS() be changed with the above suggestion, then add a new asyncSendFDRS() that will be used when there is a loop. Or maybe sendFDRS() with an extra argument to differentiate it... I'm flexible on the terminology.
When do you call sendFDRS and when do you call asyncSendFDRS? What in code would be used to differentiate those calls? Gateways and Controllers use sendFDRS currently.
The increased code required to handle ACKs both synchronously and Asynchronously would add quite a bit of complexity.
timmbogner
commented
5 months ago When do you call sendFDRS and when do you call asyncSendFDRS? What in code would be used to differentiate those calls? Gateways and Controllers use sendFDRS currently.
The user would make that decision based on the situation. If they are just sending the data and then going to sleep (as in most sensor examples), it will be fine for the system to idly wait for things to complete within the sending function. There's nothing else happening for it to interfere with. The controller and gateway examples will probably need to be changed to use the async version.
IF what I have in mind is really as simple as I'm making it, I should be able to mock-up an example tonight. I'll keep thinking on it while I'm gardening today too.
timmbogner
commented
5 months ago I'll confess that the actual method I had in mind didn't work, but I did get it how I want it in the end. With this change, the examples work again. This isn't final... there should also be a way to call sendFDRS(true); which can ignore the hang-time and act asynchronously as intended. What do you think?
~EDIT: It actually may not be going as smoothly as I originally thought.~ Never mind, it seems okay. 😅
aviateur17
commented
5 months ago I won't have time to comment today.
Basically what that is doing is making the async call synchronous by putting in a spinning loop. Is it the intention to use that for all sendFDRS calls or just certain situations? I have to think about it for a little bit. I think there are other issues that may come up.
Edit: If just certain situations then what situations and how do we code for them? If just sensor then how do we know, in runtime, that the device is a sensor?
timmbogner
commented
5 months ago If just certain situations then what situations and how do we code for them? If just sensor then how do we know, in runtime, that the device is a sensor?
Most examples are designed to run synchronously, and many people have already written code with the understanding that the data will be transmitted upon calling sendFDRS(), thus the default behavior should remain as such. The idea is that you can call alternatively call sendFDRS(true); and the function will then be run without the spinning loop, leaving it up to the user to call loopFDRS(). I'll work on that soon.
It won't hurt to still clear the send buffer before sleeping too.
EDIT: I've discovered that when deep sleep is enabled it still needs to delay about 50ms between sendFDRS() and sleepFDRS() or else the packet isn't transmitted. I'm thinking maybe I can use a different flag to signal to the program to leave the spinning loop. I'm hoping I can either use the existing LoRa state machine or transmitFlag. I'm finishing up for the night but will pick up on this investigation tomorrow.
fairoldi
commented
5 months ago Hi, first thank you for this amazing project! I'm trying to build a sensor based on a heltec cubecell HTCC-AV01 board. Using the main branch, lora messages aren't sent, even if not enabling deep sleep. I'm testing the synchronous_sendFDRS branch and as far as I can tell it fixes the issue, but I had to add
void yield() {} just before void beginFDRS() in fdrs_node.h to avoid a 'yield' was not declared in this scope yield(); compilation error. I'm sure this is not the right way to address it, as it is probably architecture / board dependent, but I don't have a lot of experience with this.
For what is worth, I think letting the user decide if the message should be sent synchronously is a good idea. An alternative could be to expose a flush() function that blocks until the buffer is sent, like you did in your branch, functionally it would be equivalent.
timmbogner
commented
5 months ago @fairoldi Thanks for the feedback! I meant to link that branch to this topic somehow. I just submitted #214
I am impressed that it works on CubeCell! It has been a while since I've heard of a user running FDRS on one, and I don't have one to test with. A quick compile test for me shows an error from the ArduinoJSON lbrary about using C++11 or newer.
It's kinda strange that it can't find yield() too. Not to go too far off-topic, but what board definition are you using?
fairoldi
commented
5 months ago @timmbogner I'm using "Cubecell development framework v1.5.0". I didn't have issues with ArduinoJson, but I had to downgrade RadioLib to 6.5.0 to avoid a bunch of constexpr which were causing c++11 errors.
timmbogner
commented
4 months ago This should be all fixed up! Thanks to everyone for your help!
If further problems arise, feel free to reopen or start a new issue!
Hi everyone,
I have a strange issue when I try to use FDRS with ESP32 LoRa Heltec v2 with display and the 1_UART_Gateway example.
I'm a newbie in CPP/Arduino programming.
I use PlatformIO V6.1.15 on code OSS (VS Code for arch Linux)
I'm on Linux
Build console output :
Does anyone have any idea how to resolve the problem?
Some line from my config file :