helium / router

router combines a LoRaWAN Network Server with an API for console, and provides a proxy to the Helium blockchain
Apache License 2.0
70 stars 31 forks source link

Data shows passing through hotspot in console but not in explorer #562

Closed Destable closed 2 years ago

Destable commented 2 years ago

Describe the bug Sensor is successfully sending packets through hotspot Mythical Maroon Bat according to console. But in the data tab for that hotspot in explorer shows no recent data. This has gone on since sensor activation 5 days ago.

7CFD4098-B281-448B-A464-1574EC01C47F 4A853610-43F0-4D6B-8113-A9E206576511

*To Reproduce

images of console showing the packet transfers attached. Image of the same hotspots explorer not showing the data attached.

Expected behavior I expected that after some reasonable delay, all data packets the transferred through mythical maroon bat would also show in explorer.

Device Info (please complete the following information):

NOTE: do not post your device app_key

Additional context According to console it looks like my hotspot is the only one receiving data from my sensor. At first I assumed there was a lag and that’s why I was not seeing it appear under explorer for my hotspot, but after so many days I reported it on Discord. Cyborg on discord asked me to post it here..

Screenshots Screenshots are attached, but may be above in the initial problem description.

ahmadsayed commented 2 years ago

Getting similar issue, the data I sent does not reported on the explorer of any hotspots showed on in the uplink or acknowledge.

It only shows the data I sent 7 days ago, any data I sent during this week and today , does not appear in the explorer, while it appears int he console, and it successfully received by mqtt server and my app.

hotspots - not mine - Tangy Saffron Orangutan Sour Blush Ferret

jdgemm commented 2 years ago

Thanks for reporting. Currently, the team is out on winter holidays for much needed rest, but when they return this will be a high priority item to investigate.

anders462 commented 2 years ago

I reported the same thing but in another discord channel but never got any responses, so I am glad this issue has been raised @Destable . Have a sensor sending every 5 min and many of the packets are being picked up by my home hotspot according to console, but never reported in explorer or visible in the etl.dewi.org. Be happy to provide more info.

macpie commented 2 years ago

Issue was identified, the main production Router is unable to close state channels due to a chain var sc_grace_blocks beeing set to 1. Giving only a window of 1 block to submit and absorb the txn making it nearly impossible to submit in time. We will update this chain var to allow 5 blocks (99% of the time we can get the txn absorbed in 2 blocks)

anders462 commented 2 years ago

Sounds like this fix should boost the total data (recorded) transfer in the network

macpie commented 2 years ago

Sounds like this fix should boost the total data (recorded) transfer in the network

It should yes.

Note that we are waiting on a validator update to push new chain var updates, I will "fix/close" this issue as soon as the var as been updated and that I can confirm that state channels are now closing properly.

Thanks for the report!

ahmadsayed commented 2 years ago

@macpie will the fix applied retroactively to historical packets, or only new packet will show up ??

ahmadsayed commented 2 years ago

Sounds like this fix should boost the total data (recorded) transfer in the network @anders462

I noticed that behavior and tried to report it in discord, but i did not have solid evidence back then, because I was picking random hotspots from my area., I was not aware of etl.dewi back then, I thought it was issue with my location Singapore only, but looking at etl.dewi now, I noticed global pattern , before 21 Nov the data credit is picking it up, then a spike happen in 21 Nov, then a drop the overall data credit per hotspot decrased, may be this is not related and just coincidence some major operator was testing something or helium team was testing something I am not sure,
image US image Italy image

macpie commented 2 years ago

@macpie will the fix applied retrospectively to historical packets, or only new packet will show up ??

No we cannot apply retrospectively

jdgemm commented 2 years ago

@Destable pls confirm with latest chain variable release the issue has been resolved.

Destable commented 2 years ago

It appears to have solved it. My hotspot has been flatlined for a couple days on loss of sync and has just caught up. Immediately following the chain variable release however I began getting regular data traffic as expected on explorer for my hotspot. I just synced up this evening so I should have more samples to check moving forward. I consider this item solved based on what I’m seeing.

jdgemm commented 2 years ago

Great news and thanks for reporting back.

mfalkvidd commented 2 years ago

The global DC usage drop (reported above by @ahmadsayed ) does not seem to have recovered. From https://etl.dewi.org/dashboard/22-data-credit-usage?relative_date=lastyear 1DB1A9AA-F909-4505-8ACB-0209D2DC375C

ahmadsayed commented 2 years ago

@mfalkvidd I am quiet confident Helium team on top of it, but if you filter the results to the last 30 days you will find something happened few days ago (https://etl.dewi.org/dashboard/22-data-credit-usage?relative_date=past30days)

image

based on my understanding the fix is not retroactive, it will be effective for the recent data, I started to see packet on my data only hotspot and my neighbor full hotspots.

mfalkvidd commented 2 years ago

@ahmadsayed that looks great, thank you! I think my mistake was that last year != past year image

dvdadie2 commented 2 years ago

this is exactly what has been happening to my hotspot for the last 28 hours... same issue?