search

helium / erlang-libp2p

An Erlang implementation of libp2p swarms
https://helium.github.io/erlang-libp2p
Apache License 2.0
121 stars 34 forks source link

Hotspots going relayed into after 2021-09-03~ update multiple users #381

Open MMOStars opened 3 years ago

MMOStars commented 3 years ago

Hello, there are at least 10-15 people on Sensecap discord reporting same issue that after 2021-09-03~ update

100% this change broke something for us: https://github.com/helium/erlang-libp2p/commit/f31e45fd98b32e3d0142da0932a941b14cbf7578

This is what we get via peerbook:

| address | name |listen_add|connectio| nat |last_updat| +----------------------------------------------+-----------+----------+---------+-------+----------+ |/p2p/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5uj|odd-clay-wo| 2 | 5 |symmetr|15306.244s| +----------------------------------------------+-----------+----------+---------+-------+----------+

+--------------------------------------------------------------------------------------------------+ | listen_addrs (prioritized) | +--------------------------------------------------------------------------------------------------+ | /ip4/xxxxxxxxxxxxxx/tcp/44158 | |/p2p/11KgYQR9h3Rz9cmzVZLwtZiHxThMs731uFTraLiyPmfHcxp7DYx/p2p-circuit/p2p/113gAgf8DTFi4YQFrQYMRhN7W| +--------------------------------------------------------------------------------------------------+

+---------------------------------------------------------+ | connections | +---------------------------------------------------------+ |/p2p/11KgYQR9h3Rz9cmzVZLwtZiHxThMs731uFTraLiyPmfHcxp7DYx | |/p2p/1128k7YK3UfSh5qaAp37qe2jw3LaG6ycQUAAqD33L9v9DoRHdrkK| |/p2p/112BwJXdn117Y9pNcte1VKpZWFViakbi9tuUMrDsmpnZsM6DEfPb| |/p2p/11GxM19uKgCpoArMGBUDGeNLxRYM8L9ferCCd8sHkgFdZtVU86P | |/p2p/11XG5Dw6L1jmu9XE61b1LJ6TdgWm7An1a1Nut34XwMbUyiUWoQV | +---------------------------------------------------------+

Sensecap dashboard reporting this:

Listen IP(Based on Helium API): /ip4/xxxxxxxx/tcp/44158,/p2p/1121Gc63SGenk9emVaEe3TsQejLgQHp193ptawGmtp7q34M4X9yQ/p2p-circuit/p2p/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8 NAT Type: Symmetric Relayed: Yes

Port 44158 has been confirmed to be open as per https://www.yougetsignal.com/tools/open-ports/

The hotspot WAS NOT relayed until last update! The NAT type on Router is COG (In other words I presume CONE), NOT SYMMETRIC! And this change somehow changes our nat type to symmetric, but it's not symmetric! https://explorer.helium.com/hotspots/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8/activity HELIUM EXPLORER DOES NOT REPORT THAT HOTSPOT IS RELAYED!

After we reboot the hotspot, the relay is reporting is NONE and Relayed status is: No. After 2-4 hours, poof, we become relayed and start listing on both IP + P2P, which should not happen. I own 3 hotspots, 1 hotspot on same ISP, does not have this type of issue, which is super odd, but this can't because of our router settings as technically we are not choosing to somehow assign 2 listening IPs to API, which should not be happening.

Please help solve this issue ASAP!

ranezinjo commented 3 years ago

I am having THE SAME issue!

sullyjman commented 3 years ago

Been seeing a lot of reports of this in the helium discord

Vagabond commented 3 years ago

This won't actually harm you, although it is an issue. This is likely fallout from the IP changing issue fixes over at #374 . I know the app might scream at you but if you have a valid external IP as the first row in the listen addrs you're fine, they're tried in the order they appear.

I will take a look as to how this is happening again. If you have an affected hotspot you can email me the name/serial number/manufacturer and I can try to work with the manufacturer to pull logs.

MMOStars commented 3 years ago

This won't actually harm you, although it is an issue. This is likely fallout from the IP changing issue fixes over at #374 . I know the app might scream at you but if you have a valid external IP as the first row in the listen addrs you're fine, they're tried in the order they appear.

I will take a look as to how this is happening again. If you have an affected hotspot you can email me the name/serial number/manufacturer and I can try to work with the manufacturer to pull logs.

Good day, my person is odd-clay-worm (https://explorer.helium.com/hotspots/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8)

If you need logs, Jerry (dev) from Sensecap would be willing to work on this 1 with you.

I would like to encourage other posters to share their addresses for this issue to be fixed ASAP.

Vanlushai commented 3 years ago

I have the same issue. . .please let me know how to fix it

trillion95 commented 3 years ago

I have the same issue 🙃

arnoudius commented 3 years ago

I have the same issue. Rebooting the device does indeed temporarily fix the issue, but then a few hours later the issue occurs again. I have had this since the 2021-09-03 firmware update. Before that, it was working fine / not switching to relayed after a few hours.

ArnoutVerbeken commented 3 years ago

So MMOstars was able to compile a list of all hotspots affected by this issue. He could not access Github and asked me to post this json file here. See attached. Maybe it can help you troubleshooting or prioritizing this issue. As an overview, here are the total numbers: "{"helium": 1, "calchip": 922, "nebra": 137, "syncro": 21, "bobcat": 1800, "LongAP": 38, "mimic": 0, "rak": 4, "kerlink": 0, "dewi": 0, "sensecap": 281, "unknown": 0}" Which boils down to the following numbers in percentage (of the total fleet of that brand): Bobcat ~3% calchip ~1.6% Sensecap ~2.4% Nebra ~2.1% LongAP ~7.6% bugged_miners.zip

arnoudius commented 3 years ago

It also seems to happen more frequently now. I just rebooted the miner an hour ago (sensecap m1), and now (+/- one hour later) it's already saying it's relayed again. It seems the service can still be reached though. If I for example use canyouseeme.org, it does report the port is open / can see the service, but on the Sensecap dashboard it reports it as being relayed / p2p status unhealthy. I also don't receive any rewards while it's reporting that it is being relayed. As soon as I restart the device, I start receiving rewards again, until it gets relayed again (after a few hours).

thatsaint commented 3 years ago

This same issue is happening to 2 of my Bobcats

trillion95 commented 3 years ago

They need to fix this ASAP!!! I shouldn’t have to restart my hs every time!!!!!

Vagabond commented 3 years ago

If you read my earlier response, you will know that this bug is not actually going to harm you. I will work on fixing it but the relay status is not going to cause you any problems itself.

kzkz0 commented 3 years ago

This is happening to my SenseCap M1 Hot Spot after I switched it from WiFi to ethernet cable. The dashboard says it's not relayed but the Helium app says it is and I have a really long string of numbers and characters for my Listen IP.

I have confirmed port 44158 is open and I have created challenges and a few beacons but haven't seen any witnesses since this has happened. If you need more details please let me know.

GoviCoin commented 3 years ago

having same issue , i switched from WiFI to Ethernet, still says relayed...checked API. says p2p https://explorer.helium.com/hotspots/11MwQFbLUPso73YZQY2mjdbiCBVNjTkCVX2yB8asTFtcBsk7Knk any help?

kzkz0 commented 3 years ago

Update - My Listen IP is now correct and I am no longer relayed I guess it just needed time.

rockthereelstape commented 3 years ago

Same thing just happened to 2 of my hotspots , sensecap

MMOStars commented 3 years ago

Hi devs, was this patch pushed with the latest release 2021.09.14.0? Seems to be still happening. Etc:

https://explorer.helium.com/hotspots/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8

address | name |listen_add|connectio| nat |last_updat| +----------------------------------------------+-----------+----------+---------+-------+----------+ |/p2p/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5uj|odd-clay-wo| 2 | 8 |symmetr| 188.172s | +----------------------------------------------+-----------+----------+---------+-------+----------+

+--------------------------------------------------------------------------------------------------+ | listen_addrs (prioritized) | +--------------------------------------------------------------------------------------------------+ | /ip4/xxxxxxxxxxxxx/tcp/44158 | |/p2p/11M9iUMErdCpgx3HvvJoeLBos3vnbTBBg7Pb36922ECYA1Tigwv/p2p-circuit/p2p/113gAgf8DTFi4YQFrQYMRhN7W| +--------------------------------------------------------------------------------------------------+

+---------------------------------------------------------+ | connections | +---------------------------------------------------------+ |/p2p/11M9iUMErdCpgx3HvvJoeLBos3vnbTBBg7Pb36922ECYA1Tigwv | |/p2p/11N6nbmHR81LcWZvxyfJQG5wFxXVqGAgg3qJeEjtKpGGB5muVTt | |/p2p/112SwEwjkPxkLPqzHr5AodiZovnk4EbvR6BsVpRomJ96ggvx7BZV| |/p2p/13xhTvtw9bFKgN6zbzKXVptuXzs1cb1bJFjaLP3oWLULviAjktM | |/p2p/11G9UnvWtaQQsxWKFX9fgk2iqDksqTEndKqKyg4dL5aEG8y1QpQ | |/p2p/112cPFo5r9UW4uHWfEDCVa5JH4mufmDhQr6SRagdsJPi2RXgY4e9| |/p2p/112VDXfCV1iTe1GufSkEKhoCRDJa7J6LZYYAp8VVQYiXpDvCGaFE| |/p2p/1197AexoUhZozLqpagTGw4959faXWcHT699KsVEaoLwdNpoq7Ub | +---------------------------------------------------------+

arnoudius commented 3 years ago

Doesn't seem like it indeed. I just received the update (2021.09.14.0), but after a few hours it's now saying 'relayed' again on the dashboard (sensecap).

ArnoutVerbeken commented 3 years ago

Same here, new FW 2019.09.14, relayed according to dashboard and peerbook gives this:

|                   address                   |    name     |listen_add|connectio|  nat  |last_updat|
+---------------------------------------------+-------------+----------+---------+-------+----------+
|/p2p/112kt1425oAGpqCPBQS8qyYK3qeKyTv1W1Wzos5J|dizzy-pecan-y|    2     |    9    |symmetr|  85.3s   |
+---------------------------------------------+-------------+----------+---------+-------+----------+

+--------------------------------------------------------------------------------------------------+
|                                    listen_addrs (prioritized)                                    |
+--------------------------------------------------------------------------------------------------+
|                                  /ip4/xx.xx.xx.xx/tcp/44158                                   |
|/p2p/1183NZjePCqisgHEcyHNkrfEMRTQW3Lc3j9VE1GDr8V1PryNQuU/p2p-circuit/p2p/112kt1425oAGpqCPBQS8qyYK3|
+--------------------------------------------------------------------------------------------------+

+---------------------------------------------------------+
|                       connections                       |
+---------------------------------------------------------+
|/p2p/112dADs1gLzPckMkkjDdk9ihp1EdEQmVw5i7hJztf6wyWArvrL6b|
|/p2p/11rkWQQptJzyFKhqTvZRt9W1zBW6Jd1v3FLXg5fGP6K3C3Sj68w |
|/p2p/11tRi5ouPAuodT7j7bPJ36cKZot6txhre6XgaEtcfgpiPEFY3BS |
|/p2p/112BwJXdn117Y9pNcte1VKpZWFViakbi9tuUMrDsmpnZsM6DEfPb|
|/p2p/11D9oVhjTAMZmfgnUn1d3PMtrV8xSNXrRf51fGFnSjWbzovtctW |
|/p2p/11rVv72FaqA3wmWgT1dH7XYRB21GGKm5p9BSzSCniFHbussXKw5 |
|/p2p/112ua5DtuAsH9FuqvWQqvSZbsBXXVjaQ2enJCeDtzsijPs8zctx5|
|/p2p/112ke7UfbGA41DPYEmnt1FZHh4BE9wfL3hCbqy5MxqZRq2i8BmYm|
|/p2p/1183NZjePCqisgHEcyHNkrfEMRTQW3Lc3j9VE1GDr8V1PryNQuU |
+---------------------------------------------------------+
chowdarynaresh111 commented 3 years ago

I also have the same issue after restarting the Device it started creating challenges and dashboard shows ok now. From morning almost 6 hours once this issue started no activity recorded .

chowdarynaresh111 commented 3 years ago

I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online.

Vanlushai commented 3 years ago

I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online.

Is it a permanent fix? is it still out of relay?

chowdarynaresh111 commented 3 years ago

It is out of relay now and double IP it took some time to reflect at API.

Sent from my iPhone

On Sep 17, 2021, at 12:33 PM, Vanlushai @.***> wrote:

 I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online.

Is it a permanent fix? is it still out of relay?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

Vanlushai commented 3 years ago

It is out of relay now and double IP it took some time to reflect at API. … Sent from my iPhone On Sep 17, 2021, at 12:33 PM, Vanlushai @.***> wrote:  I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online. Is it a permanent fix? is it still out of relay? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

Oh. .ok. .but your miner is showing as relayed on the API ? my miners are showing as relayed only on the sensecap dashboard. .

chowdarynaresh111 commented 3 years ago

Mine also shows relayed only on Dashboard and API has two lines for Lisner IPV4 and P2P. AFter I removed the wifi connection, relayed status was gone on Dashboard and it took few hours to remove one line from API.

On Fri, Sep 17, 2021 at 10:58 PM Vanlushai @.***> wrote:

It is out of relay now and double IP it took some time to reflect at API. … <#m-2580854958049372941> Sent from my iPhone On Sep 17, 2021, at 12:33 PM, Vanlushai @.***> wrote:  I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online. Is it a permanent fix? is it still out of relay? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

Oh. .ok. .but your miner is showing as relayed on the API ? my miners are showing as relayed only on the sensecap dashboard. .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/helium/erlang-libp2p/issues/381#issuecomment-922167534, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVVG5KSY6KF2NYSVVDCN5RLUCP545ANCNFSM5DSNUOXQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Vanlushai commented 3 years ago

Mine also shows relayed only on Dashboard and API has two lines for Lisner IPV4 and P2P. AFter I removed the wifi connection, relayed status was gone on Dashboard and it took few hours to remove one line from API. … On Fri, Sep 17, 2021 at 10:58 PM Vanlushai @.> wrote: It is out of relay now and double IP it took some time to reflect at API. … <#m-2580854958049372941> Sent from my iPhone On Sep 17, 2021, at 12:33 PM, Vanlushai @.> wrote:  I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online. Is it a permanent fix? is it still out of relay? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. Oh. .ok. .but your miner is showing as relayed on the API ? my miners are showing as relayed only on the sensecap dashboard. . — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#381 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVVG5KSY6KF2NYSVVDCN5RLUCP545ANCNFSM5DSNUOXQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

👍👍i shall try the same thing. Can you please tell me what miner are you using?

MMOStars commented 3 years ago

Update from me, after exactly 6 days of uptime now my 2/4 units experience this bug?

The unit was untouched based on logs all that time.

https://explorer.helium.com/hotspots/11iRTRDKEQ77etK8EwbB3deLEpK43o3m4NeUwFDJXgr6EKj8xDa

This bug also prevents us from running discovery mode with is plain tilting!

jakub2907 commented 3 years ago

Just got the same issue. Hot Spot running fine for weeks on Public IP with ports open and all of a sudden it got Relayed. I also have two Listen Ips when looking into a Peerbook on SenseCap dashboard. Here's my Hot Spot that is experiencing this https://explorer.helium.com/hotspots/11vyHF7YcEVu4xgXxxGAGsZCoQE9bXsV7qLmM8QFdh2q9G31mch/activity

trillion95 commented 3 years ago

I’m getting the same problems with latest firmware again! 😕

trillion95 commented 3 years ago

98E57A67-7883-46CE-9D73-9B53826FC44D

Vagabond commented 3 years ago

Has anyone tried enabling uPNP/PMP instead?

Also, if you have two listen IPs, and one is the public IP you expect and one is a relay, you're actually fine as the public IP is preferred. I know the app/explorer makes a big fuss about "relayed" but in this case it is not an actual problem.

steel171 commented 3 years ago

Hello everyone! I have the same issue as everyone here.. explorer is fine while the api shows two listening address, one is from my public IP and the other is a p2p address. image

i was on wifi and then moved to ethernet (i'm connected to wifi repeater) do you guys think this has something to do with my miner's low activity?

PaulVMo commented 3 years ago

In case it wasn't obvious, this issue is also impacting validators. At present, approximately 1400 validators are showing an IP4 and P2P address at the same time. While annoying and causing repeat questions on Discord, there is no indication that it causes an performance impact at this point.