helium / miner

Miner for the helium blockchain
Apache License 2.0
608 stars 266 forks source link

2021.12.14 Reports from Hotspot Owners #1300

Open abhay opened 2 years ago

abhay commented 2 years ago

The core team is going to try a new way of tracking issues related to specific miner releases.

We will create release specific threads and see if that helps with miner support so that we don't loose track of issues from deployed Hotspots. Please verify that your Hotspot is on the current release version and if you're still having issues, please let us know on this thread.

RAK and OG Hotspots must be on 2021.12.14.1 and all other Hotspots could be on 2021.12.14.0.

Manufacturer specific issues should be directed to individual manufacturer support.

vanjulio commented 2 years ago

I thought all the raspberry Pi's are ARMS? What would use AMD? the OG helium kits? Making me think of a big endian vs little endian although I believe both are little endien

bkauii commented 2 years ago

I have 36 hotspots. Everyday i wake up there is a minimum of 5 offline. Right now I have 10 offline and no idea why

serbyxp commented 2 years ago

miner-arm64_2022-01-19.0-test1 error logs I know its a test image, and not all if any one else is running it, and/or on the AMD64 version (validators) Im just posting but maybe it helps if any of the failed peers are running it. not going to dig into it. maybe it helps.

error_logs_miner-ARM64_2022_01_20-test1.txt

I thought all the raspberry Pi's are ARMS? What would use AMD? the OG helium kits? Making me think of a big endian vs little endian although I believe both are little endien

I havent looked up more then just helium OG miners or nebra miner since that was my origonal miner manufacturer. But through random advertisements i have seen some miners use an intel chip similar to the intel NUC I could be wrong about that though. But the validators I believe are the ones running the AMD version from what I understand. Since they run on AWS servers I could be wrong about this. But the miners that are runing ARM64 image seem to only talk with the same ARM64 images. With that being said, and the amount of various manufacturers. They could be compiling there own images, to combat the current issue, They may have found their own fix in a sense. But I can not confirm that, Its just speculation, rather then saying that they are doing it on purpose for what some may call "stealing, or gaming" I dont believe that to be true, But my speculation could be the actual reason that some work better then others, ( that they made their own fix) but some manufacturers I have tried looking for repos for their images, and have been unsucesful. The only way would be for someone that has a good working miner to rip the image out of it, No one in their right mind will go about this, Im only into this because mine isnt working " If it aint broke dont fix it"

The thing with the ARM64 is that you can use AVX but its not being compiled that way as per the documentation, Im sure there is a reason for that, The dev team knows what they are doing Im sure of it. But somethings are outside of their control. I.E different manufactures with different hardware, Helium isnt responsible for the manufactures getting their hardware to work with the network, They have a small responsibility of keeping their OG hardware up to date, and functional. But they don't have much control over the validators, and how they operate their AWS machines. The point being that if manufacture X implements something that's different than manufacture Y or even the "Official image", and their is no " reporting" mechanism in place like that they have to send or post their software, which they can claim as being " proprietary software", and they don't share it for inspection. Then theirs no way of making sure whats different. The same way helium "silently" implemented a deny list, The validators could just as easily " silently" create their own. So as to block people they feel like blocking, for whatever reason. But their should be a way of anyone checking against their list, But if they are going that far, They will just work around it and block or spoof whatever it is that helium implements as a checksum. So it would have to be done in " silence" I don't need the NSA at my door so that's beyond what I am willing to do for free without a legally binding contract putting the responsibility on them and not me

serbyxp commented 2 years ago

I dont know if its possible. but if their is a validator out there that has a bunch of hotspots that are ARM64 and is noticing this same error, on their fleet of miners, They might want to test or try running the ARM64 version vs the AMD64 version, That could be a way to confirm this. At least have 1 validator that's available for the rest of the miners with this issue I dont know if that will work, but it could at least confirm if this is even the issue.

Shoot it might even make them alot more $$ because they will scoop up all the failed errors for themselves.. Think about it..

serbyxp commented 2 years ago

arm64-beta error logs. I have started beaconing more often, and it seems more of the hotspots around me have too, thus increasing my witnessing. I feel comfortable enough now to properly place the hotspot outside on the roof again. Still lots of errors, I had a weird replay error which Is not shown in this logs it was on the 1/25/2022 alpha and a bunch of gen server errors and issues so I reverted to the miner arm64 beta then the replay block error came out, I waited until the Snapshotter was up beyond my block and Grabed it and resolved that issue. I also notice a random new volume called " HOT_FIX" maybe that's just for the testing purposes of alpha. Either way I know this isn't a official release but here you go. ( I deleted the 1/25/2025_alpha logs and all that when trying to troubleshoot the replay error when reverting back) but It was a bunch of line errors an gen_server call errors that would " have a child" and terminate, restart, it would connect to some peers, but it would get stuck in a loop like that, So I could never get passed like 20 seconds on the console) I did notice and read the beaconing documentation about relayed challangers, and relayed witness receipts that was informative, but most of the errors are validators. I get a connection refused every now and then but those are out of the country typically. Atleast theirs hope for now until gRPC comes out.

error_beta.txt

serbyxp commented 2 years ago

alpha-arm64 running for about an hour got first witness and with it first failed to dial. no nat dialable and connected

Line 1676: 2022-01-26 21:38:49.270 8 [warning] <0.1526.0>@libp2p_group_worker:connecting:{237,5} Failed to connect to "/p2p/114ZvENBvWtD8kPfgSG1Z9jxXJuzZ5WKDjQfuee2eH8KKuAh11W": [{"/p2p/11qL63DAz6WkTBn8TQmb1AQxVyD4SNE58U6pEwHXYTi9MTpAsRi/p2p-circuit/p2p/114ZvENBvWtD8kPfgSG1Z9jxXJuzZ5WKDjQfuee2eH8KKuAh11W",not_found}]
Line 1708: 2022-01-26 21:39:09.318 8 [warning] <0.1526.0>@libp2p_group_worker:connecting:{237,5} Failed to connect to "/p2p/112LGjH3TQSJ5PQWD1sVcRoPgwHyoQqed25zbAxU6QkQ8XXnMjx1": [{"/p2p/11xcw8oJ4NVDfoArscLzm6zyiDibL8dKjrwWWynmzveNsPKACN2/p2p-circuit/p2p/112LGjH3TQSJ5PQWD1sVcRoPgwHyoQqed25zbAxU6QkQ8XXnMjx1",not_found}]
Line 2128: 2022-01-26 21:48:49.237 8 [warning] <0.1527.0>@libp2p_group_worker:connecting:{237,5} Failed to connect to "/p2p/112B53ALUV5tqE68q2PMXpLiEmGZ8q6xehSRWWv45JtbXAxkv9mp": [{"/p2p/112g9KFKxQJF4KnzTa4bVuy2w5kkMEGZ6eLTW4wdgNRDvCYBFxAu/p2p-circuit/p2p/112B53ALUV5tqE68q2PMXpLiEmGZ8q6xehSRWWv45JtbXAxkv9mp",not_found}]
Line 2154: 2022-01-26 21:49:14.268 8 [warning] <0.1527.0>@libp2p_group_worker:connecting:{237,5} Failed to connect to "/p2p/11LxYQxJuBnWppfNKZe8JdkdBeAynYghhqZJg9y8hR8ndH5xoXx": [{"/ip4/51.81.155.198/tcp/2154",timeout},{"/p2p/142e4nzzEjGYLx6nsJgmVGreht4wHN7qTSfXSYfCbKgdy2sUqFa/p2p-circuit/p2p/11LxYQxJuBnWppfNKZe8JdkdBeAynYghhqZJg9y8hR8ndH5xoXx",not_found}]
Line 2410: 2022-01-26 21:54:07.084 8 [warning] <0.1527.0>@libp2p_group_worker:connecting:{237,5} Failed to connect to "/p2p/112DZHc7TX17yCawg1zQQb5oHxg41pNK93UvBM3UczrBAwGqsaLV": [{"/p2p/11UJwidtEpYEbXPaq78H4ymkbd4q5m1GoNSViEJz8he3ELnZsSY/p2p-circuit/p2p/112DZHc7TX17yCawg1zQQb5oHxg41pNK93UvBM3UczrBAwGqsaLV",timeout_relay_session}]
Line 2418: 2022-01-26 21:54:27.122 8 [warning] <0.1527.0>@libp2p_group_worker:connecting:{237,5} Failed to connect to "/p2p/112noBrguM594KYMcS5HsPv6TsK9krhUtFq7QRJQeFPu3VrJsrv5": [{"/p2p/112ZM5omHsfvYs4PKcahQHBL9dz4NJSX4oaP3aQsedm2gCm7XPDf/p2p-circuit/p2p/112noBrguM594KYMcS5HsPv6TsK9krhUtFq7QRJQeFPu3VrJsrv5",not_found}]
Line 2810: 2022-01-26 22:04:10.853 8 [warning] <0.4243.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/112LiVQnQ1wAvGxa9kdmXhT6EsiH5E6Sh2AtuVJcuv9ZM1Br85NV": not_found
Line 2876: 2022-01-26 22:04:57.419 8 [warning] <0.4243.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/112LiVQnQ1wAvGxa9kdmXhT6EsiH5E6Sh2AtuVJcuv9ZM1Br85NV": [{"/p2p/112TAY3QAwMXdr5EmyRo96pnhzLzPWC4stJtSijQpww21eDTXFTg/p2p-circuit/p2p/112LiVQnQ1wAvGxa9kdmXhT6EsiH5E6Sh2AtuVJcuv9ZM1Br85NV",timeout_relay_session}]
Line 3566: 2022-01-26 22:18:49.265 8 [warning] <0.1526.0>@libp2p_group_worker:connecting:{237,5} Failed to connect to "/p2p/11BXrYy1pYPDsAq5kPgZcujtMQ2iZri8q12gZgeTkPEdFRNcs4b": [{"/p2p/112SeiCxNWn49iDHX2WJJp545dxnmMvKc3imKtczDCs68FYvd224/p2p-circuit/p2p/11BXrYy1pYPDsAq5kPgZcujtMQ2iZri8q12gZgeTkPEdFRNcs4b",not_found}]

suggestion, If the hotspot or miner is relayed. make that a part of the algo that allows a hotspot to create or participate in challanges. Save everyone the hassle.

serbyxp commented 2 years ago

alpha-arm64 Line 19403: 2022-01-27 00:27:00.375 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19421: 2022-01-27 00:27:30.489 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19479: 2022-01-27 00:28:00.599 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19497: 2022-01-27 00:28:30.715 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19531: 2022-01-27 00:29:00.839 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19561: 2022-01-27 00:29:30.956 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19609: 2022-01-27 00:30:01.074 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19627: 2022-01-27 00:30:31.345 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19665: 2022-01-27 00:31:01.614 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found Line 19711: 2022-01-27 00:31:31.726 8 [warning] <0.1961.0>@miner_onion_server:send_witness:{243,37} failed to dial challenger "/p2p/113QTBVRENeJvNTH36WBKmr7urNj2hV5Z66GGDfm3DoMfhz95wk": not_found

just loaded beta...

This looks like a much bigger issue now that all the line errors have been resolved since like 1/12/2022

I think your main issue now is government / ISP firewalls...

hypothetical situation.. Country A , Country B , Country C Open, Open, Closed, Inbound Ports Open, Open, Open, Outbound Ports

A and B can accept Inbound Challanges from A, B, C, A and B can send recipts to A, B, not C Error failed to dial.

its more complex then that since there are more then 3 Countries or ISP.

say Country C has open inbound port to Country D and country D has open inbound to Country B but not to Country A, Logically to get the message from A to C You would have to Send ("circuit" "hop" "relay" w/e). A to C Through A-> B -> D -> C . ^^ maybe? if that made sense you get the jist. Now, same as FCC and EU etc... have regulatory issues, Maybe the network needs to compensate for Regulatory Firewalls. so a US915 pool of challangers might need to construct challanges for US915, EU868, for EU868 ... etc... that OR Based off hotspot Geo Location (Asserted location) can use some sort of DataBase or Routing Table... To construct the challanges...

Now your not even talkin Hotspot antenna "gaming" now your talking whole fleets using cross country / VPN gaming. That is something easy to catch. But a hell of a workload to construct that database, Good luck with that one. Cus now your getting involved with national security in every country.

ghost commented 2 years ago

hi guys, I recently bought a P100, after having mounted it I found problems with invalid witnes despite everything being declared correctly, I read on github that it was necessary to change 2 "rssi offset" parameters. so I did and actually the invalid witnesses have become valid, however the p100 often crashes and loses sync with the blockchain, a few hours ago I carried out a portforwarding on port 44158 (tcp and udp) but in any case I notice that during the resync process there are things that crash, below are the logs

errorlog.txt minerlog.txt witnesslog.txt

Could you point me in the direction or explain me how you changed the rssi offset on the Pisces? Thanks!

serbyxp commented 2 years ago

hi guys, I recently bought a P100, after having mounted it I found problems with invalid witnes despite everything being declared correctly, I read on github that it was necessary to change 2 "rssi offset" parameters. so I did and actually the invalid witnesses have become valid, however the p100 often crashes and loses sync with the blockchain, a few hours ago I carried out a portforwarding on port 44158 (tcp and udp) but in any case I notice that during the resync process there are things that crash, below are the logs errorlog.txt minerlog.txt witnesslog.txt

Could you point me in the direction or explain me how you changed the rssi offset on the Pisces? Thanks!

I think you would have to contact Pisces for that, it would be in your lora concentrator sx130x global_conf.json or local_conf.json it is diffrent for all manufactures so consult with your manufacturer. You will see a json format file which if changed ( formatted) incorrectly will cause issues and not load but if your going to do it. Just find and change , “rssi_offset” under “radio_0 and “radio_1” “rssi_offset”: -215.4,

the -215.4 is what you would change depending on your concentrator / setup it might show something different then -215.4 by default, mine shows -215.4 as per the helium hardware global_conf.json for US915… it varies I believe from frequency / concentrator / hardware

note that if there is a local_conf.json file that would be the one to change , as it overrides / overlays the global_conf.json

here is some examples from heliums GitHub https://github.com/helium/sx1302_hal/tree/helium/hotspot/packet_forwarder

note that all manufacturers have different settings and setups do consult with them and their documentation. As their configuration files , hardware and settings are different then other manufacturers.

*some manufacturers use specific operating systems / firmware , that may automatically delete your changes when the system restarts, or updates the firmware. So be weary of that. You may make the changes, and when you restart or a firmware upgrade is issued it will clear it out and you will have to re do it every time. So it’s one of those things… you got to find out, which if any file can even be modified

I am not associated with helium or any manufacturer. But that is a manufacturer specific inquiry for the customer service and support of the manufacturer. Don’t do anything that could void your warranty, without contacting them first…

ghost commented 2 years ago

hi guys, I recently bought a P100, after having mounted it I found problems with invalid witnes despite everything being declared correctly, I read on github that it was necessary to change 2 "rssi offset" parameters. so I did and actually the invalid witnesses have become valid, however the p100 often crashes and loses sync with the blockchain, a few hours ago I carried out a portforwarding on port 44158 (tcp and udp) but in any case I notice that during the resync process there are things that crash, below are the logs errorlog.txt minerlog.txt witnesslog.txt

Could you point me in the direction or explain me how you changed the rssi offset on the Pisces? Thanks!

I think you would have to contact Pisces for that, it would be in your lora concentrator sx130x global_conf.json or local_conf.json it is diffrent for all manufactures so consult with your manufacturer. You will see a json format file which if changed ( formatted) incorrectly will cause issues and not load but if your going to do it. Just find and change , “rssi_offset” under “radio_0 and “radio_1” “rssi_offset”: -215.4,

the -215.4 is what you would change depending on your concentrator / setup it might show something different then -215.4 by default, mine shows -215.4 as per the helium hardware global_conf.json for US915… it varies I believe from frequency / concentrator / hardware

note that if there is a local_conf.json file that would be the one to change , as it overrides / overlays the global_conf.json

here is some examples from heliums GitHub https://github.com/helium/sx1302_hal/tree/helium/hotspot/packet_forwarder

note that all manufacturers have different settings and setups do consult with them and their documentation. As their configuration files , hardware and settings are different then other manufacturers.

*some manufacturers use specific operating systems / firmware , that may automatically delete your changes when the system restarts, or updates the firmware. So be weary of that. You may make the changes, and when you restart or a firmware upgrade is issued it will clear it out and you will have to re do it every time. So it’s one of those things… you got to find out, which if any file can even be modified

I am not associated with helium or any manufacturer. But that is a manufacturer specific inquiry for the customer service and support of the manufacturer. Don’t do anything that could void your warranty, without contacting them first…

Thank you very much for your reply! Very helpful

joselinoaraujo commented 2 years ago

I bought a used Pisces P100 Miner on ebay on January 10, 2022, I received it at my house on January 15, and I changed my location to Portugal on January 15.

The problem is that almost 2 weeks have passed and the miner is only making challenges, he is not communicating with any miner in my area.

The miner is abroad at 10 Meters Height and has a 6 DBi antenna. It has no obstacles. Miner is connected to internet correctly, antena connect correctly and port 44158 is open in the router

Transmit Scale 1.00

Sync Status Synced Last Updated 2 week ago

you can check in this web: https://explorer.helium.com/hotspots/11Sv8WiH4mNKwugYnEFxm7qEqcs1jJGWZJaEgPEF3gW7nY1bgM2

I don't understand, the miners in my area are working very well, but my Pisces P100 is not working properly. Dashboard Error Logs.txt Miner Logs.txt Witness Logs.txt

I already change the rssi_offset to : -230.4

Someone can help me?

serbyxp commented 2 years ago

🤔 it’s the right frequency plan, maybe look in the explorer app and find some hotspots in Portugal and try to connect to them and see if that helps 🤷‍♀️ … if you got access to the cli ( console) then “miner peer connect /p2p/address here” or even a “miner peer ping /p2p/address here” try to connect to as many peers in your area that might do something Atleast it’ll let you know if you can reach other people near you you can see who your connected to with “miner peer session” and check your peer book with “miner peer book -s” I was only creating challanges for a while… also make sure your firmware is the latest. “Miner versions” it should say 2022.01.27.0 permanent, if not upgrade it, might take a while to upgrade fully , will take Atleast 3 hours ( Atleast for me it did) gateway_v2 ledger took a long time to populate all the files in ledger.db, when it finished it was fully synced … no idea what else to tel you heh, we’ve all been in a similar situation here … just make sure you are up to date with the latest firmware and see if any of that changes your challanges only issue … otherwise We all stuck here with failed to dial errors and challanger not found errors . Even though it’s second hand, you might still get support from Pisces .. won’t hurt to ask them, don’t mention it’s used …

joselinoaraujo commented 2 years ago

Hello I don't know how i do this.

I check another problem is error logs:

2022-01-30 12:07:34.812 [error] <0.1606.0>@blockchain_ledger_v1:config:1335 Supervisor blockchain_state_channel_sup had child blockchain_state_channels_client started with blockchain_state_channels_client:start_link(#{swarm => <0.1504.0>}) at <0.1622.0> exit with reason bad argument in call to rocksdb:get(#Ref<0.1245402570.1818361873.46240>, #Ref<0.1245402570.1818361873.46241>, <<"$var_sc_grace_blocks">>, []) in blockchain_ledger_v1:config/2 line 1335 in context child_terminated

2022-01-30 12:07:34.811 [error] <0.1622.0>@blockchain_ledger_v1:config:1335 CRASH REPORT Process blockchain_state_channels_client with 0 neighbours crashed with reason: bad argument in call to rocksdb:get(#Ref<0.1245402570.1818361873.46240>, #Ref<0.1245402570.1818361873.46241>, <<"$var_sc_grace_blocks">>, []) in blockchain_ledger_v1:config/2 line 1335

what is this? : Supervisor blockchain_state_channel_sup

and what is this? @blockchain_ledger_v1:config:1335 CRASH REPORT Process blockchain_state

joselinoaraujo commented 2 years ago

How i can update miner version to 2022.01.27.0?

in dashboard tool last version is: Updating from: miner-arm64_2022.01.12.1_GA Updating to: miner-arm64_2022.01.12.1_GA

not have any 2022.01.27.0 update

I can update in putty? Do you have comands for it?

serbyxp commented 2 years ago

@joselinoaraujo that I couldn't really tell you on Pisces, since I don't know anything about what it has on it software wise.., I have access to my docker environment and I just do a docker stop miner and remove the old image and let it pull the new image .. It should be set to pick up the arm-latest, But I know that a lot of manufacturers have it set to pull the miner:-version_GA << GA I did not see get uploaded to the Quay repo for helium .. https://quay.io/repository/team-helium/miner?tab=tags .. when I do a docker run. I have a docker-compose file also that does a auto pull on restart. I know there is a way with the "miner upgrade command but I never done it that way... I think the "miner install version_here " or "miner upgrade version_here" commands might do it, again I never done that so I have no clue Maybe just restarting might trigger the manufacturers auto upgrade software, If they haven't auto pushed it to you already, It might be due to the fact that a _GA release wasn't uploaded so when your manufacturers auto update searches the repo it only finds the _GA image which seems to be the one you mentioned of 2022.12.1_GA, I saw here that they were going to post it but its been some days since this https://engineering.helium.com/2022/01/28/blockchain-release-sync-improvements.html ( under plan on the bottom) There might be a reason for that?? But Your not missing out on anything special... still pulling failed to dial errors .. and didn't witness anything over night... yet I seen the little light turn on several times that tells me it received or sent things... but yeah if your miner is set to auto pull the _GA then its never going to update automatically untill a newer _GA is uploaded to the repo, you will have to find out which software to update it , IF you change anything in that file then your going to mess up the default manufacturers settings, They might know more about the updates then what helium is letting the general public know, So I would stick with what they say or just do one manual update and not change the any auto update options. error.log.2022.1.27.latest.txt

Mmm a right when I was going to revert dev throw me a curve ball with beta-arm64 ^_^ on to the next one!

joselinoaraujo commented 2 years ago

@joselinoaraujo that I couldn't really tell you on Pisces, since I don't know anything about what it has on it software wise.., I have access to my docker environment and I just do a docker stop miner and remove the old image and let it pull the new image .. It should be set to pick up the arm-latest, But I know that a lot of manufacturers have it set to pull the miner:-version_GA << GA I did not see get uploaded to the Quay repo for helium .. https://quay.io/repository/team-helium/miner?tab=tags .. when I do a docker run. I have a docker-compose file also that does a auto pull on restart. I know there is a way with the "miner upgrade command but I never done it that way... I think the "miner install version_here " or "miner upgrade version_here" commands might do it, again I never done that so I have no clue Maybe just restarting might trigger the manufacturers auto upgrade software, If they haven't auto pushed it to you already, It might be due to the fact that a _GA release wasn't uploaded so when your manufacturers auto update searches the repo it only finds the _GA image which seems to be the one you mentioned of 2022.12.1_GA, I saw here that they were going to post it but its been some days since this https://engineering.helium.com/2022/01/28/blockchain-release-sync-improvements.html ( under plan on the bottom) There might be a reason for that?? But Your not missing out on anything special... still pulling failed to dial errors .. and didn't witness anything over night... yet I seen the little light turn on several times that tells me it received or sent things... but yeah if your miner is set to auto pull the _GA then its never going to update automatically untill a newer _GA is uploaded to the repo, you will have to find out which software to update it , IF you change anything in that file then your going to mess up the default manufacturers settings, They might know more about the updates then what helium is letting the general public know, So I would stick with what they say or just do one manual update and not change the any auto update options. error.log.2022.1.27.latest.txt

  • the last image that was producing decent results for me was the beta-arm release on 1.27.2022 I was witnessing and beaconing, But still getting dial errors, I think I will revert to that. I'm not sure if that is the case for anyone else.. But Instead of re installing that gateway_v2 all over again I'm going to see if its possible to just rename it and revert to the one that was working for me. I Figure if no one else is on it, that the few that are vs the manufacturers that are on the _GA , we might be missing out on being on different versions, better I think to be on the same version as the majority of the network... I don't know I testing everything out so don't go by what I do. when it comes to updates. Just makes logical sense if all manufacturers are on _GA to stay on _GA since the majority of the hot spots will be on the same version.

Mmm a right when I was going to revert dev throw me a curve ball with beta-arm64 ^_^ on to the next one!

I don't understand support of Pisces, because no one help me and no one reply my emails and tickets.

I want update the miner, but i don't steps of comands in Putty

Thank You

serbyxp commented 2 years ago

Hmm 🤔 see if in putty what happens if you do “docker ps” and see what the image name that’s running , should be “miner”, then try “docker stop miner” and go to the quay repo above , click the grey download button on the side of the image you want, then copy the URL, of the docker pull “quay.io/… “ and maybe try docker run “quay.io/…” 🤷‍♀️ I’m not the best person to ask heh but I doubt anyone else is going to suggest anything. Again, I spent over a month trying to figure this out and my hardware software is diffrent then yours so results may vary… seeing as I have no idea what your miner is doing… or your containers Ebus packet forwarded … here is the direction I followed. https://docs.helium.com/mine-hnt/full-hotspots/become-a-maker/basic-miner-operation/

https://docs.helium.com/mine-hnt/full-hotspots/become-a-maker/docker-integration

First thing that popped up on Google which corresponds to the above

https://www.whitesourcesoftware.com/free-developer-tools/blog/update-docker-images/

I mean this is already beyond the scope of this thread. So I mean you might want to open a new one or look through other issues etc for updating etc… don’t want to turn this thread into something it’s not meant to be…

i just updated to the beta-arm64 and I will post my error logs here.

note that you might f up everything on your miner… doing any sort of changes , but aslong as you don’t mess with the main miner software and just update the image you should be ok, and when Pisces updates automatically all your settings and changes will most likely revert back to whatever they push to their “fleet” Remember that this update is going to take over 3 hours to install the gateway_v2 so don’t even look at it turn it on and off for a long while , by turning it off or restarting it you run the risk of it reverting back to “stock “

As far as the beta-arm 2022.30.1 running for 30 mins now I haven’t received a witness yet ( I did finally moments before I updated , on the arm-latest 2022.27.1

only errors seem typical for a restart… 2022-01-30 19:08:30.401 [error] <0.19526.1> gen_server <0.19526.1> terminated with reason: killed 2022-01-30 19:08:30.401 [error] <0.19526.1> CRASH REPORT Process <0.19526.1> with 0 neighbours exited with reason: killed in gen_server:decode_msg/9 line 481 2022-01-30 19:10:30.371 [error] <0.1568.0>@libp2p_transport_proxy:connect_to:{69,13} failed to dial proxy server "/p2p/116keetqW1WHx9nmRSDNmDQgXg3d5f9uYhmvqaS1JBjyLnHhcUJ" not_found 2022-01-30 19:39:03.098 [error] <0.2797.0>@libp2p_stream_relay:handle_server_data:{193,13} fail to pass request {libp2p_relay_bridge_cr_pb,"/p2p/11L17bznU6kbLxqwbkftNKR3yFtrtmduBPiib4YCGJaVnQmZbeT","/ip4/75.179.17.189/tcp/44158"} Server not found

I mean the first peer is in Lithuania that could be one issue but the ip is in Akron Ohio so I dunno why I can’t reach them

joselinoaraujo commented 2 years ago

Thank you

mschroebel commented 2 years ago

Is there really a reason for the miner to be checking these status values every 1 second? Goes on ad-naseum. Seems just a hair too often. quay.io/team-helium/miner:miner-arm64_2022.02.22.0_GA on Pisces

Feb 24 21:06:44 raspberrypi systemd[1]: Started Reboot flag checker. Feb 24 21:06:44 raspberrypi systemd[1]: reboot-check.service: Succeeded. Feb 24 21:06:44 raspberrypi systemd[1]: Started VPN checker. Feb 24 21:06:44 raspberrypi systemd[1]: vpn-check.service: Succeeded. Feb 24 21:06:44 raspberrypi systemd[1]: Started New WiFi Config checker. Feb 24 21:06:45 raspberrypi systemd[1]: wifi-config-check.service: Succeeded. Feb 24 21:06:45 raspberrypi systemd[1]: Started Miner updater checker. Feb 24 21:06:45 raspberrypi systemd[1]: update-miner-check.service: Succeeded. Feb 24 21:06:45 raspberrypi systemd[1]: Started BlueTooth Advertise service start/stop flag checker. Feb 24 21:06:45 raspberrypi systemd[1]: bt-service-check.service: Succeeded. Feb 24 21:06:45 raspberrypi systemd[1]: Started Password reset checker. Feb 24 21:06:45 raspberrypi systemd[1]: password-check.service: Succeeded. Feb 24 21:06:45 raspberrypi systemd[1]: Started WiFi service start/stop flag checker. Feb 24 21:06:45 raspberrypi systemd[1]: wifi-service-check.service: Succeeded. Feb 24 21:06:45 raspberrypi systemd[1]: Started Miner service start/stop flag checker. Feb 24 21:06:45 raspberrypi systemd[1]: Started Clear Blockchain checker. Feb 24 21:06:45 raspberrypi systemd[1]: miner-service-check.service: Succeeded. Feb 24 21:06:45 raspberrypi systemd[1]: clear-blockchain-check.service: Succeeded. Feb 24 21:06:45 raspberrypi systemd[1]: Started Dashboard updater checker. Feb 24 21:06:45 raspberrypi systemd[1]: update-dashboard-check.service: Succeeded. Feb 24 21:06:45 raspberrypi systemd[1]: Started Packet Forwarder service start/stop flag checker. Feb 24 21:06:45 raspberrypi systemd[1]: pf-service-check.service: Succeeded. Feb 24 21:06:46 raspberrypi systemd[1]: gps-check.service: Succeeded. Feb 24 21:06:46 raspberrypi systemd[1]: temp-check.service: Succeeded. Feb 24 21:06:46 raspberrypi systemd[1]: bt-check.service: Succeeded. Feb 24 21:06:46 raspberrypi systemd[1]: Started Bluetooth status checker. Feb 24 21:06:46 raspberrypi systemd[1]: miner-check.service: Succeeded. Feb 24 21:06:47 raspberrypi systemd[1]: Started BlueTooth Advertise service start/stop flag checker. Feb 24 21:06:47 raspberrypi systemd[1]: bt-service-check.service: Succeeded. Feb 24 21:06:47 raspberrypi systemd[1]: Started GPS status checker. Feb 24 21:06:47 raspberrypi systemd[1]: Started WiFi Status checker. Feb 24 21:06:47 raspberrypi systemd[1]: wifi-check.service: Succeeded. Feb 24 21:06:47 raspberrypi systemd[1]: Started CPU Utilization checker. Feb 24 21:06:47 raspberrypi systemd[1]: Started Packet Forwarder checker. Feb 24 21:06:47 raspberrypi systemd[1]: pf-check.service: Succeeded. Feb 24 21:06:47 raspberrypi systemd[1]: gps-check.service: Succeeded. Feb 24 21:06:47 raspberrypi systemd[1]: cpu-check.service: Succeeded. Feb 24 21:06:47 raspberrypi systemd[1]: bt-check.service: Succeeded. Feb 24 21:06:49 raspberrypi systemd[1]: Started BlueTooth Advertise service start/stop flag checker. Feb 24 21:06:49 raspberrypi systemd[1]: bt-service-check.service: Succeeded. Feb 24 21:06:49 raspberrypi systemd[1]: Started Reboot flag checker. Feb 24 21:06:49 raspberrypi systemd[1]: reboot-check.service: Succeeded. Feb 24 21:06:50 raspberrypi systemd[1]: Started WiFi service start/stop flag checker. Feb 24 21:06:50 raspberrypi systemd[1]: wifi-service-check.service: Succeeded. Feb 24 21:06:50 raspberrypi systemd[1]: Started Miner service start/stop flag checker. Feb 24 21:06:50 raspberrypi systemd[1]: miner-service-check.service: Succeeded. Feb 24 21:06:50 raspberrypi systemd[1]: Started Packet Forwarder service start/stop flag checker. Feb 24 21:06:51 raspberrypi systemd[1]: pf-service-check.service: Succeeded. Feb 24 21:06:51 raspberrypi systemd[1]: Started BlueTooth Advertise service start/stop flag checker. Feb 24 21:06:51 raspberrypi systemd[1]: bt-service-check.service: Succeeded. Feb 24 21:06:51 raspberrypi systemd[1]: Started Bluetooth status checker. Feb 24 21:06:52 raspberrypi systemd[1]: bt-check.service: Succeeded. Feb 24 21:06:53 raspberrypi systemd[1]: Started BlueTooth Advertise service start/stop flag checker. Feb 24 21:06:53 raspberrypi systemd[1]: bt-service-check.service: Succeeded.

mschroebel commented 2 years ago

MAybe you have a hardware problem? Maybe the LoRa receiver in your miner is dead and can't receive anything? Maybe because the antenna port was open or shorted for a moment and transmission of LoRa Beacon completely reflected into antenna port and destroyed Receiver.

There are so many possibilities. Without further diagnostic data it does not matter if your ports are open and your internet is fast. It is impossible to tell what is wrong and why you are not witnessing other stations. Maybe your antenna is full with rain water and not working anymore?!? Who can tell?

Search youtube with "Squirrel fills Antenna with Acorns" also a possibility why RF system can fail.

Could say exactly same thing about the logs. Full of junk that means nothing. For instance, if validators won't respond to a p2p request, why try then fill the log with things that are not errors yet say they are? Instead of everyone wondering what the K is going on when they startup a new miner, how about a simple informational log showing the status of things, what's being waited for, what the whatever. I had a miner stuck doing I don't know what for 5 days, finally fought it enough that the fast sync actually did something and it seems like it's getting close --- but for 5 days all I could see was it stuck 95% of the way, going online and offline for who knows what, eating CPU like the devil doing what? Totally lack of information. Would be better if has nice clear status messages (comma separated list here) like Fast Sync 40% done, 50% done, ... fast sync done, now catching up on last 500 blocks, 400 to go, 300 to go, offline until synced. At one point ort 44158 wasn't open on the pi, why, if closed should be a reason shared. I stopped the miner, did a fast sync, and 44158 opened, why? I mean geez, send some info. Keep the other logs full of the details for yourselves in debugging. Your bank doesn't give you interest statements by the second .. it's overload. Simple, summary data. Of course if you like all of the same questions over and over, keep it this way.

vanjulio commented 2 years ago

I have been disassembling my miner lately looking for exactly that squirrel nut stash. Hadn't considered the nuts could be inside the freaking antenna!

MAYKEPIZO commented 2 years ago

Check recumbent sapphire snake and huge bamboo fox. This on the upper image is recumbent sapphire snake. Both of them set up in the middle of cell station with 10 or more panel antennas around them. Both were working fine. After the update and me having to move huge bamboo fox due to low transmit scale it stopped witnessing others. This is huge bamboo fox. IMG_20211201_122819

You can not tell me that this is not a good set up.

Also check

man ... u have pices miner its crap .. i have same miner only problem ... i install bobcat indoor and antenna was in tvättstuga ,,gss what ,,next day i had 10 witnees and my out dor sht pices miner 2 .. i tryid allmost every thing , new sd card new saw filter and same sht only sht miner

mschroebel commented 2 years ago

I think the antenna ought to be above the pole or farther away otherwise interferes with RF reception/transmission. Cut pole, or move up

serbyxp commented 2 years ago

Heh your antena is scared of heights ? Is that pole properly ground..

MAybe you have a hardware problem? Maybe the LoRa receiver in your miner is dead and can't receive anything? Maybe because the antenna port was open or shorted for a moment and transmission of LoRa Beacon completely reflected into antenna port and destroyed Receiver. There are so many possibilities. Without further diagnostic data it does not matter if your ports are open and your internet is fast. It is impossible to tell what is wrong and why you are not witnessing other stations. Maybe your antenna is full with rain water and not working anymore?!? Who can tell? Search youtube with "Squirrel fills Antenna with Acorns" also a possibility why RF system can fail.

Could say exactly same thing about the logs. Full of junk that means nothing. For instance, if validators won't respond to a p2p request, why try then fill the log with things that are not errors yet say they are? Instead of everyone wondering what the K is going on when they startup a new miner, how about a simple informational log showing the status of things, what's being waited for, what the whatever. I had a miner stuck doing I don't know what for 5 days, finally fought it enough that the fast sync actually did something and it seems like it's getting close --- but for 5 days all I could see was it stuck 95% of the way, going online and offline for who knows what, eating CPU like the devil doing what? Totally lack of information. Would be better if has nice clear status messages (comma separated list here) like Fast Sync 40% done, 50% done, ... fast sync done, now catching up on last 500 blocks, 400 to go, 300 to go, offline until synced. At one point ort 44158 wasn't open on the pi, why, if closed should be a reason shared. I stopped the miner, did a fast sync, and 44158 opened, why? I mean geez, send some info. Keep the other logs full of the details for yourselves in debugging. Your bank doesn't give you interest statements by the second .. it's overload. Simple, summary data. Of course if you like all of the same questions over and over, keep it this way.

I mean those logs are not from the helium miner image, looks like a “supervisor” log from something like balenaOS or something of the sort. But ,

i have noticed with the latestet update I posted it somewhere else ,

the miner logs are saying observed address of the miner has all sorts of diffrent port numbers , and that’s from inside the miner image, I didn’t touch any settings , just new latest and it just decided to say my port is whatever port it feels like , it says my observed IP address is all sorts of things that it isn’t, I haven’t been relayed or anything for weeks, I figured out the miner dial errors, and everything was fine until latest update 3.7.22 , heh the 2.28.22 was working fine, why they break it one week later 😂🤷‍♀️

deebeegee commented 2 years ago

Woops, sorry. Latest

witness log.txt error log.txt

Can i please ask how you accessed these log files? My bobcat miner goes into endless reboot loops when installed in one location but not in others and I'm desperately trying to decipher if there's something coming over the airwaves causing it distress