HIP 83: Restore First to Respond Witness Rewarding

[vincenzospaghetti]

HIP 83: Restore First to Respond Witness Rewarding

• Author(s): @BFGNeil, @mawdegroot
• Start Date: 2023-05-05
• Category: Economic, Technical
• Original HIP PR: #605
• Tracking Issue: #632
• Voting Requirements: veIOT Holders

Summary

Currently, the Proof-of-Coverage Oracles collect all the witnesses for a beacon and randomly reward a selection of 14 witnesses. This HIP proposes to revert to rewarding the first 14 Hotspots responding to a beacon, incentivizing the most useful Hotspots to sensor traffic by prioritizing the low latency connections that sensors need for uplinks, downlinks, and join requests to work correctly.

Motivation

Rewarding witnesses that are the first to respond will incentivize Hotspots to provide the low-latency connection that the sensors desperately need.

Sensors use uplinks to transfer their data over the Helium network. For a sensor to join the Helium network, it must perform a handshake that requires both an uplink and a downlink. As a power-saving measure, a sensor often has a limited time window to listen for the downlink.

Because the sensor only listens for the downlink for a limited time, the Helium LNS and the Hotspots must minimize latency to ensure the Hotspot can deliver downlinks to sensors within the narrow sensor listening window.

The Helium network originally rewarded the fastest witnesses and moved to a random selection for several technical reasons that no longer apply to the Oracle-based POC architecture introduced as part of HIP 70.

Technical limitations included, but are not limited to:

• Regular retransmissions on the libp2p network left Hotspot unable to deliver witness reports.
• Hotspots struggled to sync the Helium blockchain before Validators and needed more time to determine which challenger to deliver a witness to.

Rendered View

https://github.com/helium/HIP/blob/main/0083-restore-first-to-witness.md

Supporting Code

https://github.com/helium/oracles/compare/main...mawdegroot:oracles:mg/first-to-respond-witnessing

[disk91]

Don't really see the relation with the expected behavior:

• hotspot connectivity is a consequence, not really a choice
• hotspot response time to witness is more related to the distance to Oracle than the connectivity type.
• witness response time is not involved in packet selection
• router should get best packet, not first packet, router is time constrained on reception, whatever is the hotspot response time
• Join Accept windows is 5s .... so this Hip about ms is non sense for this.
• Downlink windows is 2s, tens of Ms don't change anything, wait windows is about 500ms
• This will make some hotspot not earning due to good but not fast enough connectivity and will conduct these hotspot owner to disconnect miner. The second hand market is already flooded so the only consequence of this will be a reduction of the coverage.
• High latency HS are usually 4G, most of 4G are well placed hotspot (otherwise you have wifi, Eth...) so this will basically impact some of the best Hotspot in term of coverage

[JoToSystems]

Reward Systems in general are implemented to drive the behaviour/expansion of a system in the direction which is in the best interests of that system.
Therefore I would suggest the Reward System in this HIP should be structured to match the best outcome for the Helium network and that would be to maximise the coverage area. To to do this would be to maximise the number of Hotspots and to reduce the overlap (one way to think of this would be to reduce the number of Hotspots per Hex). With the creation of HIP84, this HIP(83) is now redundant and the Rewards System should be proposed in HIP84 to match the outcomes which are determined in that HIP.

[jsloane]

the low-latency connection that the sensors desperately need.

Do you have any evidence to support this?

Because the sensor only listens for the downlink for a limited time, the Helium LNS and the Hotspots must minimize latency to ensure the Hotspot can deliver downlinks to sensors within the narrow sensor listening window.

Do you have any evidence to show that this is a problem on a wider scale? If high latency hotspots are dropping sensor packets, wouldn't it be better to reduce the PoC window to match the network timeout? That way these hotspots will not be rewarded, according to the network timeout configuration. Most developed countries have low latency fibre to the premises available in population dense areas, so is this really a problem?

I think this is a bad change that will result in rewards being centralised to those that can configure low latency networks, hardware overclocking, etc. Some manufacturers who cheap out on low spec hardware may lose rewards, when these units are still fit for purpose. The casual users who don't know anything about this will eventually turn their perfectly fine hotspots off, resulting in poorer coverage and a poorer network.

[TXR72]

This HIP would be a massive step backwards in coverage. It skews rewards towards hotspots with the highest internet speed / lowest latency available - typically urban areas, which are already oversaturated and should experience a disincentive rather than additional incentives.

Conversely, the hotspots doing the most for extending coverage - those further away from urban centres, hex "pioneers", etc. - usually have less fancy internet connections, e.g. 4G. They also typically invested a lot more in their setups. We need those hotspots in particular in order to grow the network in the right places, yet, their rewards would massively suffer.

This is counterproductive and feels like an attempt to concentrate rewards towards a few, super-connected hotspots, with little regard for what that means for the network.

[Blue-S0ul]

I have this miner in London, UK. The miners is with a 8 DBI antenna. I use this HeliumGeek app which shows what my miner is doing. I think and believe that I am not being treated very well because of this useless random selection. My miner is there and antenna is there. It is there to be used and to provide service which it is doing. My internet is 1GB download and 150 Mb upload. Miner is connected to the modem via cable. When I test the speed, it's 50 upload and 40 download. Latency is 30 and this can change every time I test. I have this app called Helium Geek. It has 3 important icons. Beaconed, Witnessed and Unselected. The Beacon I sent is always being picked up by 14 other hotspots. The Witnessed icon is what is annoying me because here I see it go up to 150 minutes without rewards. The unselected icon, which is in grey is never above 10-15 minutes. This means my hotspot is seeing many many other hotspots but I don't get selected to be rewarded. This is what you guys need to address. People with good hotspots should get good rewards. My antenna is up on 30 m high. It is seeing up to 700 other hotspots but does not get rewarded.

So HIP 83 can and should be good.

[disk91]

So … basically there is 700 hotspots around you and you would like to be selected more often than the other one … because you are really important and the other are useless if i understand well. i don’t understand your arguments ? You have a standard setup, a standard ISP connection but you think it is better than the 700 other hotspot around .. My two cents, with a such density of hostpot, if HIP83 where applied, you will never see a witness anymore …

[Blue-S0ul]

So … basically there is 700 hotspots around you and you would like to be selected more often than the other one … because you are really important and the other are useless if i understand well. i don’t understand your arguments ? You have a standard setup, a standard ISP connection but you think it is better than the 700 other hotspot around .. My two cents, with a such density of hostpot, if HIP83 where applied, you will never see a witness anymore …

What I am trying to say is that does it mean anything to me to have my antenna there which is spotting so many and is not getting a thing out of it? I also wonder what is going on with all the miners that seem to be red but are around me.

[anyn99]

Basically ALL LoraWAN Applications do not care about latency!

There is a reason for that, it is meant to be low power, not low latency.... Have the authors any idea of LoRa in general? Then they should be aware of the fact that latency is not a problem.

[BFGNeil]

Basically ALL LoraWAN Applications do not care about latency!

There is a reason for that, it is meant to be low power, not low latency.... Have the authors any idea of LoRa in general? Then they should be aware of the fact that latency is not a problem.

Hi, thats not true, remember Join Confirmations, Downlinks and ACK's are an important part. These need to be fast to work.

We also have Class C support now with chirp stack, bidirectional is an important part

[Crash0v3r1de]

This HIP would be a massive step backwards in coverage. It skews rewards towards hotspots with the highest internet speed / lowest latency available - typically urban areas, which are already oversaturated and should experience a disincentive rather than additional incentives.

Conversely, the hotspots doing the most for extending coverage - those further away from urban centres, hex "pioneers", etc. - usually have less fancy internet connections, e.g. 4G. They also typically invested a lot more in their setups. We need those hotspots in particular in order to grow the network in the right places, yet, their rewards would massively suffer.

This is counterproductive and feels like an attempt to concentrate rewards towards a few, super-connected hotspots, with little regard for what that means for the network.

This is actually a really good point. I'm still getting shafted by not having many hotspots around me purely because I'm in a mountain valley. Now I'll get shafted for not many neighboring hotspots AND slow latency to the collectors.

Glad I diverted my spending money away from this even more after reading this HIP.

[Ryan-Goldstein]

This is actually a really good point. I'm still getting shafted by not having many hotspots around me purely because I'm in a mountain valley. Now I'll get shafted for not many neighboring hotspots AND slow latency to the collectors.

Glad I diverted my spending money away from this even more after reading this HIP.

If the beacons you witness have 14 or fewer witnesses (common in unsaturated, rural areas), then HIP 83 will not affect you at all. If the beacons you witness have more than 14 witnesses, then that's not as unsaturated an area as you may think.

[Crash0v3r1de]

This is actually a really good point. I'm still getting shafted by not having many hotspots around me purely because I'm in a mountain valley. Now I'll get shafted for not many neighboring hotspots AND slow latency to the collectors. Glad I diverted my spending money away from this even more after reading this HIP.

If the beacons you witness have 14 or fewer witnesses (common in unsaturated, rural areas), then HIP 83 will not affect you at all. If the beacons you witness have more than 14 witnesses, then that's not as unsaturated an area as you may think.

o cool, I'm still rewarded less regardless

[Ryan-Goldstein]

o cool, I'm still rewarded less regardless

Ok, so you're in an oversaturated area with more than 14 hotspots providing coverage for a single sensor in that area. It'd benefit the network for some of those hotspots to be redeployed to less saturated areas anyway, to expand total network coverage.

[Crash0v3r1de]

o cool, I'm still rewarded less regardless

Ok, so you're in an oversaturated area with more than 14 hotspots providing coverage for a single sensor in that area. It'd benefit the network for some of those hotspots to be redeployed to less saturated areas anyway, to expand total network coverage.

I didn't know less than 8 meant more than 14 - thanks for the quick maths

[Ryan-Goldstein]

I didn't know less than 8 meant more than 14 - thanks for the quick maths

If HIP 83 would negatively affect your earnings if implemented, that means the beacons you witness necessarily have to have more than 14 witnesses.

[lthiery]

I'm a little late to the game here, but was it ever considered starting a cut-off 1 (or 2) seconds after the first receipt?

I think this would mirror the LoRaWAN spec better as the first RX window is 1 second for roundtrip. So anyone that shows up a full second later would be "useless" from a coverage perspective due to latencies.

Currently, this cutoff feels too arbitrary.

[disk91]

Join request are 5s - 6s windows so basically you currently wait more than a single second for this type of packets ( around 2s) and uplink with no ack (most common type of packets) does not have time constraints. Also 2s wait is a standard windows for reception of a larger amount of packets.

[VeryOuch]

Seeing this modification in action, it is apparent that good placements are not rewarded any more. In other words, in my area: mediocre placements with low-latency internet are up 2x (taking almost any beacon they can "hear") and near-excellent placements with medium-latency internet are 2x down ("hearing" much more beacons but being selected for almost none, due to the latency). I have very high and rare placements (e.g. 100m and 60m) that used to perform outstandingly but I cannot change the connection latency.

[FieserKiller]

people are complaining about the effects while the status of this HIP is approved and not deployed. So is it deployed by now and status tracking is out of date or is it still in the pipeline and when will it arrive?

[BFGNeil]

people are complaining about the effects while the status of this HIP is approved and not deployed. So is it deployed by now and status tracking is out of date or is it still in the pipeline and when will it arrive?

It's live, been live for a few days now, it went live the day after the hip vote ended.

[waveform06]

@FieserKiller I actually thought that had been updated. Yes it is active, I will create a change request for that to be updated.

[jsloane]

So HIP 83 can and should be good.

@Blue-S0ul how's your London hotspot performing now that this change has been implemented, out of interest.

[Firaso123]

I already invested a lot on my setup and location to reach max possible hotspots so I ended up having very bad ROI. Each time I say fine, we are hear to support the project espicially in the initial phase. But each time I feel more none is caring about us, people who invested a lot for a good network coverage. After this HIP my rewards reduced to half. Do u know why? This is because I've chosen the location carefully for best coverage and didn't conecntrate on the internet connection. And now who has normal location but good internet connection are being higher rewarded. Frustrating :-/

[RuDee-BG]

This is my neighbour, from 300-400 meters away in a saturated urban area: 10x times increase! Mine was mining around 450-500, now it mines 5x lower, ie around 120-140

[waveform06]

HIP 83 was approved by the community on 12th July 2023 with a passing rate of 70.3% https://realms.heliumvote.com/dao/iot/proposal/H5mGJg9927DBRr1NH64VVk6hoSTJdnAC5kngGxgvumUS

[Firaso123]

Honestly, I find the voting system unfair as one wallet has 25% of the whole votig power! The community is not only who is holding the currency but also who is participating in the network as well. This part of the community has been always neglected in the Helium voting system :-/

[RuDee-BG]

Honestly, I find the voting system unfair as one wallet has 25% of the whole votig power! The community is not only who is holding the currency but also who is participating in the network as well. This part of the community has been always neglected in the Helium voting system :-/ Moreover, can you please give us one example of a HIP had not passed in the authority (sorry "the cummunity") voting system?

Yes, just like anything else in the world, who has the more money pulls the strings... i just manage to get my lattency under 2ms, and still i am far away from my previous rewards..

[Ryan-Goldstein]

And what would be your proposed solution? This has been discussed ad infinitum on the Helium Discord. The one alternative proposal that keeps coming up is "one hotspot one vote", and all that would do is give control of the Helium network to hotspot hosting companies and/or anyone willing to buy up a large quantity of hotspots and/or any rogue manufacturer that decides to onboard a large number of hotspots to tip any vote in their favor.

The current system is the most fair implementation the Helium Network has had so far. With our current voting system, the most voting power is given to those willing to live with the outcome of their vote the longest (staking IOT for 4 years provides 100x voting power, for instance).

It's easy to point out problems and difficult to propose a good solution that doesn't cause more problems than it solves.

[waveform06]

@RuDee-BG HIPs 72,80 and 81 all failed to be voted in, see https://github.com/helium/HIP/blob/main/README.md Usually HIPs go up to vote because we think they will pass or we don't know if they will pass or fail. Its unlikely we will put forward to vote HIPs we think will fail to get passed. Usually the evidence they would fail is apparent. HIP 72 and 83 were both unknowns. Most bad ideas that probably would fail don't get as far as the voting stake, some are withdrawn as a newer "better" HIP is released eg HIP 78 was replaced with 80/81 and they were replaced with 88/89

There is already a proposal to change this HIP and another proposal to enable hotspots to compete on a more level speed and response time.

[jsloane]

Most bad ideas that probably would fail don't get as far as the voting stake, some are withdrawn as a newer "better" HIP is released eg HIP 78 was replaced with 80/81 and they were replaced with 88/89

There is already a proposal to change this HIP and another proposal to enable hotspots to compete on a more level speed and response time.

So should someone have introduced a HIP better than this one, earlier? Who's responsible for doing this? Numerous people, myself included, detailed in this github issue what was going to happen when this HIP went live, but were essentially ignored. We don't all have time to be writing better HIPs.

[disk91]

Most bad ideas that probably would fail don't get as far as the voting stake, some are withdrawn as a newer "better" HIP is released eg HIP 78 was replaced with 80/81 and they were replaced with 88/89 There is already a proposal to change this HIP and another proposal to enable hotspots to compete on a more level speed and response time.

So should someone have introduced a HIP better than this one, earlier? Who's responsible for doing this? Numerous people, myself included, detailed in this github issue what was going to happen when this HIP went live, but were essentially ignored. We don't all have time to be writing better HIPs.

I fully agree with your statement. Fortunately i found some hours during my vacations for that and you can take a look on my github on the hip starting XX that is a proposal for having a reception windows.

[Delitants]

Hello? Why official, now Nova owned FreedomFi gateways got banished with this undertested HIP due to 5G boxed required to be encrypted and therefore slower, and literally nobody gives a rat's behind?

[VeryOuch]

I have decided to disconnect 12 of my 14 hotspots with very good placements (almost all higher than 30 metres), as the rewards were decreased by 40% (stable internet but latency a bit over 15ms). I am fed up and will not invest any more. I will start dismounting the antennas tomorrow.

[RuDee-BG]

I have decided to disconnect 12 of my 14 hotspots with very good placements (almost all higher than 30 metres), as the rewards were decreased by 40% (stable internet but latency a bit over 15ms). I am fed up and will not invest any more. I will start dismounting the antennas tomorrow.

The low latency is really is good, but even thou i manage to get my latency under 2 ms. Have a good hex, and my earnings are 40% still down. The neighbour with super bad hex and ok latency has 10X increase/1000%

[lthiery]

@RuDee-BG it could be down to your hardware which is what @Delitants shows in their chart.

This is the reason I am interested in the approach being worked on by @disk91 which avoids these millisecond optimizations that end up being out of the users control to some extent.

[RuDee-BG]

@lthiery Well, ur right, but that big of difference?! Between bobcat and kerlink?

[Firaso123]

I was investigating with my hotspots (different brands) the last 10 days via swapping the locations and I think it's a mix of all.

• Internet connection speed.
• Internet provider.
• Router and connection to router (Wifi/ cable).
• Brand of your Hotspot. At the end still wondering, why would a LoraWan network seek for such a very low latency network!? Are we going to host Gamers in our network!!!? I agree quicker is better but not at the cost of other aspects like location/ power consumption/ costs etc. (Installing a Ferrari engine for a Mini Hatch car won't improve the overal performance)

[abhay]

For folks who haven't been seeing this work, there's an update to the rules of witness selection being worked on here: https://github.com/helium/HIP/pull/749

[HeliosHyperion1111]

Basically ALL LoraWAN Applications do not care about latency!

There is a reason for that, it is meant to be low power, not low latency.... Have the authors any idea of LoRa in general? Then they should be aware of the fact that latency is not a problem.

nope they don't, they are just basic skill sensor providers. most of them in the Helium team also think ''amplifiers etc.'' are a method to cheat the network, had you been following Sorin you would see that's not the case of ''cheating'' but rather making your signal, as strong/efficient and clean as possible it can get for the particular area, this way that creates a unique valid coverage. the Hip 83 doesn't talk anything about selected sensors picking up all of sudden the fast low latency hotspots, it has no methods to do it because they will always pick the low latency ones regardless of who thinks rewarding a particular hotspot more or not. there are so many red flags in this HIP 83 that you can point anything at and it all leads to centralisation of rewards. nothing about improving anything.

[HeliosHyperion1111]

I was investigating with my hotspots (different brands) the last 10 days via swapping the locations and I think it's a mix of all.

• Internet connection speed.
• Internet provider.
• Router and connection to router (Wifi/ cable).
• Brand of your Hotspot. At the end still wondering, why would a LoraWan network seek for such a very low latency network!? Are we going to host Gamers in our network!!!? I agree quicker is better but not at the cost of other aspects like location/ power consumption/ costs etc. (Installing a Ferrari engine for a Mini Hatch car won't improve the overal performance)

that would all drasticaly change, if they went to their original claims for the originality of the HIP 83 that supposedely claims ''to return to original proof of coverage'' if you go back in time, they had witness selection of 25 HOTSPOTS, yeah read that well, TWENTY FIVE. the only reason they went to 14 was because the network had problems keeping up with the blockchain and on top too many hotspots were onboard so they had to take measures temporary.

moving forward and we come back to this HIP 83, we see that if they had changed it to 25 selected hotspots on top remove the 200-300ms ECC chip obstacle by the makers, you would see things change dramaticaly for the better. the HIP 83 was done with no data to test if its good for the network or not, they slapped a heliumgeek map(showing who gets the big juicy pie and who is not ) and called it a day. yeah -30k hotspots, thats what we got, most of them gone offline in unique coverage areas like rural/suburban.

[HeliosHyperion1111]

@waveform06 so who made you the police to what's gonna be voted or not? since when we allow central figures ''their opinions of what might pass or might fail'' in a democratic voting system? or don't tell me it isn't democratic but dictatorial in a sense, because that's how it sounds. REDICULOUS.

disk91 does fantastic job trying to modify the fraudelent hip 83 that caused more damage than good.

[waveform06]

@HeliosHyperion1111 Police don't decide what is to be voted on, I don't decide what will or wont. I am doing some governance admin though. We do straw polls of people and we take advice of experts, as I said there is no point sending thru a HIP to vote that the community thinks will fail to pass or is impossible/illegal to implement. If the HIP author cannot persuade a few community members how can they persuade a majority?

HIP 83 moved what people like Sorin (https://www.youtube.com/watch?v=BRO7ziKL7-U) called "Proof of Luck" back to a form of "Proof of Coverage", Disk91's proposal looks to modify that to a mix of both scenarios.

[HeliosHyperion1111]

yes Sorin did say its proof of luck, but the way the network was back then, had 25 witnesses not 14 and surely didn't implement so strict latency selection of the witnesses like HIP 83 does. in fact back then it was much more balanced/fair because the witnessing was more focused to the RF signal itself rather the latency. and we didn't have oracles to tell us who is fast and who is not.

[BFGNeil]

yes Sorin did say its proof of luck, but the way the network was back then, had 25 witnesses not 14 and surely didn't implement so strict latency selection of the witnesses like HIP 83 does. in fact back then it was much more balanced/fair because the witnessing was more focused to the RF signal itself rather the latency. and we didn't have oracles to tell us who is fast and who is not.

Sure, when we wrote this hip we knew if we had said return to 25 it would automatically win. we wanted it to go through on merit alone. if 25 is needed, write an amendment now we have clear data.

[RuDee-BG]

Coincidence or pure luck...just not fair. My hotspot is the unlucky one, the other one i can see from my balcony ... Yeah, just bad luck.

[mrfizzy99]

A fix has been found!~ All will be right again soon!~ All thanks to @mawdegroot @nosmaster89

[Delitants]

A fix has been found!~ All will be right again soon!~ All thanks to @mawdegroot @nosmaster89

What's found? Where is it?

[Delitants]

???

[HeliosHyperion1111]

A fix has been found!~ All will be right again soon!~ All thanks to @mawdegroot @nosmaster89

What's found? Where is it?

i guess nothing has been found, looks like this guy is biased towards groot who is so against these fair changes.

[mawdegroot]

Several changes to the code have significantly improved the signing speed for the majority of makers. In preliminary testing the majority of makers now show a very similar signing performance. These changes will be included in the new gateway-rs v1.1.0 release which has been released to makers for testing. The new gateway-rs release will also include session keys which will significantly increase data transfer throughput and should free up time on the ECC chip for further improvements in high traffic areas.