Define settings definition strategy for device integration

[mullerch]

There are 3 different actors of settings definition identfied so far, each willing to set part of the configuration:

• the network provider : default gateways / routers
• the gateway manufacturer : update strategy / logs / keypair location
• the gateway user : region / alternative network / any other custom fancy change

Actually we have 4 different sources: default settings value in binary, default.toml, settings.toml, environment.

It would be good to define a strategy of what the purpose of each source is, or what manufacturers are supposed to use for their integration. This can be quite a mess with frequent app and os updates but it can also be very easy if it's well defined.

Proposition

As a manufacturer and integrator of such software, what generally provides the best flexibility/complexity ratio, is to have a config directory (as possible now) and use any .toml file in the directory with priority defined by alphabetically ordered file names (see https://developer-old.gnome.org/NetworkManager/stable/NetworkManager.conf.html for an example)

With that, we can define a convention for file names (no breaking change):

• default.toml: overrides of binary default. Mostly doesn't exist. Can be defined to ensure config does not change over binary update if nedded
• platform.toml (or gateway.toml, model.toml, anything starting with a char between d and s): manufacturer/platform specific settings
• settings.toml: user settings

A custom integration could also define numeric prefix to make the priority more obvious.

[shawaj]

not convinced, at least in a helium setting, that the user would need to change any settings. the design of helium network is that all user-configurable stuff (location assert/region) should come directly from the blockchain. the blockchain should be the single source of truth IMO for any settings that it can be.

having said that - your proposition doesn't break this...however, might need to include the ENV variables in there somewhere

[mullerch]

I agree, it would be best if all user settings are defined by the server/blockchain. Actually it's not the case for the region which has to be defined for many users.

[madninja]

hopefully not for long though.. with PoCv11 the region would be defined by the asserted h3 location, which is then mapped by chain variables into region and regional parameters

[pvolodin]

@shawaj > not convinced, at least in a helium setting, that the user would need to change any settings

In some regions the user anyhow will need to create/choose correct packet forwarder config file, if it is assumed that almost any gateway can be used with gateway-rs, right? The vast majority of 8-channel gateways on the US market are by default configured for FSB1 (channels 0-7). I don't think there's any way to make these changes automatically in a reliable way. There can be too many subtle differences in config file formats.

[madninja]

I agree that there's a lot of subtle details there.. Perhaps the best gateway-rs can do is expose the region for the asserted gateway as inferred by the chain. That way maker specific functionality can translate that to picking the right configuration files and settings?

[mullerch]

An alternative would be to use another daemon for the LoRa concentrator access, e.g. https://github.com/brocaar/chirpstack-concentratord/. It allows for channels configuration over an API.

[pvolodin]

@madninja

Perhaps the best gateway-rs can do is expose the region for the asserted gateway as inferred by the chain. That way maker specific functionality can translate that to picking the right configuration files and settings?

Yes, I also think so.

It also would be great if gateway-rs warns the gateway owner in an easily understandable way if there's any (potential) mismatch between gateway-rs and packet forwarder settings. This shouldn't be hard to implement, and this will unlock some opportunities for both helium and the guys already owning the gateways manufactured before.

[madninja]

An alternative would be to use another daemon for the LoRa concentrator access, e.g. https://github.com/brocaar/chirpstack-concentratord/. It allows for channels configuration over an API.

The only issue I've seen with this in the past is that trying to "own" the gwmp part of the stack outside of the maker is incredibly painful given the changes Semtech makes and the tuning makers do as part of manufacturing. I'm concerned this will exhibit exactly the same problems

[madninja]

@madninja

Perhaps the best gateway-rs can do is expose the region for the asserted gateway as inferred by the chain. That way maker specific functionality can translate that to picking the right configuration files and settings?

Yes, I also think so.

It also would be great if gateway-rs warns the gateway owner in an easily understandable way if there's any (potential) mismatch between gateway-rs and packet forwarder settings. This shouldn't be hard to implement, and this will unlock some opportunities for both helium and the guys already owning the gateways manufactured before.

Potentially? The problem, as indicated by other comments here is that the packet forwarder config is located in wildly different places and actually looks different between different makers. The question then is whether it's really gateway-rs' job to be all of that for a light hotspot or whether a maker should really do this as part of integration.

[pvolodin]

@mullerch The LoraWAN community already tried this way. The problem is that there's a lot of subtle differences between HALs. And don't forget that HAL for LoRaWAN™ Interface Reference Design v2 isn't open soursed yet (though I don't think we'll see v2 gateways on the Helium network).

[pvolodin]

@madninja > Potentially? The problem, as indicated by other comments here is that the packet forwarder config is located in wildly different places and actually looks different between different makers.

I didn't mean gateway-rs shall deal with PF configs. But it easily can detect, for instance, that it is receiveng the packets TXed on FSB1 while it is configured for FSB2.

Sorry for not being clear enough.

[mullerch]

@mullerch The LoraWAN community already tried this way. The problem is that there's a lot of subtle differences between HALs. And don't forget that HAL for LoRaWAN™ Interface Reference Design v2 isn't open soursed yet (though I don't think we'll see v2 gateways on the Helium network).

I agree for HAL differences. But the problem is the same for the Semtech Packet Forwarder than for other softwares. Except the "Lora-net packet forwarder" has never claimed to be a production grade software and has been replaced by the Basic Station, which by the way also supports channels configuration over its LNS protocol and could be an option.

[madninja]

@madninja > Potentially? The problem, as indicated by other comments here is that the packet forwarder config is located in wildly different places and actually looks different between different makers.

I didn't mean gateway-rs shall deal with PF configs. But it easily can detect, for instance, that it is receiveng the packets TXed on FSB1 while it is configured for FSB2.

Sorry for not being clear enough.

yes, that's definitely possible and a good log warning

[madninja]

@mullerch The LoraWAN community already tried this way. The problem is that there's a lot of subtle differences between HALs. And don't forget that HAL for LoRaWAN™ Interface Reference Design v2 isn't open soursed yet (though I don't think we'll see v2 gateways on the Helium network).

I agree for HAL differences. But the problem is the same for the Semtech Packet Forwarder than for other softwares. Except the "Lora-net packet forwarder" has never claimed to be a production grade software and has been replaced by the Basic Station, which by the way also supports channels configuration over its LNS protocol and could be an option.

That'd be awesome, and does seem like the general direction it's heading in. The problem is that this package isn't trying to be everything a hotspot does and shouldn't really force that decision on makers since very few (none?) use basic station now.

[pvolodin]

@mullerch > Except the "Lora-net packet forwarder" has never claimed to be a production grade software and has been replaced by the Basic Station,

For me it seems to be an unsuccessfull attempt. Do you know any gateway other than TrackNet derivatives supporting Basic Station protocol ?

[mullerch]

I don't see it as a unsuccessful attempt at all. Many gateways that have received a software update within the last year are already proposing the Basic Station (Laird Sentrius™ RG1xx, Milesight UG6/8X, MultiTech Conduit AEP, Tektelic Kona Micro, Wifx LORIX One) as the LNS protocol proposes many improvements regarding security, connectivity and reliability.

Interfacing with a protocol that has channels configuration would be a huge benefit. LoRa concentratator daemons that do not support this have a major architectural issue and will imho disappear in the next years as they bring to much complexity for the configuration to makers.

[madninja]

I personally agree with the Basic Station direction. I just don't think gateway-rs should be mandating use of it for makers today.

[pvolodin]

I believe they mostly use something like station2pkfwd . But of course it would be great to have something like CUPS on all GWs.

[mullerch]

Maybe proposing multiple solutions would be the way to go to ensure both quick and universal prototyping for new adopters (with Semtech UDP) and a maintainable way to manager configurations for production grade gateways (with LNS, CS).

[madninja]

which is why the current thinking is to expose what the chain says through gateway-rs's cli (or local grpc) and have other parts of the system manage more or less sophisticated integrations. That make sense?

[mullerch]

Yes I agree. Bottom line of this is to build a good abstraction for concentrator daemon communication so that contributors can extend gateway-rs with other protocols when needed.

[madninja]

Yes, and I think we're all agreeing that a better concentrator daemon or layered abstraction is outside the scope of this repo