search

TheThingsNetwork / lorawan-stack

The Things Stack, an Open Source LoRaWAN Network Server
https://www.thethingsindustries.com/stack/
Apache License 2.0
962 stars 303 forks source link

Support fetching end device information from the device repository with vendor ID and profile ID #7222

Open KrishnaIyer opened 1 month ago

KrishnaIyer commented 1 month ago

Summary

Support fetching end device information from the device repository with vendor ID and profile ID.

Current Situation

When an end device is currently registered via the console using a QR code, the console does the following

  1. ListBrands from the DeviceRepository service.
  2. Parse the QR code using the Parse RPC of the EndDeviceQRCodeGenerator service.
    • This returns and EndDeviceTemplate message which contains the EndDevice message which in-turn has the important LoRaAllianceProfileIdentifiers.
    • This has a VendorID that uniquely identifies and end device vendor (ex: The Things Industries). We store this in the index file of the device repo.
    • This also contains a VendorProfileID that is currently unused.
  3. The console then lists all the end device models defined for this end device (stored as <device-name>.yaml in the folder for that device. ex: https://github.com/TheThingsNetwork/lorawan-devices/tree/master/vendor/the-things-industries) using the ListModels RPC.
    • This returns a list of models that contain info such as the firmware/hardware versions, end device profiles (which have some LoRaWAN specific parameters), end device codec ID etc.
  4. The user then uses the drop down to choose the model and the firmware/hardware versions and the profile ID.
  5. The console fills the necessary other information.
  6. The console then fetches the end device profile using the GetTemplate RPC.
  7. Now that the generic device information is filled, the console can now move on to specific info like the EUIs, keys etc which is out of scope for this issue.

Why do we need this? Who uses it, and when?

When we parse a LoRa Alliance TR005 QR code, the QR code already contains the VendorID and VendorProfileID fields. We can simplify this flow and directly get the End Device Model and the End Device Profile from it without user intervention. This makes the onboarding experience very smooth.

Proposed Implementation

New Console flow (After the backend part below is implemented)

Steps 1 and 2 from the Current Situation is the same. Step 3: The Console calls the new GetEndDeviceProvisioningInfo RPC (see below) and passes the vendor ID and vendor profile ID. a. If these are not properly defined in the device repository or there's an error, the Console can just use the same flow as now. (OR) b. If the RPC returns the correct End Device Model and End Device Profile then the console can fill all the necessary info and skip steps 4, 5, 6 in Current Situation and move to Step 7.

Backend implementation

We need the following to make this work.

  1. Introduce a new EndDeviceInfo or EndDeviceProvisioningInfo message which contains the EndDeviceModel and EndDeviceTemplate.
  2. Add a new index to bleve (ex: https://github.com/TheThingsNetwork/lorawan-stack/blob/v3.32/pkg/devicerepository/store/bleve/init.go#L64) that stores the EndDeviceProvisioningInfo message and has the vendor ID and vendor profile ID for search.
  3. Update the Initialize flow to fetch this info from DR (remote or local) and create the index.
  4. Introduce a new method to the Store to fetch EndDeviceProvisioningInfo via the vendorID and vendor profile ID. The implementation of this method uses the index from step 2/3.
  5. Add a new RPC (ex: GetEndDeviceProvisioningInfo) to the device repository service (ex: https://github.com/TheThingsNetwork/lorawan-stack/blob/v3.32/pkg/devicerepository/grpc.go#L120) which internally uses this new store method.

@johanstokking: Let me know if this is clear/complete.

Contributing

Validation

Code of Conduct

vlasebian commented 5 days ago

Tested on staging1.

Prerequisites:

Steps:

Results: Tried the following combinations:

200 OK

{
    "end_device": {
        "version_ids": {
            "brand_id": "the-things-industries",
            "model_id": "generic-node-sensor-edition",
            "firmware_version": "1.0",
            "band_id": "EU_863_870"
        },
        "lorawan_version": "MAC_V1_0_3",
        "lorawan_phy_version": "PHY_V1_0_3_REV_A",
        "supports_join": true,
        "mac_settings": {
            "supports_32_bit_f_cnt": true
        },
        "formatters": {
            "up_formatter": "FORMATTER_REPOSITORY",
            "down_formatter": "FORMATTER_REPOSITORY"
        }
    },
    "field_mask": {
        "paths": [
            "version_ids",
            "supports_join",
            "supports_class_b",
            "supports_class_c",
            "lorawan_version",
            "lorawan_phy_version",
            "formatters",
            "mac_settings.supports_32_bit_f_cnt"
        ]
    }
}

200 OK

{
    "end_device": {
        "version_ids": {
            "brand_id": "bosch",
            "model_id": "tps110",
            "firmware_version": "0.23.3",
            "band_id": "AS_923"
        },
        "lorawan_version": "MAC_V1_0_2",
        "lorawan_phy_version": "PHY_V1_0_2_REV_B",
        "supports_join": true,
        "mac_settings": {
            "supports_32_bit_f_cnt": true
        }
    },
    "field_mask": {
        "paths": [
            "version_ids",
            "supports_join",
            "supports_class_b",
            "supports_class_c",
            "lorawan_version",
            "lorawan_phy_version",
            "mac_settings.supports_32_bit_f_cnt"
        ]
    }
}

200 OK

{
    "end_device": {
        "version_ids": {
            "brand_id": "arduino",
            "model_id": "mkr-wan-1310",
            "firmware_version": "1.2.3",
            "band_id": "US_902_928"
        },
        "lorawan_version": "MAC_V1_0_2",
        "lorawan_phy_version": "PHY_V1_0_2_REV_A",
        "supports_join": true,
        "mac_settings": {
            "supports_32_bit_f_cnt": true
        },
        "formatters": {
            "up_formatter": "FORMATTER_REPOSITORY",
            "down_formatter": "FORMATTER_REPOSITORY"
        }
    },
    "field_mask": {
        "paths": [
            "version_ids",
            "supports_join",
            "supports_class_b",
            "supports_class_c",
            "lorawan_version",
            "lorawan_phy_version",
            "formatters",
            "mac_settings.supports_32_bit_f_cnt"
        ]
    }
}

As the console doesn't use the endpoints that require the vendor profile ID + vendor ID parameters, I also did some API calls:

curl -s --location  -H "Authorization: $TTN_STAGING_AUTH_TOKEN" https://tti.staging1.cloud.thethings.industries/api/v3/dr/vendors/428/profiles/1/template
curl -s --location  -H "Authorization: $TTN_STAGING_AUTH_TOKEN" https://tti.staging1.cloud.thethings.industries/api/v3/dr/vendors/428/profiles/2/template

These return a not found for now. I believe we have to rebuild the bleve.index as a new document type was introduced. I got the same issue locally, but after rebuilding the index the endpoint worked.