Improve documentation on API responses from the registration/network servers

[proffalken]

Summary

I'm trying to write a python application to interact with The Things Stack.

Part of this is the automatic registration and retrieval of information about an end device (in this case it's an ABP device).

The API returns the AppSKey as a 32-character string without any spacing between characters, however most software libraries for embedded devices required this to be in either MSB, LSB, or hex formats.

The console is able to translate between all three formats, but there does not appear to be a documented approach to changing the output of the API into a format that can be easily understood by the end device libraries

Why do we need this ?

Automation of the platform is a key area in which TTI can provide value to large-scale customers.

Improving the documentation with examples around the API's rather than a reasonably obscure set of reference documents will increase uptake further of the platform for companies who understand the benefits of automation vs "point and click" configuration.

What is already there? What do you see now?

Please see this forum thread for the full details of what I've had to do so far in order to get the API working for registering ABP devices - it includes having to "sniff" the traffic between the console and the API just to ensure the JSON was in the correct format with the correct fields!

What is missing? What do you want to see?

Concrete examples of how to create and retrieve information about end devices, along with information on converting the output from the API into the appropriate formats for common libraries such as LMIC and uPyLoRaWAN

How do you propose to document this?

This needs to be done by someone who understands the source code of the various API's and can document it.

Can you do this yourself and submit a Pull Request?

No

[NicolasMrad]

blocked on: https://github.com/TheThingsNetwork/lorawan-stack/issues/2297

[NicolasMrad]

https://github.com/TheThingsNetwork/lorawan-stack/issues/3057

[proffalken]

@NicolasMrad I agree that those tickets are related, but I don't think they should block this?

All I'm looking for here is the JSON format or query string in order to retrieve the network_s_key and app_s_key in various formats, just like the console is able to.

All I seem able to get from the API is AABBCCDD, whereas I want to be able to retrieve the 0xAA, 0xBB, 0xCC, 0xDD and 0xDD, 0xCC, 0xBB, 0xAA formats so I can pass them on to my LMIC and Python libraries on my end devices.

Surely that's something possible with the existing API?

[proffalken]

@NicolasMrad given that both those tickets are well over a year old, is there any way to either get traction on them or just get the information I'm asking for in this ticket?

I'm concerned that there will be quite a wait to get this working!

[proffalken]

If it helps at all, the code I currently have is as follows:

def get_ttn_details(device_id=None, app_name=None, auth_token=None):
    auth_token = auth_token

    headers = {
        "Accept": "application/json",
        "Authorization": f"Bearer {auth_token}",
        "Accept": "application/json",
    }

    ns_field_mask = (
        "ids.device_id,ids.dev_eui,session.keys.f_nwk_s_int_key.key,session.dev_addr"
    )
    ns_uri = f"https://eu1.cloud.thethings.network/api/v3/ns/applications/{app_name}/devices/{device_id}?field_mask={ns_field_mask}"

    nsresponse = requests.get(
        ns_uri,
        headers=headers,
    )

    as_field_mask = (
        "ids.device_id,ids.dev_eui,session.keys.app_s_key.key,session.dev_addr"
    )
    as_uri = f"https://eu1.cloud.thethings.network/api/v3/as/applications/{app_name}/devices/{device_id}?field_mask={as_field_mask}"

    asresponse = requests.get(
        as_uri,
        headers=headers,
    )

    if nsresponse.status_code == 200:
        logger.info("Network server returned a response, compiling data...")
        rj = nsresponse.json()
        aj = asresponse.json()
        dev_eui = rj["ids"]["dev_eui"]
        dev_addr = rj["ids"]["dev_addr"]
        network_s_key = rj["session"]["keys"]["f_nwk_s_int_key"]["key"]
        app_s_key = aj["session"]["keys"]["app_s_key"]["key"]

        return {
            "dev_eui": dev_eui,
            "dev_addr": dev_addr,
            "net_s_key": network_s_key,
            "app_s_key": app_s_key,
        }
    else:
        logger.info(f"Error: {nsresponse.status_code}")
        return {"error": nsresponse.status_code}

and I just need to get back the string as 0xNN rather than NN if that makes sense?

[adriansmares]

The Console does not have any 'hidden API' compared to other applications or integrations - it uses exactly the same calls as you do.

The frontend transformations (showing bytes in different formats) are done by the frontend logic, and are not powered by any backend header/flag/etc.

If you're interested in going from this string representation to some Python byte/bytearray, consider base64.b16decode.

[proffalken]

The Console does not have any 'hidden API' compared to other applications or integrations - it uses exactly the same calls as you do.

The frontend transformations (showing bytes in different formats) are done by the frontend logic, and are not powered by any backend header/flag/etc.

If you're interested in going from this string representation to some Python byte/bytearray, consider base64.b16decode.

Thanks @adriansmares - is the code for the front-end available anywhere so I can see that logic?

I tried viewing the source, but unlike most sites it's incredible cryptic with nonsensical function names, so very difficult to reverse engineer.

[adriansmares]

https://github.com/TheThingsNetwork/lorawan-stack/blob/02d8aaa47b511d631724c75e252445340b8f2e9a/pkg/webui/components/safe-inspector/index.js#L262-L280

Edit: Updated with the public commit hash, sorry !

[proffalken]

NVM, found it at https://github.com/TheThingsNetwork/lorawan-stack/blob/v3.22/pkg/webui/components/safe-inspector/index.js#L262-L280

~Apologies, only just saw your reply @adriansmares - that link 404's for me, and I can't find the lorawan-stack repo listed under the TTI org.~

~Any chance you can past the relevant lines here?~