os.getConfig does not produce a valid configuration

[CameronDiver]

Expected Behavior

The configuration produced should contain the nested

"deviceApiKeys" = {
  "api.balena-cloud.com":
}

as without this some bootstrapping fails.

Actual Behavior

This is not included.

Steps to Reproduce the Problem

1. Generate a config
2. Check for the lack of deviceApiKey field

References

[balena-ci]

[camerondiver] This issue has attached support thread https://jel.ly.fish/#/support-thread~2732807a-5f30-474a-b8e5-09d3615fb561

[thgreasi]

@camerondiver This doesn't look like an SDK issue, since the SDK just gets the result from the API's /download-config endpoint See: https://github.com/balena-io/balena-sdk/blob/master/lib/models/os.coffee#L453-L457

Specifically this endpoint is implemented in the open-balena-api. See: https://github.com/balena-io/open-balena-api/blob/8639082f9d93c35040112489d8c565c192bff372/src/routes/index.ts#L96 Is this field new?

[majorz]

The deviceApiKeys object is only used internally by os-config. What has happened on the attached support issue is that probably deviceApiKey was missing from the configuration, and when deviceApiKeys was added it was a hackish workaround to extract a deviceApiKey from it. Closing this as invalid.

[mhodgson]

@majorz FYI, I'm the one who had the original support ticket and deviceApiKey was always present in the singular form. The reason I reported this in the first place was because it seemed very strange that the device was bricked without the plural form in addition to the singular form.

[thgreasi]

Hi @mhodgson, Can you provide some extra information about you ended up bricking the device and what you were trying to achieve? The SDK currently reaches to the API to retrieve a device configuration that's appropriate for that specific API instance, so the API only populates the singular deviceApiKey property. On the other hand the plural deviceApiKeys as you saw is a dictionary that names the appropriate api keys for all different API instances/domains that the device might be roaming (joining & leaving).

the device was bricked without the plural form in addition to the singular form

When you did run the device w/o the plural form, did it have any issues communicating with the API instance that generated the configuration? Can you also clarify the OS version that you were using as well as the open balena vesrions (if applicable)?

[mhodgson]

@thgreasi sure! I'm doing something fairly simple (I think). I have a provisioning script that places a config.json file and a splash image into the default balena img downloaded from the UI. I need to have my devices registered with both Balena and AWS, so I create the device using the Balena API and put the uuid, device ID, and API key into the config. For the rest of the config, I rely on the config generated by the API. In the initial versions of the script, I only put the API key in the top level deviceApiKey field. When doing this, the device would boot, and show up on the dashboard, but never get past the "Initializing Supervisor" message. I was not able to find any logs or error messages. Simply adding the plural 'deviceApiKeys' field made everything work as expected. I discovered the difference by downloading a stock image from the dashboard and pulling out the config.json. Sure enough (at least for the TX2) it has the plural and singular versions.

[mhodgson]

BalenaOS version was v2.38.3+rev3 for the TX2.

[thgreasi]

So, just to clarify some bits, This happens for the Nvidia Jetson TX2 v2.38.3+rev3 prod. You are not using the balena-cli in your provisioning process but instead your code uses the balena-sdk directly to fetch the configuration? Can you clarify how & in which path of the image you inject the downloaded config.json?

Balena and AWS

This was a bit unclear. Can you clarify whether the device only runs against balenaCloud?

Can you provide a sample if how the problematic & the fixed config.json looks like (but make sure to not include the actual api key strings in there).

[mhodgson]

Re: AWS/Balena: My point is just that I need to set the device name/info at provisioning time, instead of just letting Balena Cloud make one up and provision when the device connects. This is so I can provision a one-to-one mapping of balena devices to AWS things, create an AWS certificate for the device, and attach it as a device env variable in Balena Cloud. This way when the device first boots, it already has a working connection to AWS and no additional configuration is needed.

Re: Balena SDK Yes, I use the balena-sdk. Here is basically what I am doing:

const balena = require("balena-sdk")();

const appName = "My-Balena-App";
const deviceName = "My-Balena-Device";
const imageVersion = "v2.38.3+rev3";

const uuid = balena.models.device.generateUniqueKey();
device = await balena.models.device.register(appName, uuid);
await balena.models.device.rename(device.uuid, deviceName);

const config = await balena.models.os.getConfig(appName, {
    version: imageVersion
});
config.deviceId = device.id;
config.uuid = device.uuid;
config.deviceApiKey = device.api_key;
// DOESN'T WORK WITHOUT THIS:
config.deviceApiKeys = {
  "api.balena-cloud.com": device.api_key
};

// write config json to img, flash, etc.

I don't have examples handy for you unfortunately. It takes some time to go through all the steps necessary and I'm not in front of a device at the moment.

[thgreasi]

@mhodgson we are still investigating this internally and will get back to you. In the mean time it would be very helpful if you could answer the following:

• Is this happening only on specific device types (eg does it also happen on RPis)?
• Have you tried using the balena-cli to inject the config to your image and does it work when the deviceApiKeys is added manually? It would be very concerning if this didn't work, since this at the moment the way that we support for pre-provisioning devices.

[mhodgson]

@thgreasi I only have TX2 dev board to test, so I can't test on other devices. Can you give me an example of how to use the CLI to create a device in balena and then get the config for it? I have not been able to locate a suitable function in the CLI. Also, I prefer the SDK since it is much easier to integrate into our provisioning script (in javascript/node).

[thgreasi]

Hi again @mhodgson , Here is the suggested workflow for pre-provisioning a device using the balena-cli:

$ balena device register MyApp --uuid 16851436b13a4023b39c88dffc79db7d
$ balena device rename 16851436b13a4023b39c88dffc79db7d MyPi
$ balena config generate --device 16851436b13a4023b39c88dffc79db7d --version 2.46.1 --output MyConfig.json
$ balena os configure ../path/to/os.img --device 16851436b13a4023b39c88dffc79db7d --config MyConfig.json

If you also face issues with the above steps, that that would certainly mean that there is something wrong on our side. I think that trying this would be a faster approach to investigate this issue, than passing code back and forth.

[thgreasi]

Let me also point you to our balena-cli masterclass on preregistering and preloading devices: https://github.com/balena-io/balena-cli-advanced-masterclass#5-preloading-and-preregistering

[thgreasi]

And let me also point you on the balena-cli code for the os configure action: https://github.com/balena-io/balena-cli/blob/0bc1e7057fd307b6a19dfe33b7281b85799c0689/lib/actions-oclif/os/configure.ts#L159-L223

[mhodgson]

@thgreasi thanks for pointing out these options. I'm sorry for the run around, but I am no longer able to reproduce this issue. I'm not sure if it was something else entirely, but I've done my best to reproduce my previous issue to no avail. Good news is we're up and running smoothly. Please close this issue!

[thgreasi]

@mhodgson I'm glad that everything work for your use case :+1: Thanks for this detailed report and posting back that it isn't an issue any more.