Open Brammers45 opened 1 year ago
This actually isn't in the scope of ttn-lw-cli
but should be possible with the migration tool https://www.thethingsindustries.com/docs/the-things-stack/migrating/migrating-between-tts-distributions/#migrate-end-devices-and-gateways
Have you tried that?
I originally tried the migration tool first before using the CLI but Migration tool is for TTSv2 to TTSv3, not v3 to v3. Several flags needed show up as errors: Error: error:go.thethings.network/lorawan-stack-migrate/pkg/export:export (export device test-lux) error:go.thethings.network/lorawan-stack-migrate/pkg/export:export (export device test-lux) device_id=test-lux correlation_id=93f84959d4f143b19a109f4024596b36 — error:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask) forbidden_paths=[lora_alliance_profile_ids.vendor_profile_id lora_alliance_profile_ids.vendor_id serial_number last_seen_at] correlation_id=f11c2dab6fd04d349d175c898b24f9a7
I originally tried the migration tool first before using the CLI but Migration tool is for TTSv2 to TTSv3, not v3 to v3.
That's not true. The migration tool supports TTS to TTS (v3) migrations. See https://github.com/TheThingsNetwork/lorawan-stack-migrate#the-things-stack
Sorry for the misconception. I was told the migration tool wouldn't work when I posted this issue on the TTN forum. I've followed that guide through step by step and ended up with the error posted above.
Ok for future issues with the migration tool, please create an issue in the lorawan-stack-migrate
repository.
For this particular issue, there seems to be a version mismatch between your TTS and migration tool versions.
Which versions are you using for TTS and the migration tool?
TTS 3.17.1 and Migration tool 0.9.0
Ok so the issue is that v3.17.1
is 10 minor versions older than the latest v3.27.0
and the migration tooling is usually compatible with the previous two versions of TTS. You will have to downgrade the migration tooling and try one of the older versions I guess.
I've had a look at this but there's no documentation I've found which states what version of the migration tool works with which version.
Yes unfortunately the tool and TTS do not follow the same release cycle. The only (bad) suggestion I have is that you can progressively downgrade the migration tool versions and try.
Coming at it from a different approach and still facing issues, so I'm putting together a powershell script to export all the details for all the sensors into a big JSON that I can import into 3.26.0. The version I am exporting from is 3.17.0 and the command listed in the documentation that should work is
ttn-lw-cli end-devices get migration-testing lux-sens --name --description --lorawan-version --lorawan-phy-version --frequency-plan-id --supports-join --root-keys
Which returns the JSON:
{ "ids": { "device_id": "lux-sens", "application_ids": { "application_id": "migration-testing" }, "dev_eui": "00137A10000032F0", "join_eui": "C3B48C7549EF46D1", "dev_addr": "001565ED" }, "created_at": "2023-09-28T09:09:14.984Z", "updated_at": "2023-09-28T09:09:29.651865300Z", "network_server_address": "uk-nmuk-s0311.nissaneu.nmcorp.nissan.biz", "join_server_address": "uk-nmuk-s0311.nissaneu.nmcorp.nissan.biz", "lorawan_version": "MAC_V1_0_2", "lorawan_phy_version": "PHY_V1_0_2_REV_B", "frequency_plan_id": "EU_863_870", "supports_join": true, "root_keys": { "app_key": { "key": "B025343D151E9D3F66A1D90A1B4EBDC2" } } }
When importing into 3.26.0 I get the following error:
{ "code": 3, "message": "error:pkg/util/io:json_token (invalid JSON token)", "details": [ { "@type": "type.googleapis.com/ttn.lorawan.v3.ErrorDetails", "namespace": "pkg/util/io", "name": "json_token", "message_format": "invalid JSON token", "correlation_id": "7c41a6d05251425fab917dbc9ab35ebd", "code": 3 } ] }
If i trim the JSON to fit exactly like what's listed in the documentation:
{ "ids": { "device_id": "lux-sens", "dev_eui": "00137A10000032F0", "join_eui": "C3B48C7549EF46D1" }, "name": "Light Sensor", "description": "On my desk", "lorawan_version": "MAC_V1_0_2", "lorawan_phy_version": "PHY_V1_0_2_REV_B", "frequency_plan_id": "EU_863_870_TTN", "supports_join": true, "root_keys": { "app_key": { "key": "B025343D151E9D3F66A1D90A1B4EBDC2" } } }
It returns the same error. Is the JSON requirements for 3.26.0 different to what's listed in the documentation? How do I get this to work?
@Brammers45: What command are you running? This JSON is valid.
Regarding your question about the formatters, while experimenting today I found you can include the formatters object in the JSON fed to dev create
and it will bring them in. This isn't documented so may be a bug or go away in future. But it seems easier than using up to 4 options with the dev set
command.
Coming back to this issue, the latest release of the migration tool (v0.12.0
) should've fixed this issue. I'd recommend you to try now with the migration tooling.
Summary
The only way listed in the documentation for exporting devices is one at a time using the get command with a few optional flags. There's no documented method for exporting a whole applications devices, and the way that should work doesn't.
Also is there a way to keep the payload formatter attached to the device through export and import, else every device will have to be manually configured after import?
Steps to Reproduce
Using the get command listed in the documentation
ttn-lw-cli end-devices get migration-testing test-lux --name --description --lorawan-version --lorawan-phy-version --frequency-plan-id --supports-join --root-keys --mac-settings
works great for a singular device but cannot be used for multiple devices as the device-id field is required so get can only be used for one device at a time. Next I tried using:ttn-lw-cli end-devices search migration-testing --name --description --lorawan-version --lorawan-phy-version --frequency-plan-id --supports-join --root-keys --mac-settings
all of these flags are listed under the --help section for the search command yet this command returnserror:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask) forbidden_paths=[frequency_plan_id lorawan_phy_version lorawan_version mac_settings root_keys supports_join] correlation_id=46c16aee396240288137d873ee5aacee
Current Result
Using search does return all devices inside an application but the optional flags don't work, which means search can't be used for the import/export of devices
Expected Result
Using
ttn-lw-cli end-devices search <application-name> --name --description --lorawan-version --lorawan-phy-version --frequency-plan-id --supports-join --root-keys --mac-settings
should provide a JSON with all devices inside an application with all the necessary fields to import into another instance of TTS. There should also be an optional flag to include the payload formatters for each device, so there's no manual configuration required per device upon import.Relevant Logs
No response
URL
No response
Deployment
The Things Stack Open Source (self-hosted)
The Things Stack Version
3.26.0
Client Name and Version
No response
Other Information
No response
Proposed Fix
No response
Contributing
Code of Conduct