Flatten seperator problems with platform=null in Netbox

[xeiss]

I have set the "Flatten seperator" to "_" without a "Flatten keys" filter, because all objects are better to handle in Icinga director when they are flat. For my Netbox data this works flawless for Virtual Maschines import, but not for Devices, especially for "platform" info. I found out that the difference is that not all my devices have a platform assigned so the platform_id=null.

In preview I see when there is a platform_keyid="nbplatform ", then the field "platform"="" (empty string). When platform_keyid=null then platform=null.

When I remove the flatten seperator and import the data. I see platform filled with JSON Object data, so he gets the data from Netbox correct. When I add the Search filter "platform_id__n=null" (not empty filter), I get the fields platform_display, platform_id, platform_keyid, platform_name, platform_slug, platform_url. (But not a workaround, because I also need devices without platform)

So TL;DR when not all platforms in Netbox are assigned, it broke the flatten feature.

I also tried a different Flatten seperator, but that don't change anything. Also, I think this is not only for "platform" field, but I only test with this. I use latest plugin version 3.6.1.1 Together with director 1.11.0, icingadb 1.1.1, icingaweb 2.12.1

[sol1-matt]

This problem has occurred in the past and the solution is in library/Netbox/Netbox.php. I can see change introduced in commit 0d1f7c0 with multiple ways of handling Null values.

Looking at it further this code is called before the flatten happens, it wouldn't be safe to move it so that means it will need to be reimplemented or refactored. It also looks like it is adding columns before the data is added.

Without testing the problem myself I'd say there is a bug that needs fixing and the solution here will likely end up being fairly simple, could involve a refactor of existing null handling and will need a bunch of testing.

[xeiss]

Thanks for update. Can't help you with code at the moment. Hope you can fix it someday. Reach me out when I can help you with testing.

I found a Icinga Director Property Modifier workaround for now, so I don't have to use the flatten seperator:

[sol1-matt]

That will work, for platform I've taken to creating custom fields

I have 4 Import Sources for platform with the only differences being the Key Column Name which is set to the keyid and custom fields.

Then I have Sync Rules for each of the custom field based import sources that are used to create host groups.

And the final keyid based Import Source is a host template that imports the groups. (which only works because I change director to be group append +=, you could set vars on the template and read the var with the group too).

This gives me sane grouping so it is easy to target groups of systems in apply rules or dashboards.

---

I've put this on my list to fix but I won't get to is straight away so I added a few more details above so I know where to start when I get some time :)

[sol1-matt]

@xeiss I looked into this, could see the lack of nulls, figured out a workaround to add nulls, looked at the code and came up with a possible solution. But I can't actually break anything so I don't know if my solution works.

The null values from Netbox are returning nothing for Import Source, there is no value as opposed to a null value, but creating a host template with custom field for platform id as an example worked.

Are you able to provide more detail on what isn't working for you and how I go about reproducing it.

Workaround to add null Import Modifiers can create a null value by using 'combine multiple values' and the field value as the source, destination and replacement values. eg: read value platform_id, write ${platform_id} to field platform_id.

[sol1-matt]

d5e4dac is untested pending a reproducable test case.

[xeiss]

d5e4dac is untested pending a reproducable test case.

With that version I only get an import source error: "arraykeys() expects parameter 1 to be array, object given" (with a flatten seperator set to "")

You can test that easly with https://demo.netbox.dev/api and create a API Token (https://demo.netbox.dev/users/tokens/add/ User admin/admin) for some tests. Edit: Sure data is reseting in Netbox demo at night, but the always have some realistic data in it.

I went back to the master version, and wanted to replicate the error on the netbox online demo so you have easy access to it. But with that data I can't reproduce the issue. I don't find a real the difference. Our and the demo are also on same version v3.7.3. But maybe our migration path is differnt...

[sol1-matt]

I went back to the master version, and wanted to replicate the error on the Netbox online demo so you have easy access to it. But with that data I can't reproduce the issue.

@xeiss It looks like the problem you are getting is related to a specific data structure or the way you use the data.

• I'm assuming the initial error/problem you are getting shows Sync Rules, is this correct?
• If the error is in a sync rule
  • What is the error message?
  • Do you know which sync rule property is causing the problem?
  • If yes are you able to share that sync rule and the data the sync rule addresses from the Import Source preview?

Sync rule preview may let you know which device/vm is generating the error (though this could be all of them)

[xeiss]

@sol1-matt Sorry for late answer, I have overseen this issue.

I'm assuming the initial error/problem you are getting shows Sync Rules, is this correct?

No, I am talking about the "Import source" entry. I see the problem already in the preview of the import source. I also created a new import source with no sync rule to test this.

In the meantime, I upgraded to PHP 8.2.18, so the error message is slightly different (tested with https://github.com/sol1/icingaweb2-module-netbox/commit/d5e4dac9c1e8ae70d2c1814cacb8f46091a8669d): array_keys(): Argument #1 ($array) must be of type array, stdClass given (Netbox.php:342) I get this error in "Preview" and also in "Check for changes" in the Import Source.

I tested with the Netbox Demo with these settings: And also got the same "array_keys(): Argument #1 ($array) must be of type array, stdClass given (Netbox.php:342)" error in "Preview" and in "Check for changes".