search

netbox-community / netbox

The premier source of truth powering network automation. Open source under Apache 2. Public demo: https://demo.netbox.dev
http://netboxlabs.com/oss/netbox/
Apache License 2.0
15.68k stars 2.53k forks source link

api custom_field filter with negation #15867

Open davama opened 4 months ago

davama commented 4 months ago

NetBox version

v3.7.6

Feature type

Change to existing functionality

Proposed functionality

Good day everyone,

Had posted this here and was told to repost as an FR. I can repost the details of that ticket into this one, if that is desired. Hope I'm doing this ticket correctly...

Essentially, just as you can filter negation on regular fields, custom fields should also be possible.

Thank you for your consideration.

Best, Dave

Use case

We have a custom field with default value null or "empty".

Filter combination of __emtpy=true, __emtpy=1, __ic=null , __nie=null do not work.

Database changes

No response

External dependencies

No response

arthanson commented 4 months ago

From previous ticket:

Steps to Reproduce

I am trying to filter for custom fields that are not null or not a string. Saw this ticket but it is closed now. Created the below cf for testing.

custom_fields 
[
  {
    "id": 35,
    "url": "https://redacted/api/extras/custom-fields/35/",
    "display": "repoempty",
    "content_types": [
      "dcim.device"
    ],
    "type": {
      "value": "text",
      "label": "Text"
    },
    "object_type": null,
    "data_type": "string",
    "name": "repoempty",
    "label": "repoempty",
    "group_name": "",
    "description": "dummy cf - when default value is the literal string \"\"",
    "required": false,
    "search_weight": 1000,
    "filter_logic": {
      "value": "loose",
      "label": "Loose"
    },
    "ui_visible": {
      "value": "always",
      "label": "Always"
    },
    "ui_editable": {
      "value": "yes",
      "label": "Yes"
    },
    "is_cloneable": false,
    "default": "",
    "weight": 100,
    "validation_minimum": null,
    "validation_maximum": null,
    "validation_regex": "",
    "choice_set": null,
    "created": "2024-04-25T14:01:21.196733Z",
    "last_updated": "2024-04-25T14:01:21.196769Z"
  },
  {
    "id": 34,
    "url": "https://redacted/api/extras/custom-fields/34/",
    "display": "repoemptystring",
    "content_types": [
      "dcim.device"
    ],
    "type": {
      "value": "text",
      "label": "Text"
    },
    "object_type": null,
    "data_type": "string",
    "name": "repoemptystring",
    "label": "repoemptystring",
    "group_name": "",
    "description": "dummy cf - when default value is the literal string \"empty\"",
    "required": false,
    "search_weight": 1000,
    "filter_logic": {
      "value": "loose",
      "label": "Loose"
    },
    "ui_visible": {
      "value": "always",
      "label": "Always"
    },
    "ui_editable": {
      "value": "yes",
      "label": "Yes"
    },
    "is_cloneable": false,
    "default": "empty",
    "weight": 100,
    "validation_minimum": null,
    "validation_maximum": null,
    "validation_regex": "",
    "choice_set": null,
    "created": "2024-04-25T14:00:02.456419Z",
    "last_updated": "2024-04-25T14:00:02.456460Z"
  },
  {
    "id": 33,
    "url": "https://redacted/api/extras/custom-fields/33/",
    "display": "reponull",
    "content_types": [
      "dcim.device"
    ],
    "type": {
      "value": "text",
      "label": "Text"
    },
    "object_type": null,
    "data_type": "string",
    "name": "reponull",
    "label": "reponull",
    "group_name": "",
    "description": "dummy cf - when default value is null",
    "required": false,
    "search_weight": 1000,
    "filter_logic": {
      "value": "loose",
      "label": "Loose"
    },
    "ui_visible": {
      "value": "always",
      "label": "Always"
    },
    "ui_editable": {
      "value": "yes",
      "label": "Yes"
    },
    "is_cloneable": false,
    "default": null,
    "weight": 100,
    "validation_minimum": null,
    "validation_maximum": null,
    "validation_regex": "",
    "choice_set": null,
    "created": "2024-04-25T13:57:41.959924Z",
    "last_updated": "2024-04-25T13:57:41.959962Z"
  },
  {
    "id": 36,
    "url": "https://redacted/api/extras/custom-fields/36/",
    "display": "reponullstring",
    "content_types": [
      "dcim.device"
    ],
    "type": {
      "value": "text",
      "label": "Text"
    },
    "object_type": null,
    "data_type": "string",
    "name": "reponullstring",
    "label": "reponullstring",
    "group_name": "",
    "description": "dummy cf - when default value is the literal string \"null\"",
    "required": false,
    "search_weight": 1000,
    "filter_logic": {
      "value": "loose",
      "label": "Loose"
    },
    "ui_visible": {
      "value": "always",
      "label": "Always"
    },
    "ui_editable": {
      "value": "yes",
      "label": "Yes"
    },
    "is_cloneable": false,
    "default": "null",
    "weight": 100,
    "validation_minimum": null,
    "validation_maximum": null,
    "validation_regex": "",
    "choice_set": null,
    "created": "2024-04-25T14:03:07.790615Z",
    "last_updated": "2024-04-25T14:03:07.790651Z"
  }
]

The default values look like this on a device:

                "repoempty": "",
                "repoemptystring": "empty",
                "reponull": null,
                "reponullstring": "null",

The values I changed for one device:

                "repoempty": "data",
                "repoemptystring": "something",
                "reponull": null,
                "reponullstring": "null",

API calls that worked

/api/dcim/devices/?exclude=config_context&limit=0&site=usa-ht&cf_repoempty=data
/api/dcim/devices/?exclude=config_context&limit=0&site=usa-ht&cf_repoemptystring=something  ( I know it's redundant)

The API calls that did not work. I would still get all my devices for X site.

/api/dcim/devices/?exclude=config_context&limit=0&site=usa-ht&cf_repoempty=
/api/dcim/devices/?exclude=config_context&limit=0&site=usa-ht&cf_repoempty__nie=data
/api/dcim/devices/?exclude=config_context&limit=0&site=usa-ht&cf_repoemptystring__nie=something
/api/dcim/devices/?exclude=config_context&limit=0&site=usa-ht&cf_reponull__empty=false
/api/dcim/devices/?exclude=config_context&limit=0&site=usa-ht&cf_reponull__empty=0
/api/dcim/devices/?exclude=config_context&limit=0&site=usa-ht&cf_reponullstring=null (got ValueError `Cannot use None as a query value`)
davama commented 4 months ago

@arthanson Thank you for that

llamafilm commented 2 months ago

Filtering custom fields would also be very helpful for me in custom scripts. Is it fair to assume that when this is implemented for Rest API, the same filtering operators will work in Python scripts too?

For example, I can match a specific value like this:

>>> Prefix.objects.filter(custom_field_data__bloxone_id='ff249123-287b-11ec-a803-26abf9fc4e12')
<PrefixQuerySet [<Prefix: [10.56.225.0/24](http://10.56.225.0/24)>]>

But appending __n returns nothing:

>>> Prefix.objects.filter(custom_field_data__bloxone_id__n='something')
<PrefixQuerySet []>