CiscoDevNet / terraform-provider-fmc

Terraform Provider for FMC
https://registry.terraform.io/providers/CiscoDevNet/fmc/latest/docs
Mozilla Public License 2.0
16 stars 31 forks source link

data.fmc_network_objects fails when filtering on name or value when API does paging #5

Closed NordbergDK closed 2 years ago

NordbergDK commented 3 years ago

When trying to use data.fmc_network_objects to fetch existing objects, this provider does not factor in paging. This means that if the object you are trying to fetch is NOT on the first "page" (first 25 items) then it fails with an error saying "duplicates found, no exact match, length of response is: 25, expected 1, please search using a unique id, name or value".

If the object you are searching for is on the first page (first 25 items) then it works fine.

Provider should implement either fetching the next page and next page until no more remain in the response, or just add a &limit=500 or something to override the default limit of 25 per page from the API.

Function in question is here: https://github.com/CiscoDevNet/terraform-provider-fmc/blob/b7ad9f39717a06ebf2b4775f3e7c0a4e45643cb7/fmc/fmc_network_object.go#L46

In the meantime, does anyone have a workaround for this? The strange thing to me is that FMC always return all network objects even if you use "&filter=nameOrValue:%s" in your API query. This provider does implement logic to then search the response for the actual requested object: https://github.com/CiscoDevNet/terraform-provider-fmc/blob/main/fmc/fmc_network_object.go#L61-L65, but as mentioned, does not factor in paging.

Only really bad workaround we have for now, is to hardcode the ID's of the network objects and query them directly on ID.

Any help would be appreciated.

NordbergDK commented 3 years ago

From reading up on documentation and testing out api-explorer and own API client. It does not seem like the "filter=nameOrValue:%s" is supported or documented in any way. https://www.cisco.com/c/en/us/td/docs/security/firepower/670/api/REST/firepower_management_center_rest_api_quick_start_guide_670/Objects_In_The_REST_API.html#reference_drq_d5d_bcb

It does state pretty clearly that the GET operation on object/networks is used for listing.

It also states that limit is default 25 (which explains my issue), but can be increased via query parameter: The REST API will serve only 25 results per page. This can be increased up to 1000 using the limit query parameter.

I will gladly fork and make the very small fix for add &limit=1000 to the query string, which will fix this issue without implementing paging for any configuration with less than 1000 network objects (up from 25).

https://github.com/CiscoDevNet/terraform-provider-fmc/blob/b7ad9f39717a06ebf2b4775f3e7c0a4e45643cb7/fmc/fmc_network_object.go#L47 will be changed to:

  url := fmt.Sprintf("%s/object/networks?expanded=true&limit=1000&filter=nameOrValue:%s", v.domainBaseURL, nameOrValue)
adyanth commented 2 years ago

@NordbergDK that field nameOrValue was introduced in the later versions of the FMC. It means that searches will only return partial matches which might not exceed the default paging limit. But specifically maxing the limit might be a good idea as a fallback.

Screenshot 2021-12-08 at 5 13 05 PM
adyanth commented 2 years ago

Fixed with #6

NordbergDK commented 2 years ago

The PR 6 was merged and all seemed good. When PR 10 was merged the change from PR 6 was reverted.

Please apply the "limit=1000" to line 47 in fmc_network_object.go again.

url := fmt.Sprintf("%s/object/networks?expanded=true&limit=1000&filter=nameOrValue:%s", v.domainBaseURL, nameOrValue)