Issue importing full json WAF policy with v1.19.0 - resource bigip_waf_policy

amolari commented 1 year ago

Environment

TMOS/Bigip Version: 17.1.0.1
Terraform Version: 1.3.9
Terraform bigip provider Version: 1.19.0

Summary

I could import the same WAF policy json successfully on v1.18.0 When running the same code with 1.19.0 I have an error during the apply. I do see the policy being imported on the BIG-IP, though.

Steps To Reproduce

Steps to reproduce the behavior:

resource "bigip_waf_policy" "test_for_TF_l" { name = "test_for_TF_l" template_name = "POLICY_TEMPLATE_BLANK" policy_import_json = file("./Common_test_for_TF_l.json") }

The file Common_test_for_TF_l.json is a full-json policy.

Expected Behavior

WAF policy import successful, without any TF error

Actual Behavior

2023-08-30T10:54:00.018+0200 [DEBUG] provider.terraform-provider-bigip_v1.19.0: Import policy:{Filename:test_for_TF_l.json FullPath:/Common/test_for_TF_l Policy:{FullPath:/Common/test_for_TF_l}}: timestamp=2023-08-30T10:54:00.018+0200 2023-08-30T10:54:00.033+0200 [INFO] provider.terraform-provider-bigip_v1.19.0: AWAF Import policy TaskID :WPGdpF7_i-ePYQM54FUShg: timestamp=2023-08-30T10:54:00.033+0200 bigip_waf_policy.test_for_TF_l: Still creating... [10s elapsed] 2023-08-30T10:54:10.066+0200 [ERROR] provider.terraform-provider-bigip_v1.19.0: Response contains error diagnostic: diagnostic_detail= diagnostic_severity=ERROR tf_provider_addr=provider tf_resource_type=bigip_waf_policy tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_summary="error retrieving waf policy : [ERROR] WafPolicy: test_for_TF_l on partition Common not found" tf_proto_version=5.3 tf_req_id=478fca96-cc0a-c39c-3be2-482e31709a4e @module=sdk.proto timestamp=2023-08-30T10:54:10.065+0200 2023-08-30T10:54:10.069+0200 [ERROR] vertex "bigip_waf_policy.test_for_TF_l" error: error retrieving waf policy : [ERROR] WafPolicy: test_for_TF_l on partition Common not found ╷ │ Error: error retrieving waf policy : [ERROR] WafPolicy: test_for_TF_l on partition Common not found │ │ with bigip_waf_policy.test_for_TF_l, │ on waf_policy_import.tf line 1, in resource "bigip_waf_policy" "test_for_TF_l": │ 1: resource "bigip_waf_policy" "test_for_TF_l" { │ ╵ 2023-08-30T10:54:10.081+0200 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF" 2023-08-30T10:54:10.085+0200 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/f5networks/bigip/1.19.0/linux_amd64/terraform-provider-bigip_v1.19.0 pid=7644 2023-08-30T10:54:10.085+0200 [DEBUG] provider: plugin exited

pgouband commented 1 year ago

Hi,

Thanks for reporting. Added to the backlog and internal tracking ID for this request is: INFRAANO-1289.

pgouband commented 1 year ago

Hi @amolari,

The issue is not reproductible in our lab. The error is about the policy not available in Common partition. Can you check your configuration and your environment?

amolari commented 1 year ago

Hi @pgouband

the exact same code/policy works without issue with 1.18.0 (between my 2 tests I really just change the version of the provider, the rest is identical)

the policy is imported and then there is the TF error. On the BIGIP:


# list asm policy test_for_TF_l all-properties
asm policy test_for_TF_l {
app-service none
blocking-mode disabled
description "test policy configured through the UI"
encoding iso-8859-1
inactive
parent-policy none
partition Common
policy-builder disabled
policy-template POLICY_TEMPLATE_BLANK
policy-type security
virtual-servers none
}

amolari commented 1 year ago

Hi @pgouband I've found something: if I change the policy json file to have applicationLanguage=utf-8 then I can import it without an error. Could this PR be the issue: https://github.com/F5Networks/terraform-provider-bigip/pull/851 ?

pgouband commented 1 year ago

Hi @amolari,

A fix will be included in our next release.

amolari commented 11 months ago

@RavinderReddyF5 @pgouband Doing some testing with v1.20.0 and there is an issue (error undetected).

I have a basic policy (full json export) which the only setting set (from Fondamental template) is the Application language, set to "iso-8859-1". I can import (1st time) the policy successfully using this TF code:

resource "bigip_waf_policy" "test_basic_pol_iso" {
  name = "test_basic_pol_iso"
  template_name = "POLICY_TEMPLATE_BLANK"
  policy_import_json = file("./mytests/test_basic_pol_iso.json")
}

Everything's fine until I modify my json file (simple change such as description). The TF apply is successful but the policy is not updated. There is an issue for non-utf8 policies (I've seen it on the UI) and I've just opened F5 support case 00491134.

But the point is that the TF apply doesn't detect the error and return a successful status. What we see in the bigip asm logs: Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Import Policy Task Import Policy Task (1697630924.505415) [add]: Target Policy was set to /Common/test_basic_pol_iso. Filename was set to test_basic_pol_iso.json. { audit: username = admin, client IP = 172.31.10.111 } Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Import Policy Task Import Policy Task (1697630924.505415) [update]: Status was set to STARTED. Execution Start Time was set to 1697630924. { audit: username = admin, client IP = 172.31.10.111 } Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Apply Policy Task Apply Policy Task (1697630924.532803) [add]: Target Policy was set to /Common/test_basic_pol_iso. { audit: username = admin, client IP = 172.31.10.111 } Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Export Policy Task Export Policy Task (1697630924.572459) [add]: Target Policy was set to /Common/test_basic_pol_iso. Result Inline was set to 1. { audit: username = admin, client IP = 172.31.10.111 } Oct 18 14:08:44 bigip-h1-g3t.techlab.ch crit g_server_rpc_handler_async.pl[11273]: 01310027:2: ASM subsystem error (asm_config_server.pl,F5::ImportExportPolicy::Base::fatal_error): Failed action: Imported and replaced policies have different encodings. Oct 18 14:08:44 bigip-h1-g3t.techlab.ch crit g_server_rpc_handler_async.pl[11273]: 01310027:2: ASM subsystem error (asm_config_server.pl,F5::ImportExportPolicy::Base::fatal_error): InternalError - import_policy failed: fatal: Failed action: Imported and replaced policies have different encodings. Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Import Policy Task Import Policy Task (1697630924.505415) [update]: Status was set to FAILURE. End Time was set to 1697630925. Message was set to InternalError - import_policy failed: fatal: Failed action: Imported and replaced policies have different encodings. . { audit: username = admin, client IP = 172.31.10.111 } Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: Per-invocation log rate exceeded; throttling. Oct 18 14:08:49 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: Resuming log processing at this invocation; held 4 messages. Oct 18 14:08:49 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Export Policy Task Export Policy Task (1697630929.628154) [add]: Target Policy was set to /Common/test_basic_pol_iso. Result Inline was set to 1. { audit: username = admin, client IP = 172.31.10.111 } Oct 18 14:08:49 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Export Policy Task Export Policy Task (1697630929.628154) [update]: Status was set to STARTED. Execution Start Time was set to 1697630929. { audit: username = admin, client IP = 172.31.10.111 } Oct 18 14:08:50 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Export Policy Task Export Policy Task (1697630929.628154) [update]: File Size was set to 57586. Status was set to COMPLETED. End Time was set to 1697630930. Message was set to Policy '/Common/test_basic_pol_iso' was successfully exported.. { audit: username = admin, client IP = 172.31.10.111 }

TF apply should return an error not a change applied successfully.

pgouband commented 11 months ago

Hi,

Thanks for reporting. Added to the backlog and internal tracking ID for this request is: INFRAANO-1349.

pgouband commented 11 months ago

Hi @amolari,

A WAF policy can't be replaced when the application language is different so the behavior is by design.

RavinderReddyF5 commented 11 months ago

hi @amolari please open new issue if still problem seen

F5Networks / terraform-provider-bigip