Closed dlethin closed 3 years ago
Out of curiosity, is there any capability to enable a debug mode to show the underlying API calls along with payload and response? I set TF_LOG=debug
but there's no useful information that I can see to troubleshoot what might be going on.
Other terraform providers log the API request/responses, such as the google gsuite_provider, shown below. It would be a great enhancement to add this capability as well allowing for easier troubleshooting.
Maybe I should add a request for provider api debug logging as a separate github issue?
Here's example logging available from the gsuite terraform provider:
2021-06-12T10:48:16.543-0400 [WARN] ValidateProviderConfig from "provider[\"registry.terraform.io/onelogin/onelogin\"]" changed the config value, but that value is unused
2021-06-12T10:48:16.543-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: 2021/06/12 10:48:16 [DEBUG] Waiting for state to become: [success]
2021-06-12T10:48:16.544-0400 [DEBUG] ReferenceTransformer: "data.onelogin_user.self" references: []
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: 2021/06/12 10:48:16 [DEBUG] Google API Request Details:
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: ---[ REQUEST ]---------------------------------------
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: GET /admin/directory/v1/groups/REDACTED?alt=json&prettyPrint=false HTTP/1.1
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Host: admin.googleapis.com
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: User-Agent: google-api-go-client/0.5 (darwin amd64) Terraform/0.15.3
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: X-Goog-Api-Client: gl-go/1.16.3 gdcl/REDACTED
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Accept-Encoding: gzip
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62:
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62:
2021-06-12T10:48:16.544-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: -----------------------------------------------------
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: 2021/06/12 10:48:16 [DEBUG] Google API Response Details:
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: ---[ RESPONSE ]--------------------------------------
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: HTTP/2.0 200 OK
...
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Cache-Control: private
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Content-Type: application/json; charset=UTF-8
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Date: Sat, 12 Jun 2021 14:48:17 GMT
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Etag: "REDACTED"
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Server: ESF
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Vary: Origin
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Vary: X-Origin
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: Vary: Referer
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: X-Content-Type-Options: nosniff
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: X-Frame-Options: SAMEORIGIN
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: X-Xss-Protection: 0
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62:
2021-06-12T10:48:16.957-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: {
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "kind": "admin#directory#group",
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "id": "REDACTED",
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "etag": "REDACTED",
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "email": "REDACTED",
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "name": "REDACTED",
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "directMembersCount": "2",
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "description": "REDACTED",
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "adminCreated": true,
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "nonEditableAliases": [
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "REDACTED",
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: "REDACTED"
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: ]
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: }
2021-06-12T10:48:16.958-0400 [DEBUG] provider.terraform-provider-gsuite_v0.1.62: -----------------------------------------------------
This issue seems to be with the API and not the provider itself. We had another report of a similar error when using Postman and I was able to reproduce it with cURL. We're working on a fix on the API side and will update here once that ships. Given that its touching sensitive customer data (AWS Info) this may take on the order of a month or so, and again, I'll keep you posted.
Maybe I should add a request for provider api debug logging as a separate github issue?
If you wouldn't mind, could you please open an issue with the Go-SDK since thats where the HTTP call is happening. TBH I haven't really thought about logging thus far as I wasn't sure what would be useful at the initial conception of this. I think now that we have better info, we can make better logging for this and other areas that use our go package.
Thanks for the response. I circled back to the python tool I had written a year ago when we were blocked from using this provider and sure enough I'm getting the same error there as well.
I last ran this script about 2 weeks ago without any problem so it seems like something in the API has regressed since then. I've filed a support ticket with onelogin through the support channel. Fingers cross it doesn't actually take a month to fix this, as now our custom non-terraform solution is no longer working.
I'll definitely file a PR request in the Go-SDK project with regards to logging. Thanks for the tip. I'm only crawling when it comes to go-lang development, havin dabbled in writing a terraform provider myself a few years ago, otherwise I'd consider writing a PR for it -- But it seems fairly easy to do seeing as this client is based on the standard go-lang http request library which has methods for dumping requests and responses.
Cheers.
Doug
I sorta kinda remember this working a while back also. I'm literally working with my team on this as I write this and we have it logged as the highest priority task on our side.
If you're keen to try your hand at a little Go, you're welcome to submit a PR. You're right, the terraform provider is essentially a wrapper around our SDK which is a wrapper around the standard http request library. I'd take a look here and probably just log out the request body and url maybe https://github.com/onelogin/onelogin-go-sdk/blob/develop/pkg/services/olhttp/olrequest.go#L94 this olrequest.go file is where all the http magic happens.
I'm also happy to do it if you'd like to file an issue instead. Your call 🙂
I filed the issue for request/response logging here -- https://github.com/onelogin/onelogin-go-sdk/issues/37
Regarding the API issue, thanks for looking into it. I wonder if the other reported issue of this you mention was me reporting this earlier through the support portal. I got a response that engineering is aware of the issue and its a known bug.
Both, we had a handful of customers. We're zeroing in on the cause; and it looks like a botched SQL query as a result of another fix we rolled out recently.
Should have the fix out by tomorrow, and at worst end of the week. If that ends up falling through I'll let you know.
Great. thanks. With our python reporting the same behavior as our terraform experiment, it sounds likely we might finally be ready to convert over to using terraform. Though I don't remember -- is there import capability to pull in our exist app rules?
Unfortunately I haven't been able to add the convenience tool to the onelogin CLI for that. I think you can still use the vanilla terraform import functionality to update your terraform state manually, but you'll be on your own for managing the HCL.
I'm happy to add this resource to the importer in the onelogin CLI. I hate to ask because you've done so much already, but could you please file an issue to https://github.com/onelogin/onelogin to add app_rules as an importable resource. The case for getting more time allocated to that is stronger if the ask comes from a customer 😁
I'm unclear what the external CLI tool for import would be used for? Is that to generate the terraform HCL? I wouldn't have an expectations a tool write our HCL file. We have a fairly standard app rule format. I think our process would be:
terraform import
to import an app_ruleterraform plan
and validate there are no pending changes.that being said, there's conflicting info -- it seems you're suggesting using terraform import
and the code looks like its supports it, but the hashcorp documentation suggests its not available yet. Is the doc just out of date? What would the command line be to specify the correct app_rule to import?
ah ok so you're using the vanilla terraform import thing. Then in your case it should work with
terraform import onelogin_app_rules.<your app_rule name> <app_rule_id>
If that works for you then its likely a case that the documentation needs an update.
We do have a convenience tool as part of the OneLogin CLI that handles bootstrapping a terraform setup for you that we typically point new users to if they're getting started with terraform and already have OneLogin set up. It would pull in your OneLogin resources, import them with terraform import
and build HCL based on whatever is in .tfstate
. Its pretty experimental at this time but we do have a handful of customers who seem to enjoy using it.
@dlethin I'm happy to report that we have shipped the API side fix and I have confirmed locally that the terraform provider works as described for me with a request that looks like this:
resource "onelogin_app_rules" "rule1" {
name = "Test Developer"
app_id = [redacted]
match = "all"
enabled = "true"
conditions {
operator = "ri"
source = "has_role"
value = [redacted]
}
actions {
action = "set_role"
value = [
"arn:aws:iam::[redacted]:role/[redacted]",
]
}
}
If this works for you now, please close the issue.
This does indeed work!!!! Awesome. Thanks for the quick turnaround. I will close this issues.
I did receive an error trying to perform an import but I'll track that as a separate issue.
cheers.
I reported this as a comment on closed issue #16 but probably better to track this separately.
I'm getting an error when trying to apply the following code:
The error message I'm getting is:
Terraform version info:
I was trying this out in an attempt to test out the fix for issue #19 which had prevented us from moving forward with this provider. The fix was to introduce the
set_role_from_existing
action which and was introduced in release v0.1.14.Note I tried this with the latest version v0.1.19 and the same problem fails there too.
It appears the error is trying to tell me that there's a problem with the role arn I'm passing in the action value, but I know that arn exists, as we have other app rules in our configuration which are referencing it. For example, when I write a python script to hit the API directly to get our applications rules, the following comes back for one of them:
The same role I'm trying to set in rule position 10 (the next open position) is referenced in the rule in position 1, and that rule is working correctly and users are able to SAML assume into that particular AWS account.
I guess my next step to help troubleshot would be to attempt to create the rule via python and the APi and that might help to narrow things down.
Cheers
Doug