[Bug] Invalid character '<' looking for beginning of value

[torbendury]

Hello all,

we're lately experiencing problems when using the CloudAMQP provider. We're running the latest version, but it has also been happening with some earlier versions.

When running terraform plan, we sometimes see the following error:

│ Error: invalid character '<' looking for beginning of value
│ 
│   with module.cloudamqp["xxxxxxxxxx"].cloudamqp_plugin.this["rabbitmq_prometheus"],
│   on .terraform/modules/cloudamqp/cloudamqp-instance.tf line 17, in resource "cloudamqp_plugin" "this":
│   17: resource "cloudamqp_plugin" "this" {
│ 
╵

Those error messages are not limited to CloudAMQP plugins but also happen on instances, alerts, notification channels, ... - since we deploy our whole infrastructure with our pipelines and CloudAMQP is only one component of it, this is becoming very time consuming due to the pipeline needing to be restarted etc.

I guess that '<' hints to some HTML error being presented by the API, but I can't seem to find out what error exactly we are being presented.

Do you have any tips on how to further debug this? Are we calling the API too fast and you are applying e.g. some kind of rate limiting?

[dentarg]

Are you able to give us specific times when this has happened? We can debug it further via the support system if you want: https://www.cloudamqp.com/support.html

[crisp2u]

we are also seeing this, but it's intermittent. Most probaly a plain html error coming from the API, it would show up as timeout or error on your side

[dentarg]

@crisp2u are you able to tell us the specific times you have seen this? You can use the support email an reference this issue

[crisp2u]

@dentarg This is the time stamp from our PR that failed: Apr 26, 2023, 6:53 PM GMT+3

[tbroden84]

@crisp2u thanks for the timestamp, will ease narrowing down the time range from our logs. We will start investigate the issue.

[tbroden84]

Correct, we have been seeing increased request timeouts that will affect the terraform plan in these cases to fail. So far we haven't been able to find the root cause but we will keep on investigate this.

[dentarg]

We have made a few adjustments to our backend. Please let us know if (and when if so) you are still seeing this.

[dentarg]

Looking at our internal metrics, the situation looks better now. Please reach out again if you experience any problems.