Issue in eu-de region when using phillbaker/elasticsearch provider for creating index and cluster settings

[Ak-sky]

Basic example is failing everytime in ~eu-de~ all regions when elasticsearch_index and elasticsearch_cluster_settings are getting created using below elasticsearch provider. https://github.com/terraform-ibm-modules/terraform-ibm-icd-elasticsearch/blob/0df97ee41fca1b08b9d8c077f75a403f51de944a/examples/basic/version.tf#L11

Failing with below error context deadline exceeded

TestRunBasicExample 2024-10-23T06:35:14Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; ╷
│ Error: Head "https://4a92398f-969b-44b3-a9b2-44a791571995.6131b73286f34215871dfad7254b4f7d.databases.appdomain.cloud:31855/": context deadline exceeded
│ 
│   with elasticsearch_index.test,
│   on main.tf line 34, in resource "elasticsearch_index" "test":
│   34: resource "elasticsearch_index" "test" {
│ 
╵
╷
│ Error: Head "https://4a92398f-969b-44b3-a9b2-44a791571995.6131b73286f34215871dfad7254b4f7d.databases.appdomain.cloud:31855/": context deadline exceeded
│ 
│   with elasticsearch_cluster_settings.global,
│   on main.tf line 42, in resource "elasticsearch_cluster_settings" "global":
│   42: resource "elasticsearch_cluster_settings" "global" {
│ 
╵}

Have tried to wait (sleep) for 60s, 120s, and so forth but it doesn't work. But sometimes TF re-apply does work.

[ocofaigh]

@Ak-sky re: But sometimes TF re-apply does work

^ Do you have an environment where the re-apply never works? I believe its just that the backend is not actually ready yet. And we need to wait longer. We could maybe add a readiness script to the module if thats the case. But if you are seeing examples where re-apply never works, then thats a different problem. Can you please confirm?

[Ak-sky]

@Ak-sky re: But sometimes TF re-apply does work

^ Do you have an environment where the re-apply never works? I believe its just that the backend is not actually ready yet. And we need to wait longer. We could maybe add a readiness script to the module if thats the case. But if you are seeing examples where re-apply never works, then thats a different problem. Can you please confirm?

It does work on re-apply but what I have seen is ES instance status is active even then it was always failing.

[ocofaigh]

@Ak-sky Active may not mean its ready to handle index creation. Perhaps there is readiness api we could hit? We could even reach out to ICD to see what they recommend, since it seems to definitely be that the instance just isn't ready yet