Error: Unsupported argument

[patlachance]

Hi, I'm trying to use this repo to deploy nginx on OpenShift cluster using the terraform-provider-k8s but it fails.

Steps to reproduce:

1. Clone this repo

$ mkdir /tmp/test
$ cd /tmp/test
$ git clone https://github.com/mingfang/terraform-k8s-modules
$ cd terraform-k8s-modules

2. Copy a kubeconfig file

$ cp $HOME/.kube/config kubeconfig

3. Run your Terraform k8s provider image in interactive mode

$ docker run -v `pwd`/kubeconfig:/kubeconfig -e KUBECONFIG=/kubeconfig \
             -v `pwd`:/docker -w /docker --rm -it \
             registry.rebelsoft.com/terraform-provider-k8s

4. Try to deploy nginx

   /docker #  cd modules/nginx
   /docker/modules/nginx # terraform init
   ....
   <truncated>
   ....
   /docker/modules/nginx # terraform apply
   var.name
   Enter a value: testnginx

But then I get the following errors.

Error: Unsupported argument

  on .terraform/modules/deployment-service/archetypes/deployment-service/deployment.tf line 482, in resource "k8s_apps_v1_deployment" "this":
 482:                 proc_mount                = lookup(security_context.value, "proc_mount", null)

An argument named "proc_mount" is not expected here.

Error: Unsupported argument

  on .terraform/modules/deployment-service/archetypes/deployment-service/deployment.tf line 518, in resource "k8s_apps_v1_deployment" "this":
 518:                 sub_path_expr     = lookup(volume_mounts.value, "sub_path_expr", null)

An argument named "sub_path_expr" is not expected here.

Error: Unsupported argument

  on .terraform/modules/deployment-service/archetypes/deployment-service/deployment.tf line 540, in resource "k8s_apps_v1_deployment" "this":
 540:         enable_service_links = lookup(local.k8s_apps_v1_deployment_parameters, "enable_service_links", null)

An argument named "enable_service_links" is not expected here.

Error: Unsupported argument

  on .terraform/modules/deployment-service/archetypes/deployment-service/deployment.tf line 812, in resource "k8s_apps_v1_deployment" "this":
 812:                 proc_mount                = lookup(security_context.value, "proc_mount", null)

An argument named "proc_mount" is not expected here.

Error: Unsupported argument

  on .terraform/modules/deployment-service/archetypes/deployment-service/deployment.tf line 848, in resource "k8s_apps_v1_deployment" "this":
 848:                 sub_path_expr     = lookup(volume_mounts.value, "sub_path_expr", null)

An argument named "sub_path_expr" is not expected here.

Error: Unsupported argument

  on .terraform/modules/deployment-service/archetypes/deployment-service/deployment.tf line 866, in resource "k8s_apps_v1_deployment" "this":
 866:         runtime_class_name = lookup(local.k8s_apps_v1_deployment_parameters, "runtime_class_name", null)

An argument named "runtime_class_name" is not expected here.

Error: Unsupported block type

  on .terraform/modules/deployment-service/archetypes/deployment-service/deployment.tf line 989, in resource "k8s_apps_v1_deployment" "this":
 989:             dynamic "csi" {

Blocks of type "csi" are not expected here.

Error: Unsupported argument

  on .terraform/modules/deployment-service/archetypes/deployment-service/deployment.tf line 1237, in resource "k8s_apps_v1_deployment" "this":
1237:                 tenant    = lookup(quobyte.value, "tenant", null)

An argument named "tenant" is not expected here.

What am I missing? Thanks for your help!

[mingfang]

Please post the output of kubectl version

[mingfang]

Also please run the nginx example instead. https://github.com/mingfang/terraform-k8s-modules/tree/master/examples/nginx

The modules are not intended to be run directly and will keep prompting you for input, which is annoying to say to least.

[patlachance]

Please post the output of kubectl version

I'm using your registry.rebelsoft.com/terraform-provider-k8s image and there is no kubectl utility in it.

But I do have it on the docker host used.

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.3", GitCommit:"b3cbbae08ec52a7fc73d334838e18d17e8512749", GitTreeState:"clean", BuildDate:"2019-11-13T11:23:11Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0+d4cacc0", GitCommit:"d4cacc0", GitTreeState:"clean", BuildDate:"2019-02-08T01:20:07Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}

The server is running kubernetes 1.11. Your [terraform-provider-k8s}(https://github.com/mingfang/terraform-provider-k8s) doc indicate using kubernetes v1.14+ is recommended. Is it mandatory?

Kubernetes v1.14+ (Recommended for best CRD support)

[patlachance]

Also please run the nginx example instead. https://github.com/mingfang/terraform-k8s-modules/tree/master/examples/nginx

Thanks for adding this new example. Same error message.

[mingfang]

The plugin itself does not require v1.14+ but the nginx module extends this base module https://github.com/mingfang/terraform-k8s-modules/tree/master/archetypes/deployment-service that appears not to be compatible with v1.11.

I added a "plain" example that doesn't extend any base modules and should be work in v1.11. https://github.com/mingfang/terraform-k8s-modules/tree/master/examples/nginx-plain

[patlachance]

It worked using the nginx-plain example! I can see the following message at the top of the archetypes/deployment-service/deployment.tf, would it be possible for me to generate a specific one for kubernetes v1.11?

//GENERATE DYNAMIC//k8s_apps_v1_deployment////

How do you estimate the effort to disable v1.12+ features from your code? What would be the steps if I want to go down that road ?

Thanks a lot for your help!

[mingfang]

Here are the instructions for you to regenerate the code for Kubernetes v1.11. 1- pull the new image

docker pull registry.rebelsoft.com/terraform-provider-k8s

2- run the container as you did before

docker run -v `pwd`/kubeconfig:/kubeconfig -e KUBECONFIG=/kubeconfig -v `pwd`:/docker -w /docker --rm -it  registry.rebelsoft.com/terraform-provider-k8s

3- run this script

for dir in archetypes modules; do   grep -r '//GENERATE DYNAMIC' $dir|sed 's|://|//|'|awk -F'//' '{printf "%s %s \"%s\" \"%s\"\n", $1, $3, $4, $5}'|xargs -n 4 -r sh -cx 'generator -dynamic -count "$2" -lifecycle "$3" $1 > $0'; done

4- change the module source to use relative path. e.g. from

source  = "git::https://github.com/mingfang/terraform-k8s-modules.git//archetypes/deployment-service"

to

source  = "../archetypes/deployment-service"

[patlachance]

Thanks for the instructions. I still have an error message after regenerating the files. Here's what I did to try to deploy examples/nginx:

1. execute instruction as stated in previous post
   • pulled the new image
   • entered the container
   • ran the script

root@9c1af7438bb6:/docker# for dir in archetypes modules; do   grep -r '//GENERATE DYNAMIC' $dir|sed 's|://|//|'|awk -F'//' '{printf "%s %s \"%s\" \"%s\"\n", $1, $3, $4, $5}'|xargs -n 4 -r sh -cx 'generator -dynamic -count "$2" -lifecycle "$3" $1 > $0'; done
+ generator -dynamic -count  -lifecycle  k8s_core_v1_service
2019/11/19 05:43:00 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count  -lifecycle  k8s_apps_v1_deployment
2019/11/19 05:43:00 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count  -lifecycle  k8s_apps_v1_stateful_set
2019/11/19 05:43:00 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count  -lifecycle  k8s_core_v1_service
2019/11/19 05:43:01 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count  -lifecycle  k8s_apps_v1_daemon_set
2019/11/19 05:43:01 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count  -lifecycle  k8s_extensions_v1beta1_ingress
2019/11/19 05:43:01 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count count = var.cluster_role_rules == null ? 0 : 1 -lifecycle  k8s_rbac_authorization_k8s_io_v1_cluster_role_binding
2019/11/19 05:43:01 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count count = var.role_rules == null ? 0 : 1 -lifecycle  k8s_rbac_authorization_k8s_io_v1_role_binding
2019/11/19 05:43:01 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count  -lifecycle  k8s_core_v1_service_account
2019/11/19 05:43:01 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count count = var.cluster_role_rules == null ? 0 : 1 -lifecycle  k8s_rbac_authorization_k8s_io_v1_cluster_role
2019/11/19 05:43:01 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority
+ generator -dynamic -count count = var.role_rules == null ? 0 : 1 -lifecycle  k8s_rbac_authorization_k8s_io_v1_role
2019/11/19 05:43:01 Get https://console.example.org/openapi/v2?timeout=32s: x509: certificate signed by unknown authority

===> Any idea why I get these unknown authority message? Certificate was issued by Let's Encrypt and is still valid

$ echo | openssl s_client -servername console.example.com -connect console.exampe.com:443 2>/dev/null | openssl x509 -noout -dates --issuer
notBefore=Sep  6 04:39:54 2019 GMT
notAfter=Dec  5 04:39:54 2019 GMT
issuer=C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3

2. edit the modules/nginx file

$ git diff main.tf
diff --git a/modules/nginx/main.tf b/modules/nginx/main.tf
index f0e579a..0fe192d 100644
--- a/modules/nginx/main.tf
+++ b/modules/nginx/main.tf
@@ -35,6 +35,6 @@ locals {
 }

 module "deployment-service" {
-  source     = "git::https://github.com/mingfang/terraform-k8s-modules.git//archetypes/deployment-service"
+  source     = "../../archetypes/deployment-service"
   parameters = merge(local.parameters, var.overrides)
 }

3. initialize with terraform

root@d8e47e550019:/docker/examples/nginx# rm -rf .terraform
root@d8e47e550019:/docker/examples/nginx# terraform init
Initializing modules...
- nginx in ../../modules/nginx
- nginx.deployment-service in ../../archetypes/deployment-service

Initializing the backend...

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

4. deploy the solution

root@d8e47e550019:/docker/examples/nginx# terraform apply

Error: Failed to instantiate provider "k8s" to obtain schema: Unrecognized remote plugin message: 

This usually means that the plugin is either invalid or simply
needs to be recompiled to support the latest protocol.

[mingfang]

That looks like something wrong with your certificate. My guess is you need to install a custom CA bundle. I'm unable to hit openssl s_client -servername console.example.com -connect console.exampe.com:443 to troubleshoot. However I was able to run curl -I https://helloworld.letsencrypt.org fine so looks like letsencrypt does work.

I believe this is an environment issue and one which I do not know how to resolve.

[mingfang]

Perhaps this is related https://github.com/rancher/rancher/issues/18585

[patlachance]

That looks like something wrong with your certificate. My guess is you need to install a custom CA bundle.

Thanks for the suggestion that put me on track. I could get rid of the SSL errors by adding os ca-certificates missing from your container image.

root@3830cefc50ed:/docker# apt-get install ca-certificates

Now error message is different:

root@3830cefc50ed:/docker/examples/nginx# terraform apply

Error: Reference to undeclared resource

  on ../../archetypes/deployment-service/outputs.tf line 2, in output "name":
   2:   value = k8s_core_v1_service.this.metadata.0.name

A managed resource "k8s_core_v1_service" "this" has not been declared in
nginx.deployment-service.

Error: Reference to undeclared resource

  on ../../archetypes/deployment-service/outputs.tf line 6, in output "service":
   6:   value = k8s_core_v1_service.this

A managed resource "k8s_core_v1_service" "this" has not been declared in
nginx.deployment-service.

Error: Reference to undeclared resource

  on ../../archetypes/deployment-service/outputs.tf line 11, in output "deployment":
  11:   value = k8s_apps_v1_deployment.this

A managed resource "k8s_apps_v1_deployment" "this" has not been declared in
nginx.deployment-service.

[mingfang]

I added apt-get install ca-certificates to the image. It's likely the previous cert errors left things in a bad state. Can you revert your changes to the archetypes directory, pull the image again and then repeat the 4 steps above?

[patlachance]

Now it worked! Thanks a lot for your help and reactivity!