Add --keep-temp-files flag to kctrl dev

gberche-orange commented 1 year ago

Describe the problem/challenge you have

Trying to debug the behavior of kappcontroller with kctrl dev, the files passed to ytt are temporary and deleted upon completion of the command. As a result, it pretty hard to inspect them and interactively rerun the corresponding ytt command

Describe the solution you'd like

A new --keep-temp-files flag to kctrl dev to prevent deleting the temporary directory

Anything else you would like to add:

$ kctrl dev -l -f postgresql-composition-app.yaml
[...]
econciling in-memory app/postgresql-composition-ytt-app (namespace: 75-crossplane-resources-ytt) ...
==> Executing /usr/local/bin/vendir [vendir sync -f - --lock-file /dev/null]            
==> Finished executing /usr/local/bin/vendir                                                                                                                                                                                                                                                                                                                                                                                          ==> Executing /usr/local/bin/ytt [ytt -f /tmp/kapp-controller-fetch-template-deploy2547740875/0/config --data-values-file /tmp/kapp-controller-template-values437011557/external_gcp_poc_openshift_cluster_vpc_netw
ork --data-values-file /tmp/kapp-controller-template-values437011557/external_gcp_poc_route_domain --data-values-file /tmp/kapp-controller-template-values437011557/external_gcp_project_id --data-values-file /tmp
/kapp-controller-template-values437011557/external_gcp_region --data-values-file /tmp/kapp-controller-fetch-template-deploy2547740875/0/values/values.yaml]
==> Finished executing /usr/local/bin/ytt

12:39:59PM: Fetch started (1s ago)                                                                                                                                                                                 
12:40:00PM: Fetching                             
            | apiVersion: vendir.k14s.io/v1alpha1                                                                                                                                                                  
            | directories:                          
            | - contents:                                                                                
            |   - git:                                                                                   
            |       commitTitle: 'remove manual pvc creation #2...'
            |       sha: d1b65f266e5078c91ae453c0a72f191835eeff8e
            |     path: .             
            |   path: "0"               
            | kind: LockConfig
            |               
12:40:00PM: Fetch succeeded                                                                              
12:40:00PM: Template failed                    
            | ytt: Error: Overlaying data values (in following order: additional data values):
            |   Document on line values.yaml:1:
            |     Expected map, but was string                                                           
            |                           
            | Templating dir: Error (see .status.usefulErrorMessage for details)                      

App tailing error: Reconciling app: Template failed

Succeeded                                                                                           

$ ls -al /tmp/kapp-controller-fetch-template-deploy2547740875/
ls: cannot access '/tmp/kapp-controller-fetch-template-deploy2547740875/': No such file or directory

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

100mik commented 1 year ago

This has been on our mind for a bit. Giving users the ability to inspect what has happened after each stage of reconciliation.

Our initial thought was to allow users to inspect the result after the templating step. But I see how it might be helpful to see fetched resources as well.

Could you share what the spec looks like? I might be able to help you reproduce this while we iron out what the experience for mimicking a reconciliation step by step should look like!

Edit: By spec, I meant the spec of the app CR in question

gberche-orange commented 1 year ago

Thanks @100mik for your help !

Could you share what the spec looks like? I might be able to help you reproduce this while we iron out what the experience for mimicking a reconciliation step by step should look like!

Finally solved, see full details in slack thread https://kubernetes.slack.com/archives/CH8KCCKA5/p1673453880059389

100mik commented 1 year ago

Awesome! I will leave this issue up for the time being because we were thinking about how we can help users do "dry runs" of reconciliations better. Maybe we can close it if we have another issue tracking this thought 🚀 It might be worth spending some time thinking about what the experience should look like and if we can find better ways of achieving the goal here.

100mik commented 1 year ago

(I was putting this together, I will leave it here in case someone else trying to do something similar comes across this)

If you have the CLIs vendir and ytt installed you can replicate what you are doing with the following steps:

1. Copy the vendir config seen in the output

apiVersion: vendir.k14s.io/v1alpha1                                                                                                                                                                  
directories:                          
- contents:                                                                                
  - git:                                                                                   
      commitTitle: 'remove manual pvc creation #2...'
      sha: d1b65f266e5078c91ae453c0a72f191835eeff8e
    path: .             
path: "0"               
kind: LockConfig

2. Put this in a file named `vendir.yml` in a new folder.

3. Run `vendir sync` in this folder.

After this step the folder config (and any other folders being fetched) will be visible in this folder

2. Put your values in a file.

Lets say values.yml is the file we put our values in.

3. Now to run the `ytt` step described in the app spec to see the result

Would look like ytt -f config --data-values-file values.yml in this case

gberche-orange commented 1 year ago

thanks @100mik

I do believe that the displayed commands are quite useful, however reproducing the vendir commands seems to still require some work.

$ cat vendir.yml 
apiVersion: vendir.k14s.io/v1alpha1                                                                                                                                                                  
directories:                          
- contents:                                                                                
  - git:                                                                                   
      commitTitle: 'remove manual pvc creation #2...'
      sha: d1b65f266e5078c91ae453c0a72f191835eeff8e
    path: .             
path: "0"               
kind: LockConfig

$ vendir sync
vendir: Error: Parsing resource config 'vendir.yml':
  Unknown apiVersion 'vendir.k14s.io/v1alpha1' or kind 'LockConfig' for resource

$ vendir version
vendir version 0.32.2

Succeeded

After changing LockConfig into Config:

$ vendir sync
vendir: Error: Parsing resource config 'vendir.yml':
  Unmarshaling config:
    Validating config:
      Validating directory '' (0):
        Expected path to not be one of '/', '.', '..', ''

$ cat vendir.yml 
apiVersion: vendir.k14s.io/v1alpha1                                                                                                                                                                  
directories:                          
- contents:                                                                                
  - git:                                                                                   
      commitTitle: 'remove manual pvc creation #2...'
      sha: d1b65f266e5078c91ae453c0a72f191835eeff8e
    path: .             
path: "0"               
kind: Config

Simply being able to inspect the current fetched artifacts and rerun the displayed ytt commands seems to me a very useful 1st step.

100mik commented 1 year ago

Oops! I thought we were printing the actual vendir config instead of the output. I definitely agree that it will be a valuable feature. We do need to hammer out how the experience will be like.

I am going to add the triage label and we should prioritise this when we have some bandwidth 🚀

carvel-dev / kapp-controller