search

k3d-io / k3d

Little helper to run CNCF's k3s in Docker
https://k3d.io/
MIT License
5.45k stars 461 forks source link

[BUG] output should not be assumed to by a tty #950

Closed myitcv closed 2 years ago

myitcv commented 2 years ago

Thanks very much for creating k3d. We are using it in an upcoming live demo, for FOSDEM 2022, that we are sharing for others to be able to run locally... and unfortunately this bug is a show-stopper so would appreciate any thoughts/workarounds in the short term.

What did you do

k3d registry create registry.acme.com --port 5000
k3d cluster create newcluster --registry-use k3d-registry.acme.com:5000

n/a. Just looked at the output of the above commands.

What did you expect to happen

For the output not to use control characters by default. The issue appears to be rooted in the hardcoding of coloured output:

https://github.com/rancher/k3d/blob/793fbce86c225033188bd992b753fa79eb6e2fb2/cmd/root.go#L188

I realise however that with docker and other commands that ship has somewhat sailed to have coloured output by default.

Therefore, perhaps a more realistic expectation would be to be able to turn off coloured output via an environment variable and/or flag. Much like docker buildx build --progress plain. Ideally both a flag and environment variable would be supported. In our situation we would prefer to set an environment variable because we run k3d within a scripted setup where the output, whilst a tty, should not be coloured and hence it should not include any control characters.

Screenshots or terminal output

Output from above is:

$ k3d registry create registry.acme.com --port 5000
INFO[0000] Creating node 'k3d-registry.acme.com'        
INFO[0000] Successfully created registry 'k3d-registry.acme.com' 
INFO[0000] Starting Node 'k3d-registry.acme.com'        
INFO[0000] Successfully created registry 'k3d-registry.acme.com' 
# You can now use the registry like this (example):
# 1. create a new cluster that uses this registry
k3d cluster create --registry-use k3d-registry.acme.com:5000

# 2. tag an existing local image to be pushed to the registry
docker tag nginx:latest k3d-registry.acme.com:5000/mynginx:v0.1

# 3. push that image to the registry
docker push k3d-registry.acme.com:5000/mynginx:v0.1

# 4. run a pod that uses this image
kubectl run mynginx --image k3d-registry.acme.com:5000/mynginx:v0.1

$ k3d cluster create newcluster --registry-use k3d-registry.acme.com:5000
INFO[0000] Prep: Network                                
INFO[0000] Re-using existing network 'k3d-newcluster' (eb0f3ec8b79f7b90154679297ebb58ae0bca37f009f9fc6cde5b5b914d6d9d99) 
INFO[0000] Created volume 'k3d-newcluster-images'       
INFO[0000] Starting new tools node...                   
INFO[0000] Starting Node 'k3d-newcluster-tools'         
INFO[0001] Creating node 'k3d-newcluster-server-0'      
INFO[0001] Creating LoadBalancer 'k3d-newcluster-serverlb' 
INFO[0001] Using the k3d-tools node to gather environment information 
INFO[0001] HostIP: using network gateway 172.22.0.1 address 
INFO[0001] Starting cluster 'newcluster'                
INFO[0001] Starting servers...                          
INFO[0001] Starting Node 'k3d-newcluster-server-0'      
INFO[0005] All agents already running.                  
INFO[0005] Starting helpers...                          
INFO[0005] Starting Node 'k3d-newcluster-serverlb'      
INFO[0011] Injecting '172.22.0.1 host.k3d.internal' into /etc/hosts of all nodes... 
INFO[0011] Injecting records for host.k3d.internal and for 3 network members into CoreDNS configmap... 
INFO[0012] Cluster 'newcluster' created successfully!   
INFO[0012] You can now use it like this:

Which OS & Architecture

linux/arm64

Which version of k3d

$ k3d version
k3d version v5.2.2
k3s version v1.21.7-k3s1 (default)

Which version of docker

$ docker version
Client: Docker Engine - Community
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:45:04 2021
 OS/Arch:           linux/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:43:40 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
- [main|origin/main] : /home/myitcv/dev/cuelang/fosdem2022example -
$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.7.1-docker)

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 310
 Server Version: 20.10.12
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: w7f1pbzz3tys5sahrzw9s1cbv
  Is Manager: true
  ClusterID: t3uaglmigjoe2rlquld6z4jvq
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 192.168.223.129
  Manager Addresses:
   192.168.223.129:2377
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.16.2-051602-generic
 Operating System: Ubuntu Jammy Jellyfish (development branch)
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 45.85GiB
 Name: home
 ID: U7RP:NEPO:OTQI:VLSY:EACQ:ERBC:OACC:AYEF:YKED:NRTL:HM63:OBQC
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
iwilltry42 commented 2 years ago

Hi @myitcv , thanks for opening this issue! Sure, it should be possible to disable this behavior :+1: Since the community is the most important piece of k3d, I fixed it on the spot and it will be released in v5.3.0 scheduled for Thursday. What will your talk at FOSDEM be about?

myitcv commented 2 years ago

I fixed it on the spot and it will be released in v5.3.0 scheduled for Thursday.

Wonderful. Is there any way that I can test it before then?

My current means on installing k3d is as follows:

curl -s https://raw.githubusercontent.com/rancher/k3d/v5.2.2/install.sh | TAG=v5.2.2 bash

What will your talk at FOSDEM be about?

I work on the CUE project:

https://cuelang.org/

Marcel and I are presenting on the topic: "A practical guide to CUE: patterns for everyday use"

https://fosdem.org/2022/schedule/event/cue_pratical_guide/

We are using k3d as part of our demo, and are sharing that demo so that others can play around with it locally. k3d being so lightweight etc is ideal in this situation.

iwilltry42 commented 2 years ago

@myitcv ah, amazing and super interesting project 🙂 I'll release tomorrow after polishing the code a bit, so I guess any overhead is unnecessary 👍

iwilltry42 commented 2 years ago

@myitcv rc.2 is released that you can use for testing. The final release is just missing a proper changelog and compatibility matrix. No more code changes to go into it 👍

myitcv commented 2 years ago

@iwilltry42 looks like you mistagged the release?

https://github.com/rancher/k3d/tree/v3.5.0-rc.2

i.e. v3.5.0-rc.2 should be v5.3.0-rc.2?

iwilltry42 commented 2 years ago

Whoops.. yep 😬 I was just doing that at the side to have a release ready for you to test.. Will re-release later.

myitcv commented 2 years ago

No worries!

I've just tested the release in any case and spotted an issue. I think this comparison should be inverted:

https://github.com/rancher/k3d/blob/a8f937fa01c3abf7740d1c70edab830706267e1c/cmd/root.go#L159

Also, if that issue is fixed, I'm seeing quite verbose, unfriendly logging when LOG_COLORS=false:

$ LOG_COLORS=false LOG_LEVEL=info /tmp/tmp.SAYk3kAPEL/k3d/bin/k3d cluster delete newcluster
time="2022-02-03T08:59:16Z" level=info msg="Deleting cluster 'newcluster'"
time="2022-02-03T08:59:16Z" level=info msg="Deleting 2 attached volumes..."
time="2022-02-03T08:59:16Z" level=warning msg="Failed to delete volume 'k3d-newcluster-images' of cluster 'failed to find volume 'k3d-newcluster-images': Error: No such volume: k3d-newcluster-images': newcluster -> Try to delete it manually"
time="2022-02-03T08:59:16Z" level=info msg="Removing cluster details from default kubeconfig..."
time="2022-02-03T08:59:16Z" level=info msg="Removing standalone kubeconfig file (if there is one)..."
time="2022-02-03T08:59:16Z" level=info msg="Successfully deleted cluster newcluster!"

My own preference would be to drop the timestamps and the level from the output to instead give:

$ LOG_COLORS=false LOG_LEVEL=info /tmp/tmp.SAYk3kAPEL/k3d/bin/k3d cluster delete newcluster
Deleting cluster 'newcluster'
Deleting 2 attached volumes...
Failed to delete volume 'k3d-newcluster-images' of cluster 'failed to find volume 'k3d-newcluster-images': Error: No such volume: k3d-newcluster-images': newcluster -> Try to delete it manually
Removing cluster details from default kubeconfig...
Removing standalone kubeconfig file (if there is one)...
Successfully deleted cluster newcluster!

Thoughts?

myitcv commented 2 years ago

I raised https://github.com/rancher/k3d/pull/958 in case it's helpful. Please feel free to disregard if you have your own fix.

iwilltry42 commented 2 years ago

Yep, thanks for the fix @myitcv ! Release for v5.3.0-rc.1 is running now. I'm quite busy today so won't cut the final release yet.

myitcv commented 2 years ago

Thanks @iwilltry42: rc.1 looks good.