kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.23k stars 4.87k forks source link

`minikube start` with `--mount` and `--mount-string` fails with `GUEST_MOUNT_CONFLICT` #11084

Closed jmholla closed 3 years ago

jmholla commented 3 years ago

Steps to reproduce the issue:

  1. minikube stop
  2. docker rm minikube
  3. minikube start --mount --mount-string="$HOME:/src" --driver=docker

I would expect the mount to happen during container creation.

This is on Linux Mint.

Full output of failed command:

(I'll attempt to attach the output with --alsologtostderr as a file because it keeps repeatedly spitting out the same complaint about the container not existing which will just fill up this issue.)

πŸ˜„  minikube v1.18.0 on Linuxmint 20.1
✨  Using the docker driver based on existing profile
πŸ‘  Starting control plane node minikube in cluster minikube
🀷  docker "minikube" container is missing, will recreate.
πŸ”₯  Creating docker container (CPUs=2, Memory=3900MB) ...\ ^[[

❌  Exiting due to GUEST_MOUNT_CONFLICT: Sorry, docker does not allow mounts to be changed after container creation (previous mount: '', new mount: '/home/jholland:/src)'

Full output of minikube start command used, if not already included:

See above

Optional: Full output of minikube logs command:

Way too much to share
jmholla commented 3 years ago

Whelp, there's no ability to attach files, so here is the output of --alsologtostderr:

``` I0412 16:38:53.034368 326971 out.go:229] Setting OutFile to fd 1 ... I0412 16:38:53.034501 326971 out.go:281] isatty.IsTerminal(1) = false I0412 16:38:53.034509 326971 out.go:242] Setting ErrFile to fd 2... I0412 16:38:53.034516 326971 out.go:281] isatty.IsTerminal(2) = false I0412 16:38:53.034624 326971 root.go:308] Updating PATH: /home/jholland/.minikube/bin W0412 16:38:53.034737 326971 root.go:283] Error reading config file at /home/jholland/.minikube/config/config.json: open /home/jholland/.minikube/config/config.json: no such file or directory I0412 16:38:53.034900 326971 out.go:236] Setting JSON to false I0412 16:38:53.056897 326971 start.go:108] hostinfo: {"hostname":"jholland-dev-sirius","uptime":346750,"bootTime":1617916783,"procs":415,"os":"linux","platform":"linuxmint","platformFamily":"debian","platformVersion":"20.1","kernelVersion":"5.4.0-70-generic","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"host","hostId":"b76cc7b1-bbdc-489e-9390-9d2043031de8"} I0412 16:38:53.056988 326971 start.go:118] virtualization: kvm host I0412 16:38:53.058505 326971 out.go:119] * minikube v1.18.0 on Linuxmint 20.1 * minikube v1.18.0 on Linuxmint 20.1 I0412 16:38:53.059450 326971 driver.go:323] Setting default libvirt URI to qemu:///system I0412 16:38:53.111684 326971 docker.go:118] docker version: linux-20.10.6 I0412 16:38:53.111811 326971 cli_runner.go:115] Run: docker system info --format "{{json .}}" I0412 16:38:53.208435 326971 info.go:253] docker info: {ID:LSOP:MTAE:JGZA:RJW5:7LUO:JCQZ:237H:7QFH:2Z3S:3L5J:NENH:2753 Containers:31 ContainersRunning:0 ContainersPaused:0 ContainersStopped:31 Images:77 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus: Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization: Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:false KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:22 OomKillDisable:true NGoroutines:34 SystemTime:2021-04-12 16:38:53.150321701 -0500 CDT LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.4.0-70-generic OperatingSystem:Linux Mint 20.1 OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:16664735744 GenericResources: DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:jholland-dev-sirius Labels:[] ExperimentalBuild:false ServerVersion:20.10.6 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:05f951a3781f4f2c1911b05e61c160e9c30eaa8e Expected:05f951a3781f4f2c1911b05e61c160e9c30eaa8e} RuncCommit:{ID:12644e614e25b05da6fd08a38ffa0cfe1903fdec Expected:12644e614e25b05da6fd08a38ffa0cfe1903fdec} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:[WARNING: No swap limit support] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Build with BuildKit Vendor:Docker Inc. Version:v0.5.1-docker] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.7.0]] Warnings:}} I0412 16:38:53.208563 326971 docker.go:215] overlay module found I0412 16:38:53.209605 326971 out.go:119] * Using the docker driver based on existing profile * Using the docker driver based on existing profile I0412 16:38:53.209627 326971 start.go:276] selected driver: docker I0412 16:38:53.209638 326971 start.go:718] validating driver "docker" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e Memory:3900 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[ambassador:false auto-pause:false csi-hostpath-driver:false dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false helm-tiller:false ingress:false ingress-dns:false istio:false istio-provisioner:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false volumesnapshots:false] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] Network: MultiNodeRequested:false} I0412 16:38:53.209735 326971 start.go:729] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error: Reason: Fix: Doc:} I0412 16:38:53.209874 326971 cli_runner.go:115] Run: docker system info --format "{{json .}}" I0412 16:38:53.305125 326971 info.go:253] docker info: {ID:LSOP:MTAE:JGZA:RJW5:7LUO:JCQZ:237H:7QFH:2Z3S:3L5J:NENH:2753 Containers:31 ContainersRunning:0 ContainersPaused:0 ContainersStopped:31 Images:77 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus: Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization: Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:false KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:22 OomKillDisable:true NGoroutines:34 SystemTime:2021-04-12 16:38:53.246101976 -0500 CDT LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.4.0-70-generic OperatingSystem:Linux Mint 20.1 OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:16664735744 GenericResources: DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:jholland-dev-sirius Labels:[] ExperimentalBuild:false ServerVersion:20.10.6 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:05f951a3781f4f2c1911b05e61c160e9c30eaa8e Expected:05f951a3781f4f2c1911b05e61c160e9c30eaa8e} RuncCommit:{ID:12644e614e25b05da6fd08a38ffa0cfe1903fdec Expected:12644e614e25b05da6fd08a38ffa0cfe1903fdec} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:[WARNING: No swap limit support] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Build with BuildKit Vendor:Docker Inc. Version:v0.5.1-docker] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.7.0]] Warnings:}} I0412 16:38:53.306079 326971 start_flags.go:390] config: {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e Memory:3900 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[ambassador:false auto-pause:false csi-hostpath-driver:false dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false helm-tiller:false ingress:false ingress-dns:false istio:false istio-provisioner:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false volumesnapshots:false] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] Network: MultiNodeRequested:false} I0412 16:38:53.307066 326971 out.go:119] * Starting control plane node minikube in cluster minikube * Starting control plane node minikube in cluster minikube I0412 16:38:53.364603 326971 image.go:92] Found gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e in local docker daemon, skipping pull I0412 16:38:53.364638 326971 cache.go:116] gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e exists in daemon, skipping pull I0412 16:38:53.364652 326971 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker I0412 16:38:53.364682 326971 preload.go:105] Found local preload: /home/jholland/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v9-v1.20.2-docker-overlay2-amd64.tar.lz4 I0412 16:38:53.364691 326971 cache.go:54] Caching tarball of preloaded images I0412 16:38:53.364717 326971 preload.go:131] Found /home/jholland/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v9-v1.20.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download I0412 16:38:53.364723 326971 cache.go:57] Finished verifying existence of preloaded tar for v1.20.2 on docker I0412 16:38:53.364858 326971 profile.go:148] Saving config to /home/jholland/.minikube/profiles/minikube/config.json ... I0412 16:38:53.365032 326971 cache.go:185] Successfully downloaded all kic artifacts I0412 16:38:53.365053 326971 start.go:313] acquiring machines lock for minikube: {Name:mk87631657aa3bde0085c87c908a90c245898f1c Clock:{} Delay:500ms Timeout:10m0s Cancel:} I0412 16:38:53.365240 326971 start.go:317] acquired machines lock for "minikube" in 156.342Β΅s I0412 16:38:53.365257 326971 start.go:93] Skipping create...Using existing machine configuration I0412 16:38:53.365262 326971 fix.go:54] fixHost starting: I0412 16:38:53.365529 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:53.403250 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:38:53.403312 326971 fix.go:107] recreateIfNeeded on minikube: state= err=unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:53.403327 326971 fix.go:112] machineExists: false. err=machine does not exist I0412 16:38:53.404876 326971 out.go:119] * docker "minikube" container is missing, will recreate. * docker "minikube" container is missing, will recreate. I0412 16:38:53.404898 326971 delete.go:124] DEMOLISHING minikube ... I0412 16:38:53.404997 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:53.442760 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 W0412 16:38:53.442822 326971 stop.go:75] unable to get state: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:53.442839 326971 delete.go:129] stophost failed (probably ok): ssh power off: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:53.443251 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:53.482200 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:38:53.482284 326971 delete.go:82] Unable to get host status for minikube, assuming it has already been deleted: state: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:53.482373 326971 cli_runner.go:115] Run: docker container inspect -f {{.Id}} minikube W0412 16:38:53.523088 326971 cli_runner.go:162] docker container inspect -f {{.Id}} minikube returned with exit code 1 I0412 16:38:53.523123 326971 kic.go:343] could not find the container minikube to remove it. will try anyways I0412 16:38:53.523198 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:53.561884 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 W0412 16:38:53.561946 326971 oci.go:83] error getting container status, will try to delete anyways: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:53.562034 326971 cli_runner.go:115] Run: docker exec --privileged -t minikube /bin/bash -c "sudo init 0" W0412 16:38:53.606893 326971 cli_runner.go:162] docker exec --privileged -t minikube /bin/bash -c "sudo init 0" returned with exit code 1 I0412 16:38:53.606942 326971 oci.go:632] error shutdown minikube: docker exec --privileged -t minikube /bin/bash -c "sudo init 0": exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:54.608110 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:54.666364 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:38:54.666423 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:54.666433 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:38:54.666450 326971 retry.go:31] will retry after 552.330144ms: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:55.219469 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:55.287065 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:38:55.287130 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:55.287142 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:38:55.287160 326971 retry.go:31] will retry after 1.080381816s: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:56.368721 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:56.437366 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:38:56.437427 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:56.437439 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:38:56.437457 326971 retry.go:31] will retry after 1.31013006s: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:57.749206 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:57.822865 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:38:57.822985 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:57.823420 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:38:57.823462 326971 retry.go:31] will retry after 1.582392691s: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:59.407294 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:38:59.471067 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:38:59.471131 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:38:59.471160 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:38:59.471179 326971 retry.go:31] will retry after 2.340488664s: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:01.813602 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:39:01.879180 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:39:01.879255 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:01.879267 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:39:01.879286 326971 retry.go:31] will retry after 4.506218855s: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:06.387299 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:39:06.459263 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:39:06.459334 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:06.459348 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:39:06.459367 326971 retry.go:31] will retry after 3.221479586s: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:09.682446 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:39:09.750131 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:39:09.750192 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:09.750204 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:39:09.750222 326971 retry.go:31] will retry after 5.608623477s: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:15.360276 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:39:15.406422 326971 cli_runner.go:162] docker container inspect minikube --format={{.State.Status}} returned with exit code 1 I0412 16:39:15.406495 326971 oci.go:644] temporary error verifying shutdown: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:15.406509 326971 oci.go:646] temporary error: container minikube status is but expect it to be exited I0412 16:39:15.406532 326971 oci.go:87] couldn't shut down minikube (might be okay): verify shutdown: couldn't verify container is exited. %v: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1 stdout: stderr: Error: No such container: minikube I0412 16:39:15.406593 326971 cli_runner.go:115] Run: docker rm -f -v minikube I0412 16:39:15.447826 326971 cli_runner.go:115] Run: docker container inspect -f {{.Id}} minikube W0412 16:39:15.489178 326971 cli_runner.go:162] docker container inspect -f {{.Id}} minikube returned with exit code 1 I0412 16:39:15.489310 326971 cli_runner.go:115] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" I0412 16:39:15.530997 326971 cli_runner.go:115] Run: docker network rm minikube W0412 16:39:16.007239 326971 delete.go:139] delete failed (probably ok) I0412 16:39:16.007320 326971 fix.go:119] Sleeping 1 second for extra luck! I0412 16:39:17.008142 326971 start.go:126] createHost starting for "" (driver="docker") I0412 16:39:17.011721 326971 out.go:140] * Creating docker container (CPUs=2, Memory=3900MB) ... * Creating docker container (CPUs=2, Memory=3900MB) ... I0412 16:39:17.012051 326971 start.go:160] libmachine.API.Create for "minikube" (driver="docker") I0412 16:39:17.012195 326971 client.go:168] LocalClient.Create starting I0412 16:39:17.012364 326971 main.go:121] libmachine: Reading certificate data from /home/jholland/.minikube/certs/ca.pem I0412 16:39:17.012458 326971 main.go:121] libmachine: Decoding PEM data... I0412 16:39:17.012514 326971 main.go:121] libmachine: Parsing certificate... I0412 16:39:17.012943 326971 main.go:121] libmachine: Reading certificate data from /home/jholland/.minikube/certs/cert.pem I0412 16:39:17.013031 326971 main.go:121] libmachine: Decoding PEM data... I0412 16:39:17.013079 326971 main.go:121] libmachine: Parsing certificate... I0412 16:39:17.013855 326971 cli_runner.go:115] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" W0412 16:39:17.079474 326971 cli_runner.go:162] docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1 I0412 16:39:17.079564 326971 network_create.go:240] running [docker network inspect minikube] to gather additional debugging logs... I0412 16:39:17.079582 326971 cli_runner.go:115] Run: docker network inspect minikube W0412 16:39:17.122376 326971 cli_runner.go:162] docker network inspect minikube returned with exit code 1 I0412 16:39:17.122428 326971 network_create.go:243] error running [docker network inspect minikube]: docker network inspect minikube: exit status 1 stdout: [] stderr: Error: No such network: minikube I0412 16:39:17.122445 326971 network_create.go:245] output of [docker network inspect minikube]: -- stdout -- [] -- /stdout -- ** stderr ** Error: No such network: minikube ** /stderr ** I0412 16:39:17.122513 326971 cli_runner.go:115] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" I0412 16:39:17.167266 326971 network.go:193] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}} I0412 16:39:17.167341 326971 network_create.go:91] attempt to create network 192.168.49.0/24 with subnet: minikube and gateway 192.168.49.1 and MTU of 1500 ... I0412 16:39:17.167421 326971 cli_runner.go:115] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true minikube I0412 16:39:17.596677 326971 kic.go:101] calculated static IP "192.168.49.2" for the "minikube" container I0412 16:39:17.596886 326971 cli_runner.go:115] Run: docker ps -a --format {{.Names}} I0412 16:39:17.649535 326971 cli_runner.go:115] Run: docker volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true I0412 16:39:17.694451 326971 oci.go:102] Successfully created a docker volume minikube I0412 16:39:17.694542 326971 cli_runner.go:115] Run: docker run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e -d /var/lib I0412 16:39:18.380912 326971 oci.go:106] Successfully prepared a docker volume minikube W0412 16:39:18.380984 326971 oci.go:135] Your kernel does not support swap limit capabilities or the cgroup is not mounted. W0412 16:39:18.381000 326971 oci.go:119] Your kernel does not support memory limit capabilities or the cgroup is not mounted. I0412 16:39:18.381038 326971 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker I0412 16:39:18.381079 326971 cli_runner.go:115] Run: docker info --format "'{{json .SecurityOptions}}'" I0412 16:39:18.381079 326971 preload.go:105] Found local preload: /home/jholland/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v9-v1.20.2-docker-overlay2-amd64.tar.lz4 I0412 16:39:18.381096 326971 kic.go:168] Starting extracting preloaded images to volume ... I0412 16:39:18.381224 326971 cli_runner.go:115] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jholland/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v9-v1.20.2-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e -I lz4 -xf /preloaded.tar -C /extractDir I0412 16:39:18.511578 326971 cli_runner.go:115] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e I0412 16:39:19.887476 326971 cli_runner.go:168] Completed: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e: (1.375791244s) I0412 16:39:19.887647 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Running}} I0412 16:39:19.939764 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} I0412 16:39:19.994674 326971 cli_runner.go:115] Run: docker exec minikube stat /var/lib/dpkg/alternatives/iptables I0412 16:39:20.098902 326971 oci.go:278] the created container "minikube" has a running status. I0412 16:39:20.098947 326971 kic.go:199] Creating ssh key for kic: /home/jholland/.minikube/machines/minikube/id_rsa... I0412 16:39:20.319633 326971 kic_runner.go:188] docker (temp): /home/jholland/.minikube/machines/minikube/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes) I0412 16:39:20.585753 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} I0412 16:39:20.655107 326971 kic_runner.go:94] Run: chown docker:docker /home/docker/.ssh/authorized_keys I0412 16:39:20.655148 326971 kic_runner.go:115] Args: [docker exec --privileged minikube chown docker:docker /home/docker/.ssh/authorized_keys] I0412 16:39:33.632639 326971 cli_runner.go:168] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jholland/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v9-v1.20.2-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.18@sha256:ddd0c02d289e3a6fb4bba9a94435840666f4eb81484ff3e707b69c1c484aa45e -I lz4 -xf /preloaded.tar -C /extractDir: (15.251318608s) I0412 16:39:33.632843 326971 kic.go:177] duration metric: took 15.251733 seconds to extract preloaded images to volume I0412 16:39:33.633223 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} I0412 16:39:33.695830 326971 machine.go:88] provisioning docker machine ... I0412 16:39:33.695879 326971 ubuntu.go:169] provisioning hostname "minikube" I0412 16:39:33.695956 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:33.744483 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:33.744721 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:33.744739 326971 main.go:121] libmachine: About to run SSH command: sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname I0412 16:39:33.952801 326971 main.go:121] libmachine: SSH cmd err, output: : minikube I0412 16:39:33.953062 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:34.014173 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:34.014393 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:34.014418 326971 main.go:121] libmachine: About to run SSH command: if ! grep -xq '.*\sminikube' /etc/hosts; then if grep -xq '127.0.1.1\s.*' /etc/hosts; then sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts; else echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts; fi fi I0412 16:39:34.330591 326971 main.go:121] libmachine: SSH cmd err, output: : I0412 16:39:34.330716 326971 ubuntu.go:175] set auth options {CertDir:/home/jholland/.minikube CaCertPath:/home/jholland/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jholland/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jholland/.minikube/machines/server.pem ServerKeyPath:/home/jholland/.minikube/machines/server-key.pem ClientKeyPath:/home/jholland/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jholland/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jholland/.minikube} I0412 16:39:34.330872 326971 ubuntu.go:177] setting up certificates I0412 16:39:34.330942 326971 provision.go:83] configureAuth start I0412 16:39:34.331178 326971 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube I0412 16:39:34.398853 326971 provision.go:137] copyHostCerts I0412 16:39:34.398919 326971 exec_runner.go:145] found /home/jholland/.minikube/key.pem, removing ... I0412 16:39:34.398929 326971 exec_runner.go:190] rm: /home/jholland/.minikube/key.pem I0412 16:39:34.398988 326971 exec_runner.go:152] cp: /home/jholland/.minikube/certs/key.pem --> /home/jholland/.minikube/key.pem (1679 bytes) I0412 16:39:34.399061 326971 exec_runner.go:145] found /home/jholland/.minikube/ca.pem, removing ... I0412 16:39:34.399069 326971 exec_runner.go:190] rm: /home/jholland/.minikube/ca.pem I0412 16:39:34.399096 326971 exec_runner.go:152] cp: /home/jholland/.minikube/certs/ca.pem --> /home/jholland/.minikube/ca.pem (1082 bytes) I0412 16:39:34.399149 326971 exec_runner.go:145] found /home/jholland/.minikube/cert.pem, removing ... I0412 16:39:34.399156 326971 exec_runner.go:190] rm: /home/jholland/.minikube/cert.pem I0412 16:39:34.399181 326971 exec_runner.go:152] cp: /home/jholland/.minikube/certs/cert.pem --> /home/jholland/.minikube/cert.pem (1127 bytes) I0412 16:39:34.399226 326971 provision.go:111] generating server cert: /home/jholland/.minikube/machines/server.pem ca-key=/home/jholland/.minikube/certs/ca.pem private-key=/home/jholland/.minikube/certs/ca-key.pem org=jholland.minikube san=[192.168.49.2 127.0.0.1 localhost 127.0.0.1 minikube minikube] I0412 16:39:34.614247 326971 provision.go:165] copyRemoteCerts I0412 16:39:34.614319 326971 ssh_runner.go:149] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker I0412 16:39:34.614368 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:34.650808 326971 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49237 SSHKeyPath:/home/jholland/.minikube/machines/minikube/id_rsa Username:docker} I0412 16:39:34.761443 326971 ssh_runner.go:310] scp /home/jholland/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes) I0412 16:39:34.817472 326971 ssh_runner.go:310] scp /home/jholland/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes) I0412 16:39:34.838426 326971 ssh_runner.go:310] scp /home/jholland/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes) I0412 16:39:34.859527 326971 provision.go:86] duration metric: configureAuth took 528.562898ms I0412 16:39:34.859559 326971 ubuntu.go:193] setting minikube options for container-runtime I0412 16:39:34.859800 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:34.902219 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:34.902382 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:34.902397 326971 main.go:121] libmachine: About to run SSH command: df --output=fstype / | tail -n 1 I0412 16:39:35.031670 326971 main.go:121] libmachine: SSH cmd err, output: : overlay I0412 16:39:35.031829 326971 ubuntu.go:71] root file system type: overlay I0412 16:39:35.032496 326971 provision.go:296] Updating docker unit: /lib/systemd/system/docker.service ... I0412 16:39:35.032811 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:35.098718 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:35.098889 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:35.098985 326971 main.go:121] libmachine: About to run SSH command: sudo mkdir -p /lib/systemd/system && printf %s "[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket StartLimitBurst=3 StartLimitIntervalSec=60 [Service] Type=notify Restart=on-failure # This file is a systemd drop-in unit that inherits from the base dockerd configuration. # The base configuration already specifies an 'ExecStart=...' command. The first directive # here is to clear out that command inherited from the base configuration. Without this, # the command from the base configuration and the command specified here are treated as # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd # will catch this invalid input and refuse to start the service with an error like: # Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. # NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other # container runtimes. If left unlimited, it may result in OOM issues with MySQL. ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP \$MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target " | sudo tee /lib/systemd/system/docker.service.new I0412 16:39:35.266647 326971 main.go:121] libmachine: SSH cmd err, output: : [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket StartLimitBurst=3 StartLimitIntervalSec=60 [Service] Type=notify Restart=on-failure # This file is a systemd drop-in unit that inherits from the base dockerd configuration. # The base configuration already specifies an 'ExecStart=...' command. The first directive # here is to clear out that command inherited from the base configuration. Without this, # the command from the base configuration and the command specified here are treated as # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd # will catch this invalid input and refuse to start the service with an error like: # Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. # NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other # container runtimes. If left unlimited, it may result in OOM issues with MySQL. ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target I0412 16:39:35.267111 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:35.332172 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:35.332362 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:35.332387 326971 main.go:121] libmachine: About to run SSH command: sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; } I0412 16:39:40.690287 326971 main.go:121] libmachine: SSH cmd err, output: : --- /lib/systemd/system/docker.service 2021-01-29 14:31:32.000000000 +0000 +++ /lib/systemd/system/docker.service.new 2021-04-12 21:39:35.257496621 +0000 @@ -1,30 +1,32 @@ [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com +BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target -Requires=docker.socket containerd.service +Requires=docker.socket +StartLimitBurst=3 +StartLimitIntervalSec=60 [Service] Type=notify -# the default is not to use systemd for cgroups because the delegate issues still -# exists and systemd currently does not support the cgroup feature set required -# for containers run by docker -ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock -ExecReload=/bin/kill -s HUP $MAINPID -TimeoutSec=0 -RestartSec=2 -Restart=always - -# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229. -# Both the old, and new location are accepted by systemd 229 and up, so using the old location -# to make them work for either version of systemd. -StartLimitBurst=3 +Restart=on-failure -# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230. -# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make -# this option work for either version of systemd. -StartLimitInterval=60s + + +# This file is a systemd drop-in unit that inherits from the base dockerd configuration. +# The base configuration already specifies an 'ExecStart=...' command. The first directive +# here is to clear out that command inherited from the base configuration. Without this, +# the command from the base configuration and the command specified here are treated as +# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd +# will catch this invalid input and refuse to start the service with an error like: +# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. + +# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other +# container runtimes. If left unlimited, it may result in OOM issues with MySQL. +ExecStart= +ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 +ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. @@ -32,16 +34,16 @@ LimitNPROC=infinity LimitCORE=infinity -# Comment TasksMax if your systemd version does not support it. -# Only systemd 226 and above support this option. +# Uncomment TasksMax if your systemd version supports it. +# Only systemd 226 and above support this version. TasksMax=infinity +TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process -OOMScoreAdjust=-500 [Install] WantedBy=multi-user.target Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable docker I0412 16:39:40.690575 326971 machine.go:91] provisioned docker machine in 6.994701663s I0412 16:39:40.690691 326971 client.go:171] LocalClient.Create took 23.678461143s I0412 16:39:40.690762 326971 start.go:168] duration metric: libmachine.API.Create for "minikube" took 23.67871029s I0412 16:39:40.690857 326971 start.go:267] post-start starting for "minikube" (driver="docker") I0412 16:39:40.690890 326971 start.go:277] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs] I0412 16:39:40.691192 326971 ssh_runner.go:149] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs I0412 16:39:40.691447 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:40.751631 326971 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49237 SSHKeyPath:/home/jholland/.minikube/machines/minikube/id_rsa Username:docker} I0412 16:39:40.860411 326971 ssh_runner.go:149] Run: cat /etc/os-release I0412 16:39:40.870116 326971 main.go:121] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found I0412 16:39:40.870207 326971 main.go:121] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found I0412 16:39:40.870262 326971 main.go:121] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found I0412 16:39:40.870309 326971 info.go:137] Remote host: Ubuntu 20.04.1 LTS I0412 16:39:40.870362 326971 filesync.go:118] Scanning /home/jholland/.minikube/addons for local assets ... I0412 16:39:40.870492 326971 filesync.go:118] Scanning /home/jholland/.minikube/files for local assets ... I0412 16:39:40.870574 326971 start.go:270] post-start completed in 179.686978ms I0412 16:39:40.871754 326971 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube I0412 16:39:40.932690 326971 profile.go:148] Saving config to /home/jholland/.minikube/profiles/minikube/config.json ... I0412 16:39:40.932956 326971 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'" I0412 16:39:40.933019 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:40.975412 326971 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49237 SSHKeyPath:/home/jholland/.minikube/machines/minikube/id_rsa Username:docker} I0412 16:39:41.079871 326971 start.go:129] duration metric: createHost completed in 24.071583808s I0412 16:39:41.080255 326971 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}} W0412 16:39:41.145691 326971 fix.go:133] unexpected machine state, will restart: I0412 16:39:41.145733 326971 machine.go:88] provisioning docker machine ... I0412 16:39:41.145758 326971 ubuntu.go:169] provisioning hostname "minikube" I0412 16:39:41.145840 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:41.189032 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:41.189265 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:41.189282 326971 main.go:121] libmachine: About to run SSH command: sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname I0412 16:39:41.367365 326971 main.go:121] libmachine: SSH cmd err, output: : minikube I0412 16:39:41.367706 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:41.430790 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:41.430964 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:41.430986 326971 main.go:121] libmachine: About to run SSH command: if ! grep -xq '.*\sminikube' /etc/hosts; then if grep -xq '127.0.1.1\s.*' /etc/hosts; then sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts; else echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts; fi fi I0412 16:39:41.582514 326971 main.go:121] libmachine: SSH cmd err, output: : I0412 16:39:41.582549 326971 ubuntu.go:175] set auth options {CertDir:/home/jholland/.minikube CaCertPath:/home/jholland/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jholland/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jholland/.minikube/machines/server.pem ServerKeyPath:/home/jholland/.minikube/machines/server-key.pem ClientKeyPath:/home/jholland/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jholland/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jholland/.minikube} I0412 16:39:41.582563 326971 ubuntu.go:177] setting up certificates I0412 16:39:41.582571 326971 provision.go:83] configureAuth start I0412 16:39:41.582637 326971 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube I0412 16:39:41.623614 326971 provision.go:137] copyHostCerts I0412 16:39:41.623673 326971 exec_runner.go:145] found /home/jholland/.minikube/ca.pem, removing ... I0412 16:39:41.623683 326971 exec_runner.go:190] rm: /home/jholland/.minikube/ca.pem I0412 16:39:41.623736 326971 exec_runner.go:152] cp: /home/jholland/.minikube/certs/ca.pem --> /home/jholland/.minikube/ca.pem (1082 bytes) I0412 16:39:41.623823 326971 exec_runner.go:145] found /home/jholland/.minikube/cert.pem, removing ... I0412 16:39:41.623831 326971 exec_runner.go:190] rm: /home/jholland/.minikube/cert.pem I0412 16:39:41.623860 326971 exec_runner.go:152] cp: /home/jholland/.minikube/certs/cert.pem --> /home/jholland/.minikube/cert.pem (1127 bytes) I0412 16:39:41.623913 326971 exec_runner.go:145] found /home/jholland/.minikube/key.pem, removing ... I0412 16:39:41.623920 326971 exec_runner.go:190] rm: /home/jholland/.minikube/key.pem I0412 16:39:41.623946 326971 exec_runner.go:152] cp: /home/jholland/.minikube/certs/key.pem --> /home/jholland/.minikube/key.pem (1679 bytes) I0412 16:39:41.623988 326971 provision.go:111] generating server cert: /home/jholland/.minikube/machines/server.pem ca-key=/home/jholland/.minikube/certs/ca.pem private-key=/home/jholland/.minikube/certs/ca-key.pem org=jholland.minikube san=[192.168.49.2 127.0.0.1 localhost 127.0.0.1 minikube minikube] I0412 16:39:41.755031 326971 provision.go:165] copyRemoteCerts I0412 16:39:41.755099 326971 ssh_runner.go:149] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker I0412 16:39:41.755146 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:41.790805 326971 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49237 SSHKeyPath:/home/jholland/.minikube/machines/minikube/id_rsa Username:docker} I0412 16:39:41.898654 326971 ssh_runner.go:310] scp /home/jholland/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes) I0412 16:39:41.960334 326971 ssh_runner.go:310] scp /home/jholland/.minikube/machines/server.pem --> /etc/docker/server.pem (1204 bytes) I0412 16:39:42.023939 326971 ssh_runner.go:310] scp /home/jholland/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes) I0412 16:39:42.064495 326971 provision.go:86] duration metric: configureAuth took 481.911422ms I0412 16:39:42.064543 326971 ubuntu.go:193] setting minikube options for container-runtime I0412 16:39:42.064841 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:42.109656 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:42.109839 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:42.109855 326971 main.go:121] libmachine: About to run SSH command: df --output=fstype / | tail -n 1 I0412 16:39:42.263219 326971 main.go:121] libmachine: SSH cmd err, output: : overlay I0412 16:39:42.263322 326971 ubuntu.go:71] root file system type: overlay I0412 16:39:42.263911 326971 provision.go:296] Updating docker unit: /lib/systemd/system/docker.service ... I0412 16:39:42.264148 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:42.328681 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:42.328843 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:42.328928 326971 main.go:121] libmachine: About to run SSH command: sudo mkdir -p /lib/systemd/system && printf %s "[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket StartLimitBurst=3 StartLimitIntervalSec=60 [Service] Type=notify Restart=on-failure # This file is a systemd drop-in unit that inherits from the base dockerd configuration. # The base configuration already specifies an 'ExecStart=...' command. The first directive # here is to clear out that command inherited from the base configuration. Without this, # the command from the base configuration and the command specified here are treated as # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd # will catch this invalid input and refuse to start the service with an error like: # Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. # NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other # container runtimes. If left unlimited, it may result in OOM issues with MySQL. ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP \$MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target " | sudo tee /lib/systemd/system/docker.service.new I0412 16:39:42.485563 326971 main.go:121] libmachine: SSH cmd err, output: : [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket StartLimitBurst=3 StartLimitIntervalSec=60 [Service] Type=notify Restart=on-failure # This file is a systemd drop-in unit that inherits from the base dockerd configuration. # The base configuration already specifies an 'ExecStart=...' command. The first directive # here is to clear out that command inherited from the base configuration. Without this, # the command from the base configuration and the command specified here are treated as # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd # will catch this invalid input and refuse to start the service with an error like: # Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. # NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other # container runtimes. If left unlimited, it may result in OOM issues with MySQL. ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target I0412 16:39:42.485873 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:42.548523 326971 main.go:121] libmachine: Using SSH client type: native I0412 16:39:42.548689 326971 main.go:121] libmachine: &{{{ 0 [] [] []} docker [0x7fb7a0] 0x7fb760 [] 0s} 127.0.0.1 49237 } I0412 16:39:42.548712 326971 main.go:121] libmachine: About to run SSH command: sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; } I0412 16:39:42.696902 326971 main.go:121] libmachine: SSH cmd err, output: : I0412 16:39:42.696942 326971 machine.go:91] provisioned docker machine in 1.551196189s I0412 16:39:42.696959 326971 start.go:267] post-start starting for "minikube" (driver="docker") I0412 16:39:42.696971 326971 start.go:277] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs] I0412 16:39:42.697067 326971 ssh_runner.go:149] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs I0412 16:39:42.697126 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:42.742721 326971 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49237 SSHKeyPath:/home/jholland/.minikube/machines/minikube/id_rsa Username:docker} I0412 16:39:42.846923 326971 ssh_runner.go:149] Run: cat /etc/os-release I0412 16:39:42.856369 326971 main.go:121] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found I0412 16:39:42.856460 326971 main.go:121] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found I0412 16:39:42.856500 326971 main.go:121] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found I0412 16:39:42.856523 326971 info.go:137] Remote host: Ubuntu 20.04.1 LTS I0412 16:39:42.856552 326971 filesync.go:118] Scanning /home/jholland/.minikube/addons for local assets ... I0412 16:39:42.856696 326971 filesync.go:118] Scanning /home/jholland/.minikube/files for local assets ... I0412 16:39:42.856770 326971 start.go:270] post-start completed in 159.795094ms I0412 16:39:42.856952 326971 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'" I0412 16:39:42.857109 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:42.918349 326971 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49237 SSHKeyPath:/home/jholland/.minikube/machines/minikube/id_rsa Username:docker} I0412 16:39:43.006056 326971 fix.go:56] fixHost completed within 49.640728172s I0412 16:39:43.054706 326971 start.go:80] releasing machines lock for "minikube", held for 49.689428497s I0412 16:39:43.055141 326971 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube I0412 16:39:43.123346 326971 ssh_runner.go:149] Run: curl -sS -m 2 https://k8s.gcr.io/ I0412 16:39:43.123447 326971 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0412 16:39:43.165371 326971 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49237 SSHKeyPath:/home/jholland/.minikube/machines/minikube/id_rsa Username:docker} I0412 16:39:43.186200 326971 out.go:119] W0412 16:39:43.186614 326971 out.go:181] X Exiting due to GUEST_MOUNT_CONFLICT: Sorry, docker does not allow mounts to be changed after container creation (previous mount: '', new mount: '/home/jholland:/src)' X Exiting due to GUEST_MOUNT_CONFLICT: Sorry, docker does not allow mounts to be changed after container creation (previous mount: '', new mount: '/home/jholland:/src)' I0412 16:39:43.243922 326971 out.go:119] ```
jmholla commented 3 years ago

This does work if you call minikube delete. It is not apparent why when reconstructing the container without this that it does not want to mount the paths at the right time.

spowelljr commented 3 years ago

@jmholla If you want to delete your existing minikube instance minikube delete is the recommended way.

Does using minikube delete resolve your issue?

jmholla commented 3 years ago

minikube delete did work. What about the way the container is being run causes mounting without minikube delete to happen so late in the process?

ilya-zuyev commented 3 years ago

@jmholla Docker driver does not support changing mount options. Removing underlying container using docker rm doesn't help here.

sharifelgamal commented 3 years ago

The issue here is that minikube still thinks it has an active container since you pulled the rug out from underneath minikube by deleting it directly via docker. The error message you're seeing would be the same if you tried to change mount options on a second minikube start without having stopped or deleted anything. minikube stop then docker rm minikube is never a recommended flow.

sharifelgamal commented 3 years ago

I'm going to go ahead and close this issue for now. If you have any questions, please feel free to reopen this issue.