Replace Gopkg with go.mod

[TheLonelyGhost]

Some packages were renamed (e.g., github.com/codegangsta/cli -> github.com/urfave/cli) but this still seems to compile okay on all platforms.

I'm still sorting through test failures and fixing those, but wanted to get eyes on this sooner rather than later.

[afbjorklund]

That's a lot of files... (almost 800) Would be interesting to compare go mod vendor with the previous, once you are done.

Was there a feature behind this, or mostly cleaning up for the future and newer go version ? Because machine is deprecated

[afbjorklund]

The codegangsta module was reported back in 2017, but I don't think it ever went anywhere...

I'm still sorting through test failures and fixing those,

There are tests ? ;-)

[TheLonelyGhost]

Mainly I wanted to contribute back upstream. I was looking at forking this to update for podman 2.x support on macos, perhaps with hyperkit eventually. Would also have to overhaul the boot2podman concept, but using prior art here would be preferrable to Vagrant.

As for tests, all I know is that there was a test target in the makefile, I saw a few *_test.go files, and assumed there were some tests at all that were used. No sign of CI here so I'm just taking a stab in the dark.

as for differences between vendored dependencies, I'll check and post the diff here if it's not too large. Otherwise I'll reevaluate whether it's worth it to pursue the go.mod route upstream here. 😄

[afbjorklund]

Mainly I wanted to contribute back upstream. I was looking at forking this to update for podman 2.x support on macos, perhaps with hyperkit eventually. Would also have to overhaul the boot2podman concept, but using prior art here would be preferrable to Vagrant.

The upstream situation is complicated. The original project is dead in the water, so there are lots of patches pending.

As described (somewhere), the podman code is basically the same as docker but with swarm and cloud removed. So it suffers the same fate, there has been no major changes since 2019. And it's all based on varlink, of course...

The Vagrant approach was an easy way out, and easy to "maintain". It does increase the VM "footprint" by quite a lot. However, there is no interest from Red Hat to drive this project. They already have FCOS and CodeReady Containers.

So that is why it was cancelled, last year: https://boot2podman.github.io/2020/11/03/boot2podman-project.html

Just as docker-machine continues to limp along to spin Ubuntu machines in the cloud (with problems after 19.03), I suppose podman-machine could go on extra life support and spin up Fedora machines (with issues after 1.9.3)

Most of the podman 2.x support has been added for minikube, that has forked the libmachine and the drivers too. So it is doable, just not sure that I want to do it anymore. Most likely, it will break in podman 3.x when least expected.

Instead I'm going to do a new project, that will handle all the container engines, like a "boot2engine" if you will.

I called it Floe, it's going back to the Core: https://boot2podman.github.io/2021/01/10/introducing-floe-linux.html

It's going quite well, should have something ready by the end of the month or so. It's going to need a manager, too... Right now I'm just using QEMU directly, but maybe it could be useful to have more hypervisors ? Haven't decided yet.

Too bad that there isn't anything cross-platform similar to VirtualBox, we have MultiPass but it's a bit too tied to Ubuntu. It would be much easier if there was just one API to talk to, instead of having to do multiple such plugins in the client.

Most likely I will just do something simple with go-qemu. And then trust them to do the hardware acceleration (KVM).

---

As for tests, all I know is that there was a test target in the makefile, I saw a few *_test.go files, and assumed there were some tests at all that were used. No sign of CI here so I'm just taking a stab in the dark.

You can look at the upstream/sidestream/whatever files at https://github.com/docker/machine but they're a bit dated...

as for differences between vendored dependencies, I'll check and post the diff here if it's not too large. Otherwise I'll reevaluate whether it's worth it to pursue the go.mod route upstream here. smile

Basically I just added enough make and vendor to build the binaries, made no attempt at Continuous Integration.

We could convert it to go modules and add some better build scripts. But I'm not sure about adding ssh support.

Even if I already wrote most of it here: https://github.com/kubernetes/minikube/blob/master/cmd/minikube/cmd/podman-env.go

If it is done (in machine), it would be nice to have something that would work for both docker and podman instead of duplicating it. There is a similar change, from the previous tcp/2376 transport over to using unix sockets over ssh, also on the docker host side.

Currently it is still optional to use, like: https://github.com/kubernetes/minikube/blob/master/cmd/minikube/cmd/docker-env.go

But it cannot be done with a hostile upstream (like docker), so that's why it is better to starting the project over (again).

There is also a lot of legacy, like aufs and swarm, that can be avoided. And also a chance to change the commands ?

As I tried to explain above, the plan was to make something for Floe. Will hack on it some more, over the FOSDEM weekend. And also in the vagrant-machine tool, where I was trying to iron out the common CLI (as an early rapid prototype for this).

[TheLonelyGhost]

Ah, I didn't dig into this project enough to know whether it was a fork of docker-machine or just inspired by it. My initial assessment was to take the same command wrappers from podman-machine, same driver support, and to gut the rest of it as needed since we don't need as much flexibility (and instability) as Vagrant seems to offer.

All that said, TIL that QEMU has support for linux, mac, and windows! That solves one big issue I had with layers of abstraction involved with Vagrant. The other one is how heavy Fedora is, which happens to be the only system I know of that to support podman and cgroup v2 (necessary for rootless).

Either way, it seems contributing here is pointless and it would indeed be easier to start fresh. Thanks for the heads up!

[afbjorklund]

There is also some talk within the podman community, about doing a "proper" mac version. You might want to join that.

Currently there is some discussion on whether using a VM is good enough, or if people mean being able to do syscalls (like WSL1). So it needs more input from the users, I'm guessing what Whatever-Docker-Is-Doing (i.e. VM) would be the usual requirement ?

Or you can go the other way, and start a new VM for each and every container. The way that "hyper.sh" / kata is doing.

---

You can also use the minikube VM already today, and use that as a podman environment. It doesn't do rootless, but anway.

minikube start --container-runtime=cri-o
brew install podman
minikube podman-env
podman version

I added rootless support*, and will probably do it again. But I don't really see the use case of "fakeroot", when using a VM ?

* https://github.com/boot2podman/boot2podman/blob/master/building_rootless.md

I can see it as a workaround (with poor performance) for multi-user systems, but not so much when having a dedicated system

Then you would rather want to use real overlayfs and real network (in kernel), instead of just fuse-overlayfs and slirp4netns ?

[afbjorklund]

@TheLonelyGhost :

Ah, I didn't dig into this project enough to know whether it was a fork of docker-machine or just inspired by it. My initial assessment was to take the same command wrappers from podman-machine, same driver support, and to gut the rest of it as needed since we don't need as much flexibility (and instability) as Vagrant seems to offer.

I haven't posted the actual Vagrant project yet, but basically everything is on this page:

https://boot2podman.github.io/2020/07/22/machine-replacement.html

But podman-machine does not use Vagrant, and boot2podman does not use Fedora. Those were just the projects that I referred to, as a replacement for the functionality ?

[afbjorklund]

Also, I was able to convert the old files to go modules without too much effort. And the results looks reasonable...

 Gopkg.lock                                                                     | 486 -------------------------------------------------------------------------------------------
 Gopkg.toml                                                                     |  39 --------
 go.mod                                                                         |  43 ++++++++
 go.sum                                                                         |  78 +++++++++++++++
 vendor/github.com/digitalocean/godo/.gitignore                                 |   0
 vendor/github.com/docker/docker/contrib/syntax/vim/LICENSE                     |  22 -----
 vendor/github.com/docker/docker/docs/static_files/contributors.png             | Bin 23100 -> 0 bytes
 vendor/github.com/docker/docker/hack/generate-authors.sh                       |  15 ---
 vendor/github.com/docker/docker/integration-cli/fixtures/https/ca.pem          |   1 -
 vendor/github.com/docker/docker/integration-cli/fixtures/https/client-cert.pem |   1 -
 vendor/github.com/docker/docker/integration-cli/fixtures/https/client-key.pem  |   1 -
 vendor/github.com/docker/docker/integration-cli/fixtures/https/server-cert.pem |   1 -
 vendor/github.com/docker/docker/integration-cli/fixtures/https/server-key.pem  |   1 -
 vendor/github.com/docker/docker/pkg/symlink/LICENSE.APACHE                     | 191 ------------------------------------
 vendor/github.com/docker/docker/pkg/symlink/LICENSE.BSD                        |  27 ------
 vendor/github.com/docker/docker/project/CONTRIBUTING.md                        |   1 -
 vendor/github.com/stretchr/testify/LICENCE.txt                                 |  22 -----
 vendor/github.com/vmware/govmomi/govc/test/license.bats                        |  55 -----------
 vendor/github.com/vmware/govmomi/scripts/contributors.sh                       |  24 -----
 vendor/github.com/vmware/govmomi/scripts/license.sh                            |  48 ---------
 vendor/golang.org/x/sys/unix/mkall.sh                                          |   0
 vendor/golang.org/x/sys/unix/mkerrors.sh                                       |   0
 vendor/golang.org/x/sys/unix/mksyscall.pl                                      |   0
 vendor/golang.org/x/sys/unix/mksyscall_solaris.pl                              |   0
 vendor/golang.org/x/sys/unix/mksysctl_openbsd.pl                               |   0
 vendor/golang.org/x/sys/unix/mksysnum_darwin.pl                                |   0
 vendor/golang.org/x/sys/unix/mksysnum_dragonfly.pl                             |   0
 vendor/golang.org/x/sys/unix/mksysnum_freebsd.pl                               |   0
 vendor/golang.org/x/sys/unix/mksysnum_netbsd.pl                                |   0
 vendor/golang.org/x/sys/unix/mksysnum_openbsd.pl                               |   0
 vendor/google.golang.org/appengine/internal/datastore/datastore_v3.proto       |   0
 vendor/google.golang.org/appengine/internal/regen.sh                           |   0
 vendor/modules.txt                                                             | 174 +++++++++++++++++++++++++++++++++
 33 files changed, 295 insertions(+), 935 deletions(-)

The trick was using the old version (go1.11) to do the conversion, and wait with the go mod tidy until next step.

go get golang.org/dl/go1.11.13
go1.11.13 download
export GO111MODULE=on
go1.11.13 mod init github.com/docker/machine
go1.11.13 mod vendor

As you can see there are no vendor code changes, it just removed the non-go files from the vendor directory.

After that, one can go ahead and update the go version and tidy the go modules and all those other 800 changes

---

Added as PR #30