ppggff / vagrant-qemu

Use Vagrant to manage machines using QEMU. Test with Apple Silicon / M1 and CentOS aarch64 image
MIT License
408 stars 32 forks source link

How to create box from vagrant cloud #24

Open Fashaun opened 1 year ago

Fashaun commented 1 year ago

Hello,

How could I create box for vagrant-qemu , for example debian OS?

ppggff commented 1 year ago

Sorry, the vagrant package command is not support for now. (#4) I will give a step by step manual as soon as possible.

Fashaun commented 1 year ago

got it , thanks for the quick response

ppggff commented 1 year ago

I put some basic step here. Please try it.

Fashaun commented 1 year ago

Sorry , but I mean how to vagrant up from an existing box on vagrant cloud Or , you mean I have to create my own local box first?

➜ test_debian vagrant init generic/debian10 A Vagrantfile has been placed in this directory. You are now ready to vagrant up your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on vagrantup.com for more information on using Vagrant. ➜ test_debian vagrant up --provider=qemu Bringing machine 'default' up with 'qemu' provider... ==> default: Checking if box 'generic/debian10' version '4.1.14' is up to date... ==> default: Importing a QEMU instance default: Creating and registering the VM... default: Successfully imported VM ==> default: Warning! The QEMU provider doesn't support any of the Vagrant ==> default: high-level network configurations (config.vm.network). They ==> default: will be silently ignored. ==> default: Starting the instance... ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:50022 default: SSH username: vagrant default: SSH auth method: private key

ppggff commented 1 year ago

Your way is the right way. You can use an existing box. There is more example in the Readme file and wiki. However, not all box works out of box..., the "generic/debian10" may need some fine tuning...

Fashaun commented 1 year ago

May I know how to fix it? then I could raise a merge request to you

ppggff commented 1 year ago

Please try the same config in https://github.com/ppggff/vagrant-qemu/wiki/buster64-on-x86_64 It works on my macbook.

The basic steps to debug this problem:

  1. ps -ef|grep qemu, get the detail QEMU command and arguments
  2. run these command manually (remove the -display none -vga none -daemonize to get a display window)
  3. try to figure out the right argument, or may need some work inside the vm (the ubuntu problem) (you may need to google it for help)

For this Debian issue, I didn't dig too much. It works after adding a VGA device. Maybe Debian needs a VGA device, or maybe there is another root cause. Please let me know if you find it. : )

Fashaun commented 1 year ago

Sorry but I'm the user use mac M1

ppggff commented 1 year ago

Have you tried it?

Since your target box is a x86_64 box, it should work. (without qe.qemu_dir config)

nileshtrivedi commented 1 year ago

Is there a way to have vagrant-qemu emulate the libvirt provider for pre-existing boxes that require libvirt?

I'm trying to develop Mastodon on Apple M1 machine. Mastodon's Vagrantfile requires ubuntu/bionic64 image which uses the libvirt provider. But VirtualBox does not work on Apple M1. Which leads to this error:

> vagrant up --provider qemu         
Bringing machine 'default' up with 'qemu' provider...
==> default: Box 'ubuntu/bionic64' could not be found. Attempting to find and install...
    default: Box Provider: libvirt
    default: Box Version: >= 0
==> default: Loading metadata for box 'ubuntu/bionic64'
    default: URL: https://vagrantcloud.com/ubuntu/bionic64
The box you're attempting to add doesn't support the provider
you requested. Please find an alternate box or use an alternate
provider. Double-check your requested provider to verify you didn't
simply misspell it.

If you're adding a box from HashiCorp's Vagrant Cloud, make sure the box is
released.

Name: ubuntu/bionic64
Address: https://vagrantcloud.com/ubuntu/bionic64
Requested provider: ["libvirt"]

This is even after I added the following section in the Vagrantfile as suggested by vagrant-qemu README:

  config.vm.provider "qemu" do |qe|
    qe.machine = "virt,accel=hvf,highmem=off"
    qe.cpu = "cortex-a72"
  end

I have filed an issue with Mastodon but on Apple M1, if vagrant-qemu emulates libvirt, then it would solve the problem for all projects with existing Vagrantfile that requires boxes with libvirt only.

nileshtrivedi commented 1 year ago

I discovered that VirtualBox now has a Developer Preview build for Apple M1/M2 hosts. However, this doesn't actually work for me as the machine created by vagrant with ubuntu/bionic64 box gets aborted. Even if that were to be fixed, the performance will likely be terrible. These comments are explanatory:

Treat the M1 build as a very very early technical preview please, the Beta label is completely misleading and will be changed (this comes from the automated build system and aligns with the rest of the builds). There will be no official support for M1 with 7.0, not even for ARM virtualization, let alone x86 emulation on ARM.

And,

In a way the ARM64 package "slipped out", and it's not expected to work reliably. The implementation isn't complete yet (which is what you saw), and in top of that the performance is known to be extremely low. It isn't anywhere near production ready, we know. This will not change for VirtualBox 7, and the "Technology Preview" marker will stay for the foreseeable future, indicating that it won't be supported at all.

At best you'll get some really old 32-bit Linux to run to some degree, such as DSL 4.4.10. No chance even with Ubuntu 16.04 i386.

So, it would really be valuable for vagrant-qemu to support libvirt boxes like ubuntu/bionic64 because so many developers now use Apple M1/M2 machines.

jay7x commented 1 year ago

JFYI, below is the packer HCL template to make an usable qcow2 image from the upstream Debian 11 arm64 genericcloud image.

This is not a box but still can be used with the following config in Vagrantfile:

  vagrant_root = File.dirname(__FILE__)
  config.vm.provider "qemu" do |qemu|
    qemu.qemu_dir = "/opt/homebrew/share/qemu"
    qemu.image_path = "#{vagrant_root}/deb11-arm64.qcow2"
  end

It should be easy enough to adopt this approach to Ubuntu and other distros.

qemu-deb11-arm64.pkr.hcl ```hcl packer { required_plugins { qemu = { version = ">= 1.0.7" source = "github.com/hashicorp/qemu" } } } variable "iso_url" { description = "Base ISO URL to build the image from" type = string default = "https://cloud.debian.org/images/cloud/bullseye/20221108-1193/debian-11-genericcloud-arm64-20221108-1193.qcow2" } variable "iso_checksum" { description = "Base ISO checksum" type = string default = "file:https://cloud.debian.org/images/cloud/bullseye/20221108-1193/SHA512SUMS" } variable "qemu_firmware" { description = "Path to look for qemu firmware in" type = string default = "/opt/homebrew/share/qemu/edk2-aarch64-code.fd" validation { condition = fileexists(var.qemu_firmware) error_message = "The qemu_firmware value must be a valid path." } } variable "vagrant_user" { description = "User name to create" type = string default = "vagrant" } variable "vagrant_passwd" { description = "vagrant_user password" type = string default = "vagrant" } locals { instance_data = { instance-id = "iid-local01" } cloud_config = { disable_root = true growpart = { mode = "auto" devices = ["/"] } // follow https://developer.hashicorp.com/vagrant/docs/v2.3.2/boxes/base#default-user-settings ssh_pwauth = true users = [ { name = var.vagrant_user plain_text_passwd = var.vagrant_passwd lock_passwd = false shell = "/bin/bash" sudo = "ALL=(ALL) NOPASSWD:ALL" ssh_authorized_keys = [data.http.vagrant_public_key.body] // vagrant insecure key }, ] } } # Insecure vagrant public key as recommended here https://developer.hashicorp.com/vagrant/docs/v2.3.2/boxes/base#vagrant-user data "http" "vagrant_public_key" { url = "https://raw.githubusercontent.com/hashicorp/vagrant/main/keys/vagrant.pub" } source "qemu" "deb11_arm64" { iso_url = var.iso_url iso_checksum = var.iso_checksum iso_target_extension = "qcow2" disk_image = true shutdown_command = "echo 'packer' | sudo -S shutdown -P now" ssh_username = var.vagrant_user ssh_password = var.vagrant_passwd firmware = var.qemu_firmware use_pflash = true skip_resize_disk = true format = "qcow2" disk_compression = true accelerator = "hvf" machine_type = "virt" qemu_binary = "qemu-system-aarch64" disable_vnc = true headless = true qemuargs = [ ["-cpu", "host"], ["-serial", "file:{{ .OutputDir }}/{{ .Name }}-serial.log"], ] cd_label = "cidata" cd_content = { "meta-data" = jsonencode(local.instance_data) "user-data" = "#cloud-config\n${jsonencode(local.cloud_config)}" } } build { sources = [ "source.qemu.deb11_arm64", ] } ```
ppggff commented 1 year ago

@nileshtrivedi I think the 'ubuntu/bionic64' only support virtualbox provider, see https://app.vagrantup.com/ubuntu/boxes/bionic64

There are some solutions:

  1. use the 'ubuntu/bionic64', download the box, convert the disk image to qcow2 (should be similar with Use VMware Fusion Box), and ran the qcow2 image with vagrant-qemu provier (see example at readme)
  2. find another ubuntu image that made for libvirt. (vagrant-qemu support libvirt box)

Notes:

  1. you'd better find a arm64 box for apple m1/m2 for better performance
  2. you can use a x86_64 box, try the sample config in the readme
  3. ubuntu box may not work out of box, try Fix ubuntu hang, or open a new issue
arturmartins commented 12 months ago

You might want to try https://app.vagrantup.com/perk/boxes/ubuntu-2204-arm64