Run vagrant up with virtual box gives "VirtualBox is complaining that the installation is incomplete"

[gvasilakiss]

I am trying to spin up a vagrant VM to use as the working environment.

When I try to spin up the machine, it keeps throwing me the error:

The provider 'virtualbox' that was requested to back the machine
'default' is reporting that it isn't usable on this system. The
reason is shown below:

VirtualBox is complaining that the installation is incomplete. Please
run `VBoxManage --version` to see the error message which should contain
instructions on how to fix this error.

Then it follows with this error

WARNING: The character device /dev/vboxdrv does not exist.
         Please install the virtualbox-dkms package and the appropriate
         headers, most likely linux-headers-azure.

         You will not be able to start VMs until this problem is fixed.
6.1.16_Ubuntur140961

I have tried also running sudo apt-get install linux-headers-azure but still comes up with the same error.

Any suggestions?

[Chuxel]

Did you raise this in the right location? This project is unrelated to Vagrant. If it is in the right spot, can you explain more about what your setup looks like and the steps you took to get it?

[gvasilakiss]

I believe this is on the right location; as the problem is more related to the way codespace interacts with vagrant and the permission we have.

I have a repository hosting a provisioning file that runs some commands to install and run Magento, elasticsearch, Redis and some other services to develop Magento locally.

In order to run that I am using vagrant and VirtualBox to create the local development environment.

My vagrant file looks like this:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
  # The most common configuration options are documented and commented below.
  # For a complete reference, please see the online documentation at
  # https://docs.vagrantup.com.

  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://vagrantcloud.com/search.
  config.vm.box = "debian/buster64"
  config.disksize.size = '15GB'
  config.vm.hostname = "barbour.uat"
  config.vm.network "private_network", ip: "192.168.56.7"
  #config.vm.network "private_network", type: "dhcp"
  config.vm.synced_folder ".", "/vagrant", type: "virtualbox", disabled: true
  #config.vm.synced_folder "./html", "/var/www/html/", type:"nfs", mount_options: %w{rw,async,fsc,nolock,vers=3,udp,rsize=32768,wsize=32768,hard,noatime,actimeo=2}

  #config.vm.network "forwarded_port", guest: 80, host: 8080

  # Disable automatic box update checking. If you disable this, then
  # boxes will only be checked for updates when the user runs
  # `vagrant box outdated`. This is not recommended.
  # config.vm.box_check_update = false

  # Create a forwarded port mapping which allows access to a specific port
  # within the machine from a port on the host machine. In the example below,
  # accessing "localhost:8080" will access port 80 on the guest machine.
  # NOTE: This will enable public access to the opened port
  # config.vm.network "forwarded_port", guest: 80, host: 8080

  # Create a forwarded port mapping which allows access to a specific port
  # within the machine from a port on the host machine and only allow access
  # via 127.0.0.1 to disable public access
  # config.vm.network "forwarded_port", guest: 80, host: 8080, host_ip: "127.0.0.1"

  # Create a private network, which allows host-only access to the machine
  # using a specific IP.
  # config.vm.network "private_network", ip: "192.168.33.10"

  # Create a public network, which generally matched to bridged network.
  # Bridged networks make the machine appear as another physical device on
  # your network.
  # config.vm.network "public_network"

  # Share an additional folder to the guest VM. The first argument is
  # the path on the host to the actual folder. The second argument is
  # the path on the guest to mount the folder. And the optional third
  # argument is a set of non-required options.
  # config.vm.synced_folder "../data", "/vagrant_data"

  # Provider-specific configuration so you can fine-tune various
  # backing providers for Vagrant. These expose provider-specific options.
  # Example for VirtualBox:
  #
  # config.vm.provider "virtualbox" do |vb|
  #   # Display the VirtualBox GUI when booting the machine
  #   vb.gui = true
  #
  #   # Customize the amount of memory on the VM:
  #   vb.memory = "1024"
  # end

    config.vm.provider "virtualbox" do |vb|
    vb.name = "barbour-dev-git.uat"
    vb.gui = false
    vb.memory = 7168 
    vb.cpus = 4
    vb.customize ["modifyvm", :id, "--ioapic", "on"]
    vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
    vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
    vb.customize ["modifyvm", :id, "--cpuexecutioncap", "100"]
    vb.customize ["modifyvm", :id, "--vram", "7"]

    #vb.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"]
    end

    #config.vm.provision "shell" do |s|
    #s.path = "provision.sh"
    #end

  #
  # View the documentation for the provider you are using for more
  # information on available options.

  # Enable provisioning with a shell script. Additional provisioners such as
  # Ansible, Chef, Docker, Puppet and Salt are also available. Please see the
  # documentation for more information about their specific syntax and use.

end

Now as I have included the error that I am getting when running vagrant up to initiate the VM is

The provider 'virtualbox' that was requested to back the machine
'default' is reporting that it isn't usable on this system. The
reason is shown below:

VirtualBox is complaining that the installation is incomplete. Please
run `VBoxManage --version` to see the error message which should contain
instructions on how to fix this error.

In the process of trying some commands to fix the issue I got the following error: The vboxdrv kernel module is not loaded. current kernel (5.4.0-1051-azure). That make me think if codespace is using a custom image that's why I cant run the vagrant;

Some people also mentioned to disable Secure Boot, which I found that couldn't happen as it was displaying the following error:EFI variables are not supported on this system

Running also VBoxManage --version gives me the following error:

WARNING: The vboxdrv kernel module is not loaded. Either there is no module
         available for the current kernel (5.4.0-1051-azure) or it failed to
         load. Please recompile the kernel module and install it by

           sudo /sbin/vboxconfig

         You will not be able to start VMs until this problem is fixed.
6.1.24r145767

[Chuxel]

First, note you can run an image setup for Codespaces locally using the VS Code Remote - Containers extension. So the good news is you probably don't actually need a vagrant box to accomplish what you are looking to do. That would be easier to do than setting up Vagrant here.

That said, I don't believe VirtualBox is setup on the host VM for Codespaces, correct @jkeech? Are privs setup so you could run sudo apt-get install virtualbox-dkms linux-headers-azure as a part of the initalizeCommand property in devcontainer.json?

[gvasilakiss]

I will try with docker, I was hoping to get it up with vagrant as I already use that for local development.

When I run sudo apt-get install virtualbox-dkms linux-headers-azure I get at the end the following output:

DKMS: install completed.
Setting up virtualbox (6.1.16-dfsg-6~ubuntu1.20.04.2) ...
Created symlink /etc/systemd/system/multi-user.target.wants/vboxweb.service → /lib/systemd/system/vboxweb.service.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of restart.

And then a new error comes up:

WARNING: The character device /dev/vboxdrv does not exist.
         Please install the virtualbox-dkms package and the appropriate
         headers, most likely linux-headers-azure.

         You will not be able to start VMs until this problem is fixed.
6.1.16_Ubuntur140961

[jkeech]

That said, I don't believe VirtualBox is setup on the host VM for Codespaces, correct @jkeech? Are privs setup so you could run sudo apt-get install virtualbox-dkms linux-headers-azure as a part of the initalizeCommand property in devcontainer.json?

Correct. This is definitely unsupported and not likely to work well in Codespaces. Anything outside of the docker containers that runs on the host VM would not be preserved across stop/start for the codespace. Codespaces are designed to be containerized environments with everything running in a docker container of some sort on the host (either in the main codespace container or a sibling container via Docker Compose).

I suspect you could get it to work within the context of a single session via initializeCommand and setting necessary bind mounts into the container, but persistence will be an issue and it's definitely pushing the boundaries of what codespaces is designed to do.

Edit: Actually, initializeCommand would only run the first time, not on subsequent starts.

[gvasilakiss]

So you suggest I run Magento with docker to have persistence within the dev environment?

[jkeech]

So you suggest I run Magento with docker to have persistence within the dev environment?

Yep! I don't know too much about Magento development, but that might be something you'd want to run in the main codespace container if that's your main app. And then redis, elasticsearch, etc could be sidecar containers with Docker Compose.

[gvasilakiss]

So you suggest I run Magento with docker to have persistence within the dev environment?

Yep! I don't know too much about Magento development, but that might be something you'd want to run in the main codespace container if that's your main app. And then redis, elasticsearch, etc could be sidecar containers with Docker Compose.

Great, I will give it a go and see how it goes. Thanks for your help @jkeech @Chuxel

[Chuxel]

Great, I will give it a go and see how it goes. Thanks for your help @jkeech @Chuxel

Np! You can think of the Codespace or locally running "dev container" as exactly equivalent to a Vagrant Box in your workflow. You can create, delete, rebuild, etc them much like you would in Vagrant. It's just in a much lighter weight environment and you can use Docker Compose to setup side-cars as @jkeech says. The "Add Development Configuration Files" command in codespaces includes a bunch of jump starts. If you click on "Show All Definitions" there's things like "Python and Postgres" that show a compose configuration. All of those come out of this repo.

[gvasilakiss]

I had an idea of docker, but I haven't had the chance to work with it. It is defo faster and lighter than vagrant as well. Would I have to use docker in docker to do that?

[Chuxel]

The way this works is you end up inside a container (which is a bit of a unique capability). It's as if you are actually running VS Code inside your Vagrant Box and accessing it there.

So, you can set up your environment exactly as you want it and just work from in there and recreate it at any point. devcontainer.json is basically a manifest that explains what should go inside. There's a "Rebuild" command that lets you update your config and recreate the environment while keeping your source code edits.

The only reason you would need to use docker-in-docker is if you then want to run docker from inside this development container. There's a script you can use for your custom dev container setup here: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker-in-docker.md It's just installed by default in the base Codespaces image (which is containers/codespaces-linux in this repository).

[gvasilakiss]

The way this works is you end up inside a container (which is a bit of a unique capability). It's as if you are actually running VS Code inside your Vagrant Box and accessing it there.

So, you can set up your environment exactly as you want it and just work from in there and recreate it at any point. devcontainer.json is basically a manifest that explains what should go inside. There's a "Rebuild" command that lets you update your config and recreate the environment while keeping your source code edits.

The only reason you would need to use docker-in-docker is if you then want to run docker from inside this development container. There's a script you can use for your custom dev container setup here: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker-in-docker.md It's just installed by default in the base Codespaces image (which is containers/codespaces-linux in this repository).

Got you, I don't need to use docker in docker. I just need to run docker run and have my machine up and running?

I haven't understood in full how I will implement it with devcontainer.json. Is it like the Dockerfile or something different?