WinRM usage not fixed in latest release on Pop_OS (Ubuntu derivative)

[adamc-formlogic]

Hello,

I am seeing the same behavior as https://github.com/hashicorp/vagrant/issues/13076 still, even though I am running the latest version of Vagrant. I believe I have enabled the legacy OpenSSL provider as described in this issue:

$ openssl list -providers
Providers:
  default
    name: OpenSSL Default Provider
    version: 3.0.2
    status: active
  legacy
    name: OpenSSL Legacy Provider
    version: 3.0.2
    status: active

I am able to do other things that require the legacy provider:

$ python3 -c 'import hashlib; m=hashlib.new("md4");m.update(b"foo");print(m.hexdigest())'
0ac6700c491d70fb8650940b1ca1e4b2

But running WinRM things like vagrant rdp still fails with the Digest initialization failed: initialization error message.

Debug output

Debug log gist: https://gist.github.com/adamc-formlogic/2c77f6b5bc0b1eeeb6a354f5f0955937

Expected behavior

vagrant rdp should begin an RDP session to the running Windows VM.

Actual behavior

vagrant rdp failed with the following error:

An error occurred executing a remote WinRM command.

Shell: Cmd
Command: hostname
Message: Digest initialization failed: initialization error

Reproduction information

Vagrant version

$ vagrant -v
Vagrant 2.3.7

Host operating system

$ cat /etc/os-release
NAME="Pop!_OS"
VERSION="22.04 LTS"
ID=pop
ID_LIKE="ubuntu debian"
PRETTY_NAME="Pop!_OS 22.04 LTS"
VERSION_ID="22.04"
HOME_URL="https://pop.system76.com"
SUPPORT_URL="https://support.system76.com"
BUG_REPORT_URL="https://github.com/pop-os/pop/issues"
PRIVACY_POLICY_URL="https://system76.com/privacy"
VERSION_CODENAME=jammy
UBUNTU_CODENAME=jammy
LOGO=distributor-logo-pop-os

Guest operating system

Windows Server 2016

Steps to reproduce

1. Install Vagrant 2.3.7 via the install instructions:

   $ dpkg -s vagrant
   Package: vagrant
   Status: install ok installed
   Priority: optional
   Section: default
   Installed-Size: 374132
   Maintainer: HashiCorp <support@hashicorp.com>
   Architecture: amd64
   Version: 2.3.7-1
   Description: Vagrant is a tool for building and distributing development environments.
   License: MIT
   Vendor: HashiCorp
   Homepage: https://www.vagrantup.com

2. Create an environment with the Vagrantfile below
3. Launch it with vagrant up
4. Attempt a WinRM command like vagrant rdp

Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "jborean93/WindowsServer2016"
end

[adamc-formlogic]

I also obtained and reproduced this issue on a machine running Ubuntu 23.04, same basic configuration as above.

[adamc-formlogic]

I have a working theory about what might be going on. The patch has the following block:

if have_header("openssl/opensslv.h")
  append_ldflags(["-lssl", "-lcrypto"])
  create_makefile("vagrant_ssl")
else
  # If the header file isn't found, just create a dummy
  # Makefile and stub the library to make it a noop
  File.open("Makefile", "wb") do |f|
    f.write(dummy_makefile(__dir__).join("\n"))
  end
  FileUtils.touch("vagrant_ssl.so")
end

This checks for the presence of the OpenSSL development libraries on the machine, and if they exist, builds the helper; but if they do not exist, it creates a zero-byte file where the helper would go.

The .deb and .rpm packages have this zero-byte file:

$ rpm2cpio ../vagrant-2.3.7-1.x86_64.rpm | cpio -tv | grep vagrant_ssl.so
-rw-r--r--   1 root     root            0 Jun 15 15:14 ./opt/vagrant/embedded/gems/gems/vagrant-2.3.7/ext/vagrant_ssl/vagrant_ssl.so

$ dpkg -c vagrant_2.3.7-1_amd64.deb  | grep vagrant_ssl.so
-rw-r--r-- 0/0               0 2023-06-15 15:14 ./opt/vagrant/embedded/gems/gems/vagrant-2.3.7/ext/vagrant_ssl/vagrant_ssl.so

I think the machine that's building the release packages does not itself have the development OpenSSL library installed, so it's taking the "touch an empty file" code path, and that empty file is being included in the packages.

I did try building Vagrant from source enough to get the actual version of this file:

$ cd ext/vagrant_ssl/; ruby extconf.rb; make; ls -l
checking for openssl/opensslv.h... yes
checking for whether -lssl is accepted as LDFLAGS... yes
checking for whether -lcrypto is accepted as LDFLAGS... yes
creating Makefile
compiling vagrant_ssl.c
linking shared-object vagrant_ssl.so
total 56
-rw-r--r-- 1 root   root    8639 Jun 20 23:12 Makefile
-rw-rw-r-- 1 ubuntu ubuntu   692 Jun 20 21:50 extconf.rb
-rw-r--r-- 1 root   root    3000 Jun 20 23:12 mkmf.log
-rw-rw-r-- 1 ubuntu ubuntu   710 Jun 20 21:50 vagrant_ssl.c
-rw-rw-r-- 1 ubuntu ubuntu   250 Jun 20 21:50 vagrant_ssl.h
-rw-r--r-- 1 root   root    7520 Jun 20 23:12 vagrant_ssl.o
-rwxr-xr-x 1 root   root   18296 Jun 20 23:12 vagrant_ssl.so

but I was unable to figure out where to put it in the /opt/vagrant file structure such that it actually got used. (Overwriting the zero-byte file did not do the trick.)

It does seem to get read:

$ strace -f vagrant rdp 2>&1 | grep vagrant_ssl | grep -v ENOENT
[pid 194589] stat("/opt/vagrant/embedded/gems/gems/vagrant-2.3.7/lib/vagrant/vagrant_ssl.so", {st_mode=S_IFREG|0755, st_size=18296, ...}) = 0
[pid 194589] openat(AT_FDCWD, "/opt/vagrant/embedded/gems/gems/vagrant-2.3.7/lib/vagrant/vagrant_ssl.so", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 5
[pid 194589] readlink("/opt/vagrant/embedded/gems/gems/vagrant-2.3.7/lib/vagrant/vagrant_ssl.so", 0x7ffdc6933f60, 1023) = -1 EINVAL (Invalid argument)
[pid 194589] stat("/opt/vagrant/embedded/gems/gems/vagrant-2.3.7/lib/vagrant/vagrant_ssl.so", {st_mode=S_IFREG|0755, st_size=18296, ...}) = 0
[pid 194589] openat(AT_FDCWD, "/opt/vagrant/embedded/gems/gems/vagrant-2.3.7/lib/vagrant/vagrant_ssl.so", O_RDONLY|O_CLOEXEC) = 5
$ ls -l /opt/vagrant/embedded/gems/gems/vagrant-2.3.7/lib/vagrant/vagrant_ssl.so
-rwxr-xr-x 1 root root 18296 Jun 20 15:38 /opt/vagrant/embedded/gems/gems/vagrant-2.3.7/lib/vagrant/vagrant_ssl.so

but just having it there isn't doing the trick, and I don't understand Ruby load paths well enough to troubleshoot any further.

[chrisroberts]

@adamc-formlogic Thanks for reporting this. Some modifications to the extension and the package build have resolved this.

A dev build with these changes included is available here. If you have any issues with it, please let me know.

Cheers!

[hedocode]

Hi ! @chrisroberts

The docs (https://developer.hashicorp.com/vagrant/docs/installation/source) doesn't mention how to compile it for windows, any tips about how to use it using the vagrant command please ? :)

[chrisroberts]

@hedocode Are you running Vagrant from source?

[salderma]

@chrisroberts Does this fix make it possible to undo the OpenSSL legacy providers work-around?

I am testing the updated package now. It seems to work on Ubuntu 20.04 LTS, on which I also encountered the problem despite the different OpenSSL version. I've moved on to 22.04 (with out removing openSSL legacy providers), and it also appears to get rid of the error.

FWIW, when I opened #13216, I had searched github but this ticket did not seem relevant based on the summary I saw in the search results.

[chrisroberts]

@salderma Yes, this removes the need for attempting to have them loaded via the configuration file. Instead a small extension is used to load them when Vagrant starts up.

[chrisroberts]

@hedocode If you are running from source, you can build the extension by running bundle exec rake compile. If you are not running from source, this development build contains installers for all platforms which include the modifications.

[hedocode]

@hedocode If you are not running from source, this development build contains installers for all platforms which include the modifications.

Thanks a lot ! I'm testing this right now

[adamc-formlogic]

I tested this and it worked great. Thanks so much for your help and for the quick fix!

[nrukavkov]

When version 2.3.8 will be released?

[ywei2017]

I can also confirm the dev build fixed the issue. I was using 2.3.7 and was battling with the problem for 2 weeks.

[hegyre]

Got the issue, and also confirm this works well with the 2.3.8 for Windows (this development build)

[yusukemasuda]

When version 2.3.8 will be released?

I believe it is almost about time to release. I cannot wait for.