OpenSSL not working with cross-installed Perl version

adamspofford-dfinity commented 1 year ago

When I attempt to build dfinity/sdk for Windows using x86_64-pc-windows-msvc-cross:local, I get

running "perl" "./Configure" "--prefix=/target/debug/build/openssl-sys-e1e37cc94a2d53ef/out/openssl-build/install" "--openssldir=/usr/local/ssl" "no-dso" "no-shared" "no-ssl3" "no-unit-test" "no-comp" "no-zlib" "no-zlib-dynamic" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "linux-x86_64" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64"
  Configuring OpenSSL version 1.1.1q (0x1010111fL) for linux-x86_64
  Using os-specific seed configuration

  --- stderr

  ******************************************************************************
  This perl implementation doesn't produce Unix like paths (with forward slash
  directory separators).  Please use an implementation that matches your
  building platform.

  This Perl version: 5.32.1 for MSWin32-x64-multi-thread
  ******************************************************************************
  thread 'main' panicked at '

  Error configuring OpenSSL build:
      Command: "perl" "./Configure" "--prefix=/target/debug/build/openssl-sys-e1e37cc94a2d53ef/out/openssl-build/install" "--openssldir=/usr/local/ssl" "no-dso" "no-shared" "no-ssl3" "no-unit-test" "no-comp" "no-zlib" "no-zlib-dynamic" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "linux-x86_64" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64"
      Exit status: exit status: 255

This appears to be the correct version of Perl, but OpenSSL is still not detecting it.

(Is this even a problem with cross-toolchains? Should I post this on the OpenSSL repo instead?)

Alexhuszagh commented 1 year ago

This is a known limitation, unfortunately (unless if we can patch things). We're not running natively on a Windows system, and so we need to use Strawberry Perl to handle Windows paths correctly when running our scripts through Wine, but this has other limitations (like Unix paths, unfortunately). If you know any solution, I would love to hear it.

You can try removing these 3 lines and see if it works: https://github.com/cross-rs/cross-toolchains/blob/971e8bb2cd570fe880382c97be0118969729d893/docker/Dockerfile.x86_64-pc-windows-msvc-cross#L20-L22

We might want to have flexible aliases for perl that depend on the path: so you can choose which perl (Unix or Windows paths) you want to use for a specific application. If you can use a pre-built OpenSSL version from MXE rather than the vendored one, that would be great, but I understand that not every crate exposes the vendored feature of their dependencies and this may well be impossible without forking every dependency yourself.

Feedback is very much so appreciated.

adamspofford-dfinity commented 1 year ago

If I remove the perl lines, unfortunately it just gives me the opposite error about Windows paths. And then forcing a build either passes cl the gcc flags, or tries to find nmake.

I kind of get the sense that OpenSSL wasn't meant to be cross-compiled to MSVC.

Emilgardis commented 1 year ago

is it possible to conditionally switch perl between windows and unix paths, i.e via a env var? either that or

We might want to have flexible aliases for perl that depend on the path

sounds good

Alexhuszagh commented 1 year ago

This would be probably need a custom entrypoint then to set the path. Something like:

#!/usr/bin/env bash

if [[ "${CROSS_WINDOWS_PERL}" == 1 ]]; then
    export PATH=/path/to/strawberry:"${PATH}"
fi

exec main "${@}"

It still wouldn't fix the issue above (if anyone knows a Perl that handles both Unix and Windows paths, I would love to know), but it would a nice enhancement.

cavivie commented 10 months ago

To compile openssl-src for msvc targets, you should follow things below (At least it works for me):

Rewrite cc-rs windows find tool code, use or patch for your project, issue:

/// Attempts to find a tool within an MSVC installation using the Windows
/// registry as a point to search from.
///
/// The `target` argument is the target that the tool should work for (e.g.
/// compile or link for) and the `tool` argument is the tool to find (e.g.
/// `cl.exe` or `link.exe`).
///
/// This function will return `None` if the tool could not be found, or it will
/// return `Some(cmd)` which represents a command that's ready to execute the
/// tool with the appropriate environment variables set.
///
/// Note that this function always returns `None` for non-MSVC targets.
pub fn find(target: &str, tool: &str) -> Option<Command> {
// old version:
// find_tool(target, tool).map(|c| c.to_command())

// patch version:
use std::env;

// This logic is all tailored for MSVC, if we're not that then bail out
// early.
if !target.contains("msvc") {
    return None;
}

// Early return if it the current target is not a windows target.
if let Ok(target) = env::var("CARGO_CFG_TARGET_FAMILY") {
    if target != "windows" {
        return None;
    }
}

// Early return if the environment doesn't contain a VC install.
env::var_os("VCINSTALLDIR")?;
env::var_os("VSINSTALLDIR")?;

// Fallback to simply using the current environment.
env::var_os("PATH")
    .and_then(|path| {
        env::split_paths(&path)
            .map(|p| p.join(tool))
            .find(|p| p.exists())
    })
    .map(|path| std::process::Command::new(path))
}

Set cross env CROSS_WINDOWS_PERL=1 to your cross msvc image when cross build.

Some openssl related issues:

cavivie commented 5 months ago

@Emilgardis I think we can close it because I have already fixed the cc-rs crate.

Emilgardis commented 5 months ago

Ah yes! Thank you for resolving it!

cross-rs / cross-toolchains

OpenSSL not working with cross-installed Perl version #14