search

svix / svix-webhooks

The enterprise-ready webhooks service 🦀
https://www.svix.com
MIT License
2.27k stars 156 forks source link

svix-bridge: Cannot open include file: 'fficonfig.h' Win11x64 #1128

Open davehorner opened 9 months ago

davehorner commented 9 months ago

Bug Report

cargo run --no-default-features --features gcp-pubsub,rabbitmq,redis,sqs

   Compiling swc_ecma_utils v0.120.19
   Compiling aws-smithy-client v0.55.3
   Compiling libffi-sys v2.3.0
   Compiling utf8parse v0.2.1
   Compiling trust-dns-resolver v0.22.0
   Compiling dynasm v1.2.3
   Compiling ghash v0.5.0
   Compiling rand_core v0.5.1
   Compiling prost-types v0.11.9
   Compiling prettyplease v0.1.25
   Compiling amq-protocol-types v7.1.2
   Compiling webpki-roots v0.23.1
   Compiling oid-registry v0.6.1
   Compiling unic-ucd-ident v0.9.0
   Compiling petgraph v0.6.3
   Compiling p12 v0.6.3
   Compiling digest v0.8.1
   Compiling tempfile v3.7.1
   Compiling async-io v1.13.0
   Compiling secp256k1-sys v0.8.1
   Compiling der-parser v8.2.0
   Compiling hyper-rustls v0.24.1
   Compiling fs3 v0.5.0
   Compiling swc_config v0.1.7
   Compiling hostname v0.3.1
   Compiling salsa20 v0.10.2
   Compiling deno_tls v0.104.0
   Compiling aws-smithy-json v0.55.3
   Compiling winreg v0.10.1
   Compiling ctr v0.9.2
   Compiling pbkdf2 v0.12.2
   Compiling rustls-connector v0.18.3
   Compiling aead v0.5.2
error: failed to run custom build command for `libffi-sys v2.3.0`

Caused by:
  process didn't exit successfully: `c:\w\rust\svix-webhooks\bridge\target\debug\build\libffi-sys-1eea76e9ec04104c\build-script-build` (exit code: 101)
  --- stdout
  OPT_LEVEL = Some("0")
  TARGET = Some("x86_64-pc-windows-msvc")
  HOST = Some("x86_64-pc-windows-msvc")
  cargo:rerun-if-env-changed=CC_x86_64-pc-windows-msvc
  CC_x86_64-pc-windows-msvc = None
  cargo:rerun-if-env-changed=CC_x86_64_pc_windows_msvc
  CC_x86_64_pc_windows_msvc = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  DEBUG = Some("true")
  cargo:rerun-if-env-changed=CFLAGS_x86_64-pc-windows-msvc
  CFLAGS_x86_64-pc-windows-msvc = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_pc_windows_msvc
  CFLAGS_x86_64_pc_windows_msvc = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None

  --- stderr
  Microsoft (R) C/C++ Optimizing Compiler Version 19.37.32825 for x64
  Copyright (C) Microsoft Corporation.  All rights reserved.

  win64_intel.S
  libffi/src/x86/win64_intel.S(2): fatal error C1083: Cannot open include file: 'fficonfig.h': No such file or directory
  thread 'main' panicked at C:\Users\dhorner\.cargo\registry\src\index.crates.io-6f17d22bba15001f\libffi-sys-2.3.0\build\common.rs:8:5:
  Pre-process ASM
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

Version

latest code

9d4b781e (HEAD -> main, tag: v1.14.0, origin/main, origin/HEAD) HEAD@{0}: pull: Fast-forward
30776fc9 HEAD@{1}: clone: from https://github.com/svix/svix-webhooks
svix-bridge v1.13.0 (C:\w\rust\svix-webhooks\bridge\svix-bridge)
├── svix-bridge-plugin-queue v0.1.0 (C:\w\rust\svix-webhooks\bridge\svix-bridge-plugin-queue)
│   ├── omniqueue v0.1.0 (https://github.com/svix/omniqueue-rs?rev=247904053bcf90cf693df4429092923bf97770eb#24790405)
│   ├── svix-bridge-types v0.1.0 (C:\w\rust\svix-webhooks\bridge\svix-bridge-types)
│   │   ├── svix v0.85.1
├── svix-bridge-types v0.1.0 (C:\w\rust\svix-webhooks\bridge\svix-bridge-types) (*)
├── svix-ksuid v0.7.0
svix-bridge-plugin-queue v0.1.0 (C:\w\rust\svix-webhooks\bridge\svix-bridge-plugin-queue) (*)
svix-bridge-types v0.1.0 (C:\w\rust\svix-webhooks\bridge\svix-bridge-types) (*)

Platform

Win11 x64

Description

Just following up to see if the windows bridge works. I am still unable to get a built yet. Now I am facing an issue with fficonfig.h. Let me know if you have any suggestions. Thank you.

tasn commented 9 months ago

@svix-onelson, any idea?

svix-onelson commented 9 months ago

@tasn Unfortunately, no idea.

@davehorner alas! I thought this was building well for you some weeks back, at least well enough for you to run into the var substitution problem 😓 (thought this should now be fixed).

I know we recently bumped a version of a dependency that was yanked. As a quick thing, I might try running cargo clean before re-running cargo build.

A weak web search indicates this sort of problem might come from env pollution; mixing MSVC and MinGW. No idea if this is applicable.

My guess would be most people are not using MinGW today, instead relying on WSL.

Separately, I wonder if we can get a CI job for the Windows build to help avoid these 🤔 Not sure what the options are for that with GitHub Actions.

davehorner commented 9 months ago

@svix-onelson this is a new Win11x64 machine.

You're right, we were past this compilation step and on to brighter days on my Win10x64. I'm surprised. I've turned this machine into a bowl of spaghetti so quickly.

I have found need for msys2 msys which also includes mingw32. I'm not running from a msys/mingw32 prompt so I would have hopes that this would not be the case.

error: failed to run custom build command for `libffi-sys v2.3.0`
note: To improve backtraces for build dependencies, set the CARGO_PROFILE_RELEASE_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.

Caused by:
  process didn't exit successfully: `c:\w\rust\svix-webhooks\bridge\target\release\build\libffi-sys-285f7bdc8a9bed76\build-script-build` (exit code: 101)
  --- stdout
  OPT_LEVEL = Some("0")
  TARGET = Some("x86_64-pc-windows-msvc")
  HOST = Some("x86_64-pc-windows-msvc")
  cargo:rerun-if-env-changed=CC_x86_64-pc-windows-msvc
  CC_x86_64-pc-windows-msvc = None
  cargo:rerun-if-env-changed=CC_x86_64_pc_windows_msvc
  CC_x86_64_pc_windows_msvc = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  DEBUG = Some("false")
  cargo:rerun-if-env-changed=CFLAGS_x86_64-pc-windows-msvc
  CFLAGS_x86_64-pc-windows-msvc = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_pc_windows_msvc
  CFLAGS_x86_64_pc_windows_msvc = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None

  --- stderr
  Microsoft (R) C/C++ Optimizing Compiler Version 19.37.32825 for x64
  Copyright (C) Microsoft Corporation.  All rights reserved.

  win64_intel.S
  libffi/src/x86/win64_intel.S(2): fatal error C1083: Cannot open include file: 'fficonfig.h': No such file or directory
  thread 'main' panicked at C:\Users\dhorner\.cargo\registry\src\index.crates.io-6f17d22bba15001f\libffi-sys-2.3.0\build\common.rs:8:5:
  Pre-process ASM
  stack backtrace:
     0:     0x7ff734428faa - std::sys_common::backtrace::_print::impl$0::fmt
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\sys_common\backtrace.rs:44
     1:     0x7ff734444cdb - core::fmt::rt::Argument::fmt
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\core\src\fmt\rt.rs:138
     2:     0x7ff734444cdb - core::fmt::write
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\core\src\fmt\mod.rs:1094
     3:     0x7ff734423f81 - std::io::Write::write_fmt<std::sys::windows::stdio::Stderr>
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\io\mod.rs:1714
     4:     0x7ff734428d2a - std::sys_common::backtrace::_print
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\sys_common\backtrace.rs:47
     5:     0x7ff734428d2a - std::sys_common::backtrace::print
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\sys_common\backtrace.rs:34
     6:     0x7ff73442b94a - std::panicking::default_hook::closure$1
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\panicking.rs:270
     7:     0x7ff73442b5b8 - std::panicking::default_hook
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\panicking.rs:290
     8:     0x7ff73442bffe - std::panicking::rust_panic_with_hook
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\panicking.rs:707
     9:     0x7ff73442beed - std::panicking::begin_panic_handler::closure$0
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\panicking.rs:599
    10:     0x7ff734429c69 - std::sys_common::backtrace::__rust_end_short_backtrace<std::panicking::begin_panic_handler::closure_env$0,never$>
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\sys_common\backtrace.rs:170
    11:     0x7ff73442bbf0 - std::panicking::begin_panic_handler
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\panicking.rs:595
    12:     0x7ff73444a9b5 - core::panicking::panic_fmt
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\core\src\panicking.rs:67
    13:     0x7ff7343a2312 - std::rt::lang_start::h9ebb1ec7920d0356
    14:     0x7ff7343a474a - std::rt::lang_start::h9ebb1ec7920d0356
    15:     0x7ff7343a575b - std::rt::lang_start::h9ebb1ec7920d0356
    16:     0x7ff7343a4c89 - std::rt::lang_start::h9ebb1ec7920d0356
    17:     0x7ff7343a5b21 - std::rt::lang_start::h9ebb1ec7920d0356
    18:     0x7ff7343a1b56 - std::rt::lang_start::h9ebb1ec7920d0356
    19:     0x7ff7343a1569 - __ImageBase
    20:     0x7ff7343a167c - std::rt::lang_start::h9ebb1ec7920d0356
    21:     0x7ff73441f3e8 - std::rt::lang_start_internal::closure$2
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\rt.rs:148
    22:     0x7ff73441f3e8 - std::panicking::try::do_call
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\panicking.rs:502
    23:     0x7ff73441f3e8 - std::panicking::try
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\panicking.rs:466
    24:     0x7ff73441f3e8 - std::panic::catch_unwind
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\panic.rs:142
    25:     0x7ff73441f3e8 - std::rt::lang_start_internal
                                 at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library\std\src\rt.rs:148
    26:     0x7ff7343a1657 - std::rt::lang_start::h9ebb1ec7920d0356
    27:     0x7ff7343a5b49 - main
    28:     0x7ff7344490f8 - invoke_main
                                 at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    29:     0x7ff7344490f8 - __scrt_common_main_seh
                                 at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    30:     0x7ffb3719257d - BaseThreadInitThunk
    31:     0x7ffb3916aa58 - RtlUserThreadStart
warning: build failed, waiting for other jobs to finish..

full trace.

I'm not sure what to do either. what brought on libffi as a dependency?

I also use WSL Ubuntu. I did a cargo clean and a cargo build. It is also not building due to libffi.

error: failed to run custom build command for `libffi-sys v2.3.0`

Caused by:
  process didn't exit successfully: `/mnt/c/w/rust/svix-webhooks/bridge/target/debug/build/libffi-sys-4b62515605633614/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  checking build system type... x86_64-pc-linux-gnu
  checking host system type... x86_64-pc-linux-gnu
  checking target system type... x86_64-pc-linux-gnu
  continue configure in default builddir "./x86_64-unknown-linux-gnu"
  ....exec /bin/bash .././configure "--srcdir=.." "--enable-builddir=x86_64-unknown-linux-gnu" "linux
  gnu"

  --- stderr
  .././configure: line 2251: config.log: No such file or directory
  .././configure: line 2261: config.log: No such file or directory
  cat: standard output: No such file or directory
  thread 'main' panicked at /home/dhorner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libffi-sys-2.3.0/build/common.rs:8:5:
  Configuring libffi
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

yes, I've got a bowl of spaghettis. The log is even less useful from wsl linux. build-script-build is a binary file.

davehorner commented 9 months ago

building libffi-rs works works directly in windows. 

git clone https://github.com/tov/libffi-rs.git
Cloning into 'libffi-rs'...
remote: Enumerating objects: 2820, done.
remote: Counting objects: 100% (1297/1297), done.
remote: Compressing objects: 100% (594/594), done.
remote: Total 2820 (delta 710), reused 1253 (delta 686), pack-reused 1523
Receiving objects: 100% (2820/2820), 1.94 MiB | 5.99 MiB/s, done.
Resolving deltas: 100% (1583/1583), done.

c:\w\rust>cd libffi-rs

c:\w\rust\libffi-rs>ls
 Volume in drive C has no label.
 Volume Serial Number is 1A30-207D

 Directory of c:\w\rust\libffi-rs

[.]              [..]             [.github]        .gitignore       .gitmodules      Cargo.toml       [libffi-rs]
[libffi-sys-rs]  LICENSE-APACHE   LICENSE-MIT      README.md
               6 File(s)         14,702 bytes
               5 Dir(s)  1,002,465,460,224 bytes free

c:\w\rust\libffi-rs>cargo build
    Updating crates.io index
   Compiling cc v1.0.83
   Compiling libc v0.2.150
   Compiling libffi-sys v2.3.0 (C:\w\rust\libffi-rs\libffi-sys-rs)
   Compiling libffi v3.2.0 (C:\w\rust\libffi-rs\libffi-rs)
    Finished dev [unoptimized + debuginfo] target(s) in 4.96s

the same is not true for wsl ubuntu.

**(cv27) dhorner@DESKTOP-Q7LTI6D:/mnt/c/w/rust/libffi-rs$ cargo clean
     Removed 174 files, 31.3MiB total
(cv27) dhorner@DESKTOP-Q7LTI6D:/mnt/c/w/rust/libffi-rs$ cargo build
  Downloaded cc v1.0.83
  Downloaded libc v0.2.150
  Downloaded 2 crates (787.7 KB) in 0.50s
   Compiling libc v0.2.150
   Compiling cc v1.0.83
   Compiling libffi-sys v2.3.0 (/mnt/c/w/rust/libffi-rs/libffi-sys-rs)
error: failed to run custom build command for `libffi-sys v2.3.0 (/mnt/c/w/rust/libffi-rs/libffi-sys-rs)`

Caused by:
  process didn't exit successfully: `/mnt/c/w/rust/libffi-rs/target/debug/build/libffi-sys-5657a3858379d359/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None

  --- stderr
: not foundre: 17:
  ./configure: 35: Syntax error: newline unexpected (expecting ")")
  thread 'main' panicked at libffi-sys-rs/build/common.rs:8:5:
  Configuring libffi
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace**

I realize this is more a libffi-rs issue; looking for any suggestions that might help. I'm not sure how to document it well for libffi issue. still poking around.

davehorner commented 9 months ago

I got windows built with the following:

set INCLUDE=%INCLUDE%;libffi\include;libffi-sys-2.3.0\libffi\src\x86\

I still need to set the cargo run --no-default-features --features gcp-pubsub,rabbitmq,redis,sqs no default due to jemalloc.

I haven't looked at the wsl linux build; it would be nice to understand why that build is not working. I've got a working windows build now.

Skgland commented 6 months ago

I think this is related to libffi/libffi#552 based on the WSL log from https://github.com/svix/svix-webhooks/issues/1128#issuecomment-1820077361

davehorner commented 6 months ago

I think they are unrelated. I also think with my last comment that I resolved the issue. hope that helps someone else.

Skgland commented 6 months ago

I haven't looked at the wsl linux build; it would be nice to understand why that build is not working. I've got a working windows build now.

The wsl Linux build part appears to still be broken.

davehorner commented 6 months ago

ah. I see. yes that's right I did move on. i didn't glean anything actionable from the comment you linked. if someone has an idea to try.... I'm ears.

Skgland commented 1 month ago

It looks like the WSL problem occurs when libffi is build on the windows filesystems mounted under /mnt as discovered by https://github.com/libffi/libffi/issues/552#issuecomment-1705446413 as a workaround it appears to be sufficent to move the target dir onto the WSL filesystem see https://github.com/tov/libffi-rs/issues/89#issuecomment-2245830001, so that libffi-sys-rs builds libffi on the WSL filesystem.