search

diskuv / dkml-installer-ocaml

The Windows-friendly distribution of OCaml
Apache License 2.0
61 stars 2 forks source link

Installer fails at curl due to self-signed certificate #19

Closed BarronRauEA closed 1 year ago

BarronRauEA commented 1 year ago

I am installing on a machine that is behind a proxy and uses a company certificate. The failure is:

windows_install.bc: [INFO] Downloading https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe
windows_install.bc: [INFO] Running command: C:\Users\BARRON~2\AppData\Local\Temp\7z17ECACAC\sg\staging-curl\windows_x86_64\bin\curl.exe -L -o C:\Users\BARRON~2\AppData\Local\Temp\path_eval_77c67b\curlo01eb9f https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0

curl: (60) SSL certificate problem: self-signed certificate in certificate chain More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.

FATAL [e6435b12]. A transient failure occurred.

Root cause: The conformant command had exit code 60: C:\Users\BARRON~2\AppData\Local\Temp\7z17ECACAC\sg\staging-curl\windows_x86_64\bin\curl.exe -L -o C:\Users\BARRON~2\AppData\Local\Temp\path_eval_77c67b\curlo01eb9f https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe A transient failure occurred. dkml-package.bc: [ERROR] FATAL [5f927a8b]. A transient failure occurred. Root cause: The command had exit code 20: C:\Users\BARRON~2\AppData\Local\Temp\7z17ECACAC\bin\dkml-install-user-runner.exe install-user-network-unixutils --verbosity=info --color=always --prefix C:\Users\BarronRauEA\AppData\Local\Programs\DiskuvOCaml --staging-files C:\Users\BARRON~2\AppData\Local\Temp\7z17ECACAC\sg A transient failure occurred.

It appears that the installer uses a copy of curl from a temp file at C:\Users\BarronRauEA\AppData\Local\Temp\7z6912243C\sg\staging-curl\windows_x86_64\bin. I tried adding my cert to the end of the curl-ca-bundle.crt file and rerunning the installer but had the same problem. I also tried removing the staged cert file and curl.exe and replaced them with a shortcut to my local curl installation, with the same failure.

Is it possible for the installer to use my existing instance of curl instead of the staged one or is there a way for the staged instance to use my certificate?

jonahbeckford commented 1 year ago

The only good solution is to generate a more offline setup that bundles as much content as possible into the installer.

I'll prioritize this issue. I can't bundle everything, so I may need some back and forth with you in this ticket to get an offline setup that works.

BarronRauEA commented 1 year ago

I hate to inconvenience you, but I'd love to muddle through testing to help get it working. Let me know what I can do and when.

jonahbeckford commented 1 year ago

If you are willing to get your hands dirty, can you:

  1. Run the uninstaller (https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.0.1/uninstall-diskuv-ocaml-windows_x86_64-1.0.1.exe) to remove any partial files.
  2. Install 7zip if you don't have it yet
  3. Uncompress the DKML setup-*.exe file.
  4. There should be curl.exe and curl-ca-bundle.crt in sg\staging-curl\windows_x86_64\bin\ one or two places in the directory tree ... replace them both.
  5. I'll update this ticket shortly with how to run the installer directly from the uncompressed directory tree Run: setup.exe -v -v to run with verbose logging.
  6. Update the ticket with your progress
BarronRauEA commented 1 year ago

New error code from curl now: curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.

I tried with both the original curl-ca-bundle.crt with my cert appended and with my cert renamed to curl-ca-bundle.crt. Copying my curl in is necessary to get the new error code.

jonahbeckford commented 1 year ago

Can you try running your system (company) curl directly and reply with the output?

curl -v -L -o msys2-base.exe https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe
BarronRauEA commented 1 year ago

Sorry, looks like that is my problem. I'll check with someone here and let you know if I still have a problem. Do you want me to close this issue now or wait until my install is successful?

C:\Users\BarronRauEA>curl -v -L -o msys2-base.exe https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 140.82.112.4:443...

jonahbeckford commented 1 year ago

Let's keep it open. Can you try to add -k to the curl command and see if that works?

(DKML uses sha256 checksum validation after downloading any content, so the insecure mode (-k) of curl can be safe to use if the checksum validation can be guaranteed.)

BarronRauEA commented 1 year ago

It does work with -k. I should have included that info before.

Perhaps it is too kludgy, but would it be possible to pass the curl -k flag through from the command line when calling the setup-diskuv-ocaml*?

From: jonahbeckford @.> Sent: Friday, December 2, 2022 4:08 PM To: diskuv/dkml-installer-ocaml @.> Cc: Barron Rau @.>; Author @.> Subject: Re: [diskuv/dkml-installer-ocaml] Installer fails at curl due to self-signed certificate (Issue #19)

Let's keep it open. Can you try to add -k to the curl command and see if that works?

(DKML uses sha256 checksum validation after downloading any content, so the insecure mode (-k) of curl can be safe to use if the checksum validation can be guaranteed.)

— Reply to this email directly, view it on GitHubhttps://github.com/diskuv/dkml-installer-ocaml/issues/19#issuecomment-1335846619, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOJKXISP6MI4APQCUN7Q25TWLJQMHANCNFSM6AAAAAASSGRVHU. You are receiving this because you authored the thread.Message ID: @.**@.>>

jonahbeckford commented 1 year ago

No, it is not possible to pass the curl flag down. But am making a change to the installer to embed the msys2-base-x86_64-20220128.sfx.exe inside the installer so curl is not needed.

BarronRauEA commented 1 year ago

Good morning Jonah,

Thanks for all your work on this. When I go to https://github.com/diskuv/dkml-installer-ocaml/releases and click on https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.0.2/setup-diskuv-ocaml-windows_x86_64-1.0.2.exe I get a 404 error. Let me know when you want me to try it again or if there is some other way I should be downloading the installer.

jonahbeckford commented 1 year ago

Use:

Use the uninstaller first, and then the installer. Tell me how that goes.

BarronRauEA commented 1 year ago

Well, we bypassed the first problem. Looks like the certificate is causing another problem. I had my MSP looking at it end of the day Friday and expect to hear more from him Monday. Output now is (I edited out the repetitive lines):

2022-12-03T18:41:11.974Z [MSYS2] pacman -Syu --noconfirm :: Synchronizing package databases... error: failed retrieving file 'mingw32.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain error: failed retrieving file 'mingw64.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain error: failed retrieving file 'ucrt64.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain

warning: too many errors from mirror.msys2.org, skipping for the remainder of this transaction

error: failed retrieving file 'clang64.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain

error: failed retrieving file 'msys.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain

error: failed to synchronize all databases (download library error) Setup did not complete because an error occurred. MSYS2 command failed! Exited with 1. Command was: env MSYSTEM=CLANG64 MSYSTEM_PREFIX=/clang64 HOME=/home/BarronRauEA PATH=/clang64/bin:/usr/bin:/bin env PATH=/c/Program Files/Git/cmd:/clang64 /bin:/usr/bin:/bin DKML_TMP_PARENTDIR=/c/Users/BARRON~2/AppData/Local/Temp/PA1A29~1/0 pacman -Syu --noconfirm

at Invoke-MSYS2Command, C:\Users\BarronRauEA\AppData\Local\Temp\7zAF6DF2B4\sg\network-ocamlcompiler\windows_x86_64\dkmldir\vendor\drd\src\windows\UnixInvokers\UnixInvokers.psm1: line 165 at Invoke-MSYS2CommandWithProgress, C:\Users\BarronRauEA\AppData\Local\Temp\7zAF6DF2B4\sg\network-ocamlcompiler\windows_x86_64\setup-userprofile.ps1: line 833 at <ScriptBlock>, C:\Users\BarronRauEA\AppData\Local\Temp\7zAF6DF2B4\sg\network-ocamlcompiler\windows_x86_64\setup-userprofile.ps1: line 1037 at <ScriptBlock>, <No file>: line 1 at <ScriptBlock>, <No file>: line 1

[Advanced] MSYS2 commands can be run with: C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\tools\MSYS2\msys2_shell.cmd

Bug Reports can be filed at https://github.com/diskuv/dkml-installer-ocaml/issues Please copy the error message and attach the log file available at C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\setup-userprofile.full.log

FATAL [a0d16230]. A transient failure occurred.

Root cause: The conformant command had exit code 1: C:\Users\BARRON~2\AppData\Local\Temp\7zAF6DF2B4\sg\network-ocamlcompiler\windows_x86_64\setup-userprofile.bat -AllowRunAsAdmin -InstallationPrefix C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1 -MSYS2Dir C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\tools\MSYS2 -OpamExe C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\bin\OPAM-R~1.EXE -DkmlPath C:\Users\BARRON~2\AppData\Local\Temp\7ZAF6D~1\sg\NETWOR~1\WINDOW~1\dkmldir -GlobalCompileDir C:\Users\BARRON~2\AppData\Local\Temp\7ZAF6D~1\sg\STAGIN~1\generic -NoDeploymentSlot -DkmlHostAbi windows_x86_64 -TempParentPath C:\Users\BARRON~2\AppData\Local\Temp\PA1A29~1 -SkipProgress

A transient failure occurred.

dkml-package.bc: [ERROR] FATAL [5f927a8b]. A transient failure occurred.

Root cause: The command had exit code 20: C:\Users\BARRON~2\AppData\Local\Temp\7zAF6DF2B4\bin\dkml-install-user-runner.exe install-user-network-ocamlcompiler --verbosity=info --color=always --prefix C:\Users\BarronRauEA\AppData\Local\Programs\DiskuvOCaml --staging-files C:\Users\BARRON~2\AppData\Local\Temp\7zAF6DF2B4\sg

A transient failure occurred.

Press "y" and ENTER to exit the installer.

jonahbeckford commented 1 year ago

Sigh. You really need access to the MSYS2 package manager. There is a flow documented at https://www.msys2.org/docs/faq/ for accepting company TLS certificates, but there is no way to do that today in the middle of the installation process.

The modifications to support that will have to wait until mid-next week at the earliest, and then I want to make sure everyone in your company who wants to use it has simple steps. Can you do a Zoom chat (or some alternative) on Tuesday?

BarronRauEA commented 1 year ago

Perhaps a bit of explanation -

I'm an accountant and program some for work and some for hobby. This will be mostly hobby for now and I'm the only one who will need it installed. The only internet connection I have is through my business. We're part of a conservative Mennonite church that requires we only access the internet through a church-approved filter. The filter uses the self-signed certificate to man-in-the-middle the traffic for filtering purposes. I feel bad having you put so much time into this, but perhaps you want to work the bugs out of the installer for others.

I'd be happy to do a screen share or some other connection if you like. We don't use the video capabilities on zoom so let me know what works best, Zoom, Teams, TeamViewer, etc are all options from my end.

Would it work to add the update-ca-trust command to msys command line before synchronizing the package databases? I could copy the certificate into the installer \sg\offline-unixutils\windows_x86_64\bin\msys2-base.sfx.exe\msys64\etc\pki\ca-trust\source\anchors\ before running it. I only half know what I am doing, so don't wonder if you're missing something if the idea is crazy.

Enjoy your Lord's day and thanks for your patience.

jonahbeckford commented 1 year ago

Oh no problem. Like you mentioned I think the proxy environment you have is fairly normal. I'd like to get it working for you and others.

Can you schedule some time with https://calendly.com/diskuv/30-minute-dkml-installation? We won't be able to solve the problem, but I want to test that copying the certificates actually works. We'll test it outside the installer. Then I'll know that a change to the installer will work.

Have a blessed day as well!

BarronRauEA commented 1 year ago

On for Monday the 12th. I'm out of the office Wed. through Fri. next week (7th - 12th).

jonahbeckford commented 1 year ago

Meeting Summary:

Internal notes: Oddly enough the third option (pre-existing MSYS2) already is possible ... that is how DKML is tested on GitHub and GitLab which come bundled with MSYS2. But there is not a specific option in setup.exe to pass the MSYS2 option down to the MSYS2 component (dkml-component-unixutils), nor is there a parameters file that can be read by the MSYS2 component. But I think there is an MSYS2 override option in setup-userprofile.ps1 (will check).

jonahbeckford commented 1 year ago

With that test we did, I think you should be able to complete most of the installation right now.

Using the links you had earlier in this thread,

  1. You want to keep the already-installed files that are in C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1. If that directory is empty or non-existent, rerun the uninstaller and then the installer so that it fails at the "self-signed certificate" part.
  2. Decompress the installer (using 7zip) into a directory.

Then run the following in PowerShell:

# Whatever directory you use to decompress the installer
cd C:\Users\beckf\Downloads\unsigned-diskuv-ocaml-windows_x86_64-i-1.0.1

if (!(test-path "$env:TEMP\dkml-issue-19")) { new-item -itemtype directory "$env:TEMP\dkml-issue-19" }

# Change the MSYS2Dir to your customized installation!
sg\network-ocamlcompiler\windows_x86_64\setup-userprofile.bat `
    -AllowRunAsAdmin `
    -InstallationPrefix `
    C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1 `
    -MSYS2Dir `
    C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\tools\MSYS2 `
    -OpamExe C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\bin\OPAM-R~1.EXE `
    -DkmlPath "$PWD\sg\NETWOR~1\WINDOW~1\dkmldir" `
    -GlobalCompileDir "$PWD\sg\STAGIN~1\generic" `
    -NoDeploymentSlot -DkmlHostAbi windows_x86_64 `
    -TempParentPath "$env:TEMP\dkml-issue-19" `
    -SkipProgress

Please tell me if anything else errors.

BarronRauEA commented 1 year ago

I’ll try to get to it later today or tomorrow.

jonahbeckford commented 1 year ago

Spoke too soon. Please do not use it yet. I will update this when I have a better pre-release to give you!

I have pre-release "17" that you can use.

Installer: https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.1.0-prerel17/unsigned-diskuv-ocaml-windows_x86_64-i-1.0.1.exe Uninstaller: https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.1.0-prerel17/unsigned-diskuv-ocaml-windows_x86_64-u-1.0.1.exe

Here is the relevant part of the upcoming release notes:

New security:

(I don't have an ETA when the full release will be available, although it will be fairly soon)

jonahbeckford commented 1 year ago

You can use https://github.com/diskuv/dkml-installer-ocaml/releases/tag/v1.1.0_r2 . Search for trust_anchors on that release page which will tell you how to use your .pem / .cer CA files.