Update "with-openssl-1.1.x" image to build from source

SeanTAllen commented 4 years ago

There are two problems with using the alpine openssl 1.1.x package:

The version can change out from underneath us
It doesn't necessarily have all the algos we need to test against turned on.

For CI purposes, we should be building from source so that all the algos we need are guaranteed to be on.

We should be using a multistage build for this and copying the built library over into the image. To verify that the image is good when we set up the dockerfiles, we should manually do a ponylang/crypto clone and make test run to verify it all works as expected.

damon-kwok commented 4 years ago

Get source from master branch (don't use tag:1.1.1)

git clone https://github.com/openssl/openssl.git --depth=1
cd openssl

Linux

output=$HOME/ssl

# for gcc
./Configure linux-x86_64 zlib rc5 no-shared –api=1.1.1 --prefix=$output --openssldir=$output 
# for clang   
./Configure linux-x86_64-clang zlib rc5 no-shared  –api=1.1.1 --prefix=$output --openssldir=$output 

make
make test
make install

macOS

output=$HOME/ssl

# for clang   (default)
./Configure darwin64-x86_64 zlib rc5 no-shared  –api=1.1.1 --prefix=$output --openssldir=$output 

# for gcc
./Configuredarwin64-x86_64-cc zlib rc5 no-shared –api=1.1.1 --prefix=$output --openssldir=$output 

make
make test
make install

BSD

output=$HOME/ssl

#  clang only   (default)
./Configure BSD-x86_64 zlib rc5 no-shared  –api=1.1.1 --prefix=$output --openssldir=$output 

make
make test
make install

Windows

perl Configure VC-WIN64 zlib rc5 no-shared –api=1.1.1 --prefix=d:\openssl_lib  --openssldir=d:\openssl_lib
nmake
nmake test
nmake install

pick os/compiler from:

BS2000-OSD BSD-generic32 BSD-generic64 BSD-ia64 BSD-sparc64 BSD-sparcv8 
BSD-x86 BSD-x86-elf BSD-x86_64 Cygwin Cygwin-i386 Cygwin-i486 Cygwin-i586 
Cygwin-i686 Cygwin-x86 Cygwin-x86_64 DJGPP MPE/iX-gcc OS390-Unix UEFI UEFI-x86 
UEFI-x86_64 UWIN VC-CE VC-WIN32 VC-WIN32-ARM VC-WIN32-ARM-UWP VC-WIN32-ONECORE 
VC-WIN32-UWP VC-WIN64-ARM VC-WIN64-ARM-UWP VC-WIN64A VC-WIN64A-ONECORE 
VC-WIN64A-UWP VC-WIN64A-masm VC-WIN64I aix-cc aix-gcc aix64-cc aix64-gcc 
android-arm android-arm64 android-armeabi android-mips android-mips64 
android-x86 android-x86_64 android64 android64-aarch64 android64-mips64 
android64-x86_64 bsdi-elf-gcc cc darwin-i386 darwin-i386-cc darwin-ppc 
darwin-ppc-cc darwin64-debug-test-64-clang darwin64-ppc darwin64-ppc-cc 
darwin64-x86_64 darwin64-x86_64-cc gcc haiku-x86 haiku-x86_64 hpux-ia64-cc 
hpux-ia64-gcc hpux-parisc-cc hpux-parisc-gcc hpux-parisc1_1-cc 
hpux-parisc1_1-gcc hpux64-ia64-cc hpux64-ia64-gcc hpux64-parisc2-cc 
hpux64-parisc2-gcc hurd-x86 ios-cross ios-xcrun ios64-cross ios64-xcrun 
iossimulator-xcrun iphoneos-cross irix-mips3-cc irix-mips3-gcc irix64-mips4-cc 
irix64-mips4-gcc linux-aarch64 linux-alpha-gcc linux-aout linux-arm64ilp32 
linux-armv4 linux-c64xplus linux-elf linux-generic32 linux-generic64 
linux-ia64 linux-mips32 linux-mips64 linux-ppc linux-ppc64 linux-ppc64le 
linux-sparcv8 linux-sparcv9 linux-x32 linux-x86 linux-x86-clang linux-x86_64 
linux-x86_64-clang linux32-s390x linux64-mips64 linux64-s390x linux64-sparcv9 
mingw mingw64 purify sco5-cc sco5-gcc solaris-sparcv7-cc solaris-sparcv7-gcc 
solaris-sparcv8-cc solaris-sparcv8-gcc solaris-sparcv9-cc solaris-sparcv9-gcc 
solaris-x86-gcc solaris64-sparcv9-cc solaris64-sparcv9-gcc solaris64-x86_64-cc 
solaris64-x86_64-gcc tru64-alpha-cc tru64-alpha-gcc uClinux-dist 
uClinux-dist64 unixware-2.0 unixware-2.1 unixware-7 unixware-7-gcc vms-alpha 
vms-alpha-p32 vms-alpha-p64 vms-ia64 vms-ia64-p32 vms-ia64-p64 vos-gcc 
vxworks-mips vxworks-ppc405 vxworks-ppc60x vxworks-ppc750 vxworks-ppc750-debug 
vxworks-ppc860 vxworks-ppcgen vxworks-simlinux debug debug-erbridge

damon-kwok commented 4 years ago

Now, we got: libcrypto.a libssl.a. But， I’m thinking why we do not use 3.0.0. Our static link library will not interfere with the dynamic library of OS.

SeanTAllen commented 4 years ago

Why do you say it won't interfere with the dynamic library? @damon-kwok

None of what you have here is usable in the image for CI. This is about the CI image, not anything larger. If you want to discuss something outside of the scope of this image, let's do that elsewhere so this can be focused on the issue.

SeanTAllen commented 4 years ago

I pushed a test of this and tried out with crypto and got a segfault so... not so good so far...

https://github.com/ponylang/crypto/pull/43/checks?check_run_id=986060565

SeanTAllen commented 4 years ago

Here's the docker file. It's working with latest ponyc release.

If anyone want to try and get a build of openssl that works correctly without sefgaulting on the ponylang/crypto unit tests.

ARG FROM_TAG=release
FROM ponylang/shared-docker-ci-x86-64-unknown-linux-builder:${FROM_TAG}

RUN apk update \
  && apk upgrade \
  && apk add --update \
  clang-dev \
  linux-headers \
  perl

RUN cd /tmp && \
  git clone https://github.com/openssl/openssl.git --depth=1 && \
  cd openssl && \
  ./Configure --api=1.1.1 no-shared enable-rc5 enable-md2 && \
  make install_sw && \
  cd /tmp && \
  rm -rf openssl

SeanTAllen commented 4 years ago

Ok, I have this working in a test.

SeanTAllen commented 4 years ago

PR has been opened for this.

ponylang / shared-docker