openssl build fail

[andreasdotorg]

When building everything from scratch, building CoreCrypto fails, because it fails to build OpenSSL. However, during building miTLS, another attempt at building OpenSSL is made, this time being successful. So the second run of "everest make" actually will succeed.

Complete log of both builds is attached.

everest build.txt

[sishtiaq]

line 232: gcc not found. I remember having that, I think I just installed gcc (even though you don't need it).

[andreasdotorg]

Is there any need to build OpenSSL at all instead of depending on the system version?

[msprotz]

The fact that gcc is mentioned is worrying: here's the explicit configure invocation for openssl

https://github.com/FStarLang/FStar/blob/master/ucontrib/CoreCrypto/ml/Makefile#L26

note that CC is defined explicitly. Did you, by any chance, build a wrong (i.e. cygwin) OpenSSL?

[msprotz]

windres: preprocessing failed.

[msprotz]

maybe there's a few bits of the openssl makefile that need gcc still, for preprocessing purposes...?

[msprotz]

The reason we need to build openssl from trunk is for support for newer algorithms which the released versions don't have (on windows, that would be the cross-compiled mingw openssl libraries distributes through the cygwin package manager).

[andreasdotorg]

Ok, here's what's going on. On the first pass, the static libcrypto.a is generated. Then the build fails trying to create libcrypto.dll. On the second pass, the Makefile that triggers OpenSSL build finds libcrypto.a, copies it around, skipping another build attempt.

The bug is that mkrc.pl inside OpenSSL seems to ignore the CC settings. The workaround is not building DLLs, as they are not required anyways. I'll add this to my current pull request for FStar.

[sishtiaq]

Good investigation! Is it possible to file a repro for the OpenSSL folks? Also, why do you think the .dll is not required? (I don't know whether it is or not, would be good to confirm that it's not.)

[msprotz]

But do we understand why none of us has this problem on our machines?

[andreasdotorg]

@sishtiaq The Makefile in the level above explicitly talks about the .a, links against that, and I couldn't find any other usage. I might file something with OpenSSL later.

@msprotz You either have a "gcc" callable at the right moment in time, or you just called the build process often enough so you ended up with a libcrypto.a in the right place. Once it's been built, the error vanishes.

[andreasdotorg]

Ah. It's not actually mkrc.pl that is calling gcc. The output of that script is piped into windres, which in turn calls gcc.

[sishtiaq]

@protz I had the build fail the first time I built, because the build wanted gcc to be there.

[andreasdotorg]

Here's where the "gcc" string is living that makes windres fail:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=binutils/resrc.c;h=26ac848c50b55e431ede43cbc8bdc068b88ff70f;hb=HEAD#l78

Apparently, the OpenSSL Makefile should use --preprocessor=$(CC). I'll file somthing with them.

[andreasdotorg]

Some more reading of OpenSSL build infrastructure later... there's nothing wrong with their invocation of windres. Rather, the problem is that the invocation of their Configure command here:

https://github.com/FStarLang/FStar/blob/master/ucontrib/CoreCrypto/ml/Makefile#L26

only overrides the compiler. They expect you to call their script for a cross-compilation setup (which he have when using MinGW to compile under Cygwin) like this:

CROSS_COMPILE=x86_64-w64-mingw32- ./Configure mingw64

which will then lead to x86_64-w64-mingw32-windres being called instead of windres, which then in turn calls x86_64-w64-mingw32-gcc instead of gcc, so all works smoothly. Except that it takes forever, I eventually aborted the shared library build. Passing no-shared to Configure still seems like the best route to me.

[andreasdotorg]

I've tidied up the pull request at https://github.com/FStarLang/FStar/pull/898 to only include OpenSSL-related changes. Also, fixed some more issues on the way.

[andreasdotorg]

Issue closed by merge of FStarLang/FStar#898