ccs-calendarserver: Address build issues with newer dependencies

Update to openssl 1.0.2r
Update to sqlparse 0.2.4 in ./requirements-cs.txt
WARNING: Does not yet address invoke './Configure darwin64-x86_64-cc' *manually* in .develop/src/openssl-1.0.2r/config, ld: symbol(s) not found for architecture i386 errors
See: https://github.com/apple/ccs-twistedextensions/pull/20
See: https://gist.github.com/essandess/4c9b5d906896ff4aef6170fc18f2c293

Comments:

OpenSSL build issues may be sidestepped by using the MacPorts (or other) openssl installs as system libraries. This is preferred to using a package-specific library that may not be upgraded as regularly as its dependency. Working example that yields a successful run of ./bin/package /path/to/CalendarServer:

cat <<PACKAGE_CALENDARSERVER | sh -x
(
export PATH=/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
export C_INCLUDE_PATH=/opt/local/include:/usr/include:/usr/local/include
export LD_LIBRARY_PATH=/opt/local/lib:/opt/local/lib/openssl-1.0:/usr/lib:/usr/local/lib
export CPPFLAGS='-I/opt/local/include -I/usr/include -I/usr/local/include'
export LDFLAGS='-L/opt/local/lib -L/opt/local/lib/openssl-1.0 -L/usr/lib -L/usr/local/lib'
export DYLD_LIBRARY_PATH=/opt/local/lib:/opt/local/lib/openssl-1.0:/usr/lib:/usr/local/lib

./bin/package -F /private/var/calendarserver/Library/CalendarServer 1> ~/ccs-calendarserver.log 2>&1
)
PACKAGE_CALENDARSERVER

By submitting a request, you represent that you have the right to license your contribution to Apple and the community, and agree that your contributions are licensed under the Apache License Version 2.0.

For existing files modified by your request, you represent that you have retained any existing copyright notices and licensing terms. For each new file in your request, you represent that you have added to the file a copyright notice (including the year and the copyright owner's name) and the Calendar and Contacts Server's licensing terms.

Before submitting the request, please make sure that your request follows the Calendar and Contacts Server's guidelines for contributing code.

These install issues are observed without the PR edits:

ISSUE 1: openssl fails to build, warnings about 64-bit architecture:

Operating system: i686-apple-darwinDarwin Kernel Version 18.5.0: Mon Mar 11 20:40:32 PDT 2
019; root:xnu-4903.251.3~3/RELEASE_X86_64
WARNING! If you wish to build 64-bit library, then you have to
         invoke './Configure darwin64-x86_64-cc' *manually*.
Configuring for darwin-i386-cc
…
     (maybe you meant: _TS_RESP_CTX_set_time_cb, _TS_TST_INFO_set_time , _PKCS7_add0_attrib_signing_time , _X509_VERIFY_PARAM_set_time , _X509_time_adj_ex , _X509_time_adj , _X509_cmp_time , _ASN1_UTCTIME_cmp_time_t , _X509_cmp_current_time , _X509_STORE_CTX_set_time , _TS_TST_INFO_get_time , _err_clear_last_constant_time )
  "_vfprintf", referenced from:
      _OPENSSL_showfatal in libcrypto.a(cryptlib.o)
  "_write$UNIX2003", referenced from:
      _DES_enc_write in libcrypto.a(enc_writ.o)
      _fd_write in libcrypto.a(bss_fd.o)
      _sock_write in libcrypto.a(bss_sock.o)
      _conn_write in libcrypto.a(bss_conn.o)
      _conn_puts in libcrypto.a(bss_conn.o)
      _dgram_write in libcrypto.a(bss_dgram.o)
      _RAND_query_egd_bytes in libcrypto.a(rand_egd.o)
      ...
ld: symbol(s) not found for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[4]: *** [link_a.darwin] Error 1
make[3]: *** [do_darwin-shared] Error 2
make[2]: *** [libcrypto.1.0.0.dylib] Error 2
make[1]: *** [shared] Error 2
make: *** [build_crypto] Error 1

Add #!/bin/sh -x to ./bin/package, and run as:

sh -x -c './bin/package -F /private/var/CalendarServer 1> ~/ccs-calendarserver.log 2>&1'

Examining ./bin/package and ./bin/_build.sh, we'll try to:

Update the downloaded openssal to the latest version (1.0.2r) at time of writing

diff ../ccs-calendarserver/bin/_build.sh ./bin/_build.sh 
526c526
<       local v="1.0.2h";
---
>       local v="1.0.2r";
532c532
<       c_dependency -c "config" -s "577585f5f5d299c44dd3c993d3c0ac7a219e4949" \
---
>       c_dependency -c "config" -s "b9aec1fa5cedcfa433aed37c8fe06b0ab0ce748d" \

Coerce .develop/src/openssl-1.0.2r/config to issue ./Configure darwin64-x86_64-cc (why does it create a Makefile with PLATFORM=darwin-i386-cc??)
Perhaps just use MacPorts openssl as the system openssl. That's probably best because MacPorts openssl will be patched and upgradded regularly.

Use MacPorts openssl (the cc command within ./bin/_build.sh's ssl_version="$(c_macro openssl/ssl.h OPENSSL_VERSION_NUMBER)" should catch these):

export PATH=/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
export C_INCLUDE_PATH=/opt/local/include
export LD_LIBRARY_PATH=/opt/local/lib:/opt/local/lib/openssl-1.0
export CPPFLAGS='-I/opt/local/include'
export LDFLAGS='-L/opt/local/lib -L/opt/local/lib/openssl-1.0'
export DYLD_LIBRARY_PATH=/opt/local/lib:/opt/local/lib/openssl-1.0

sh -E -x -c './bin/package -F /private/var/CalendarServer 1> ~/ccs-calendarserver.log 2>&1'

This works with log entry Using system version of OpenSSL.

ISSUE 2:

Installed /private/var/CalendarServer/virtualenv/lib/python2.7/site-packages/CalendarServer-9.3a1+unknown-py2.7.egg
Processing dependencies for CalendarServer==9.3a1+unknown
error: sqlparse 0.2.4 is installed but sqlparse==0.2.0 is required by set(['twextpy'])

This version requirement is set in apple/ccs-calendarserver/requirements-cs.txt and apple/ccs-twistedextensions/setup.py.

Correction to this PR: I am able to get a working ./package build with the current version if I deprecate but include all MacPorts libraries. There are additional issues using more recent libraries (see below).

Because is is related to Python 2.7 EOL and the need to migrate (https://github.com/apple/ccs-calendarserver/issues/537), I'll leave this PR open.

These work for me to build ccs-calendarserver. Now if I could just get a working calenderserver.plist config file…

# macOS-native Python needs the pip module
sudo /usr/bin/easy_install pip

# MacPorts openssl
sudo port install openssl

# /opt/local is deprecated, but included for openssl
cat <<PACKAGE_CALENDARSERVER | sh -x
(
    export PATH=/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/bin
    export C_INCLUDE_PATH=/usr/include:/usr/local/include:/opt/local/include
    export LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/local/lib:/opt/local/lib/openssl-1.0
    export CPPFLAGS='-I/usr/include -I/usr/local/include -I/opt/local/include'
    export LDFLAGS='-L/usr/lib -L/usr/local/lib -L/opt/local/lib -L/opt/local/lib/openssl-1.0'
    export DYLD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/local/lib:/opt/local/lib/openssl-1.0

    ./bin/package -F /private/var/calendarserver/Library/CalendarServer 1> ~/ccs-calendarserver.log 2>&1
)
PACKAGE_CALENDARSERVER

A package built with this PR will throw this Python error:

File "/private/var/calendarserver/Library/CalendarServer/virtualenv/src/twextpy/twext/enterprise/dal/parseschema.py", line 724, in significant
return (not isinstance(token, Comment) and not token.is_whitespace())
TypeError: 'bool' object is not callable

apple / ccs-calendarserver

ccs-calendarserver: Address build issues with newer dependencies #536