8.1 build fail on netbsd 7.0.1, w/pkgsrc

[georgalis]

Hi, completely new to ccs-calendarserver, tried the quick start with git clone -b CalendarServer-8.1 https://github.com/apple/ccs-calendarserver.git and got the following (sh develop) build error.... I see cyrus-sasl-2.1.26 is available as a vendor package but after installing it, the error persists.

May I have some guidance, configuring the cyrus-sasl build or configuring to use my sasl install?

Thanks, George

gcc -DHAVE_CONFIG_H -DSASLAUTHD_CONF_FILE_DEFAULT=\"/home/geo/vcs/vend/ccs-calendarserver/.develop/roots/CyrusSASL/etc/saslauthd.conf\" -I. -I. -I.. -I. -I./include -I./include -I./../include   -I/home/geo/vcs/vend/ccs-calendarserver/.develop/roots/CyrusSASL/include -I/home/geo/vcs/vend/ccs-calendarserver/.develop/roots/libffi/include    -DKRB5_HEIMDAL  -g -O2 -MT auth_rimap.o -MD -MP -MF .deps/auth_rimap.Tpo -c -o auth_rimap.o auth_rimap.c
In file included from mechanisms.h:35:0,
                 from auth_rimap.c:61:
saslauthd.h:190:0: warning: "KRB5_HEIMDAL" redefined [enabled by default]
 #define KRB5_HEIMDAL /**/
 ^
<command-line>:0:0: note: this is the location of the previous definition
auth_rimap.c: In function 'auth_rimap':
auth_rimap.c:375:24: error: storage size of 'timeout' isn't known
         struct timeval timeout;
                        ^
auth_rimap.c:488:24: error: storage size of 'timeout' isn't known
         struct timeval timeout;
                        ^
*** Error code 1

Stop.
make[3]: stopped in /home/geo/vcs/vend/ccs-calendarserver/.develop/src/cyrus-sasl-2.1.26/saslauthd
*** Error code 1

Stop.
make[2]: stopped in /home/geo/vcs/vend/ccs-calendarserver/.develop/src/cyrus-sasl-2.1.26/saslauthd
*** Error code 1

Stop.
make[1]: stopped in /home/geo/vcs/vend/ccs-calendarserver/.develop/src/cyrus-sasl-2.1.26
*** Error code 1

Stop.
make: stopped in /home/geo/vcs/vend/ccs-calendarserver/.develop/src/cyrus-sasl-2.1.26
 geo@init-vb:/home/geo/vcs/vend/ccs-calendarserver/bin uname -a
NetBSD init-vb 7.0.1 NetBSD 7.0.1 (GENERIC.201605221355Z) amd64
 geo@init-vb:/home/geo/vcs/vend/ccs-calendarserver/bin pkgin pc cyrus-sasl-2.1.26nb4 | grep -v /doc/ | grep -v /man/
Information for http://ftp.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/7.0.1/All/cyrus-sasl-2.1.26nb4.tgz:
Files:
/usr/pkg/include/sasl/hmac-md5.h
/usr/pkg/include/sasl/md5.h
/usr/pkg/include/sasl/md5global.h
/usr/pkg/include/sasl/prop.h
/usr/pkg/include/sasl/sasl.h
/usr/pkg/include/sasl/saslplug.h
/usr/pkg/include/sasl/saslutil.h
/usr/pkg/lib/libsasl2.la
/usr/pkg/lib/libsasl2.a
/usr/pkg/lib/libsasl2.so
/usr/pkg/lib/libsasl2.so.3
/usr/pkg/lib/libsasl2.so.3.0.0
/usr/pkg/lib/pkgconfig/libsasl2.pc
/usr/pkg/lib/sasl2/libsasldb.la
/usr/pkg/lib/sasl2/libsasldb.a
/usr/pkg/lib/sasl2/libsasldb.so
/usr/pkg/lib/sasl2/libsasldb.so.3
/usr/pkg/lib/sasl2/libsasldb.so.3.0.0
/usr/pkg/sbin/pluginviewer
/usr/pkg/sbin/sasldblistusers2
/usr/pkg/sbin/saslpasswd2

[georgalis]

I did find an applicable patch for the cyrus-sasl build ftp://ftp.netbsd.org/pub/pkgsrc/pkgsrc-2016Q2/pkgsrc/security/cyrus-sasl/patches/patch-saslauthd_auth__rimap.c

$NetBSD: patch-saslauthd_auth__rimap.c,v 1.3 2013/09/30 05:44:33 obache Exp $

* Added missing include files for struct timeval
  http://git.cyrusimap.org/cyrus-sasl/commit/?id=f2e77ab1ac01f611d71ae56ff7635a46b930ef8e

--- saslauthd/auth_rimap.c.orig 2012-10-12 14:05:48.000000000 +0000
+++ saslauthd/auth_rimap.c
@@ -75,6 +75,16 @@
 #include <arpa/inet.h>
 #include <signal.h>
 #include <netdb.h>
+#if TIME_WITH_SYS_TIME
+# include <sys/time.h>
+# include <time.h>
+#else
+# if HAVE_SYS_TIME_H
+#  include <sys/time.h>
+# else
+#  include <time.h>
+# endif
+#endif

 #include "auth_rimap.h"
 #include "utils.h"

seems straight forward... pkgsrc is designed to be (UNIX/Linux) OS agnostic, I suspect all the patches in that directory would be good for upstream. Not sure the status of that.

In the mean time, is there a standard way to configure for my installed cyrus-sasl-2.1.26 binary?

[georgalis]

Success.... workaround anyway.

pkgsrc is not linked to the OS, so the environment requires CPATH='/usr/pkg/include' for various header and lib includes. Not sure of the best way to patch ./bin/develop to detect and use /usr/pkg/include (maybe use $PATH to detect additional ./include directories?), or to set it via a config file. Setting CPATH does cause unexpected warnings with ./bin/develop (one [[: not found for each package dependency I think) and ./bin/run (a single [: ==: unexpected operator).

With CPATH in place. I installed memcached and postgresql95 as binaries to save time/trouble installing them. However, cyrus-sasl mit-krb5 py27-pip where installed due to various ./bin/develop detect or build failures.

With Dependency setup complete ./bin/run failures where resolved by installing the additional packages: py27-cElementTree py27-sqlite3

ImportError: No module named _elementtree
ImportError: No module named pysqlite2

pkgin in memcached postgresql95 # may use binaries
pkgin in cyrus-sasl mit-krb5 py27-pip # must use binaries ./bin/develop fails these
pkgin in py27-cElementTree py27-sqlite3 # must use binaries, ./bin/run fails these
export CPATH='/usr/pkg/include' && sh bin/develop
....
[[: not found
Dependency setup complete.

I have no postgres running (yet) so failure to connect is reasonable, looks like a successful ./bin/run -n!

I would like to help refine the ./bin/develop stack but my first priority is integrating calendarserver. I can probably offer a reasonable patch to set CPATH based on PATH data and the _build.sh logic. Not sure of the root cause for the other issues though.

[georgalis]

I've created patches to address bootstrapping ./bin/develop on NetBSD w/pkgsrc. Here is a summary.

• convert .gitignore to relative path
• Convert == equivalency tests in _build.sh to POSIX.
• Fix "[[: not found" errors, on POSIX shell, by converting to an equivalent 'case' conditional statement, in _build.sh.
• Remove "develop" shell/function ambiguity and streamline bootstrap.
• Add include directory discovery by setting $CPATH based on $PATH

(...the restructuring caused problems with ./bin/run (at least) so pull up request is retracted until the issues are resolved)

The following is from the last commit comment and describes the change in detail:

Move four function invocations from _build.sh to develop, facilitating removal of the "develop" function, now _build.sh and _py.sh are exclusively function definitions.

Move invocation to source _py.sh from a _build.sh function to the develop script.

With the above changes, bootstrapping the development platform environment is a linear sequence of events in the ./bin/develop script, and easier to follow, since the ambiguity of a function and a script both named "develop" has been removed.

Now the main function workflow is invoked from ./bin/develop script.

An additional function 'gen_cpath' was added to _build.sh, it is called as a 'conditional_set' parameter on CPATH. It uses the PATH env to locate appropriate include directories. (Initially the C_INCLUDE_PATH varable was set, but CPATH will support any future cpp dependencies that may come up). CPATH must be exported for the desired effect.