Fresh install of CalendarServer throwing END OF LIFE warnings, Certificate Errors

[essandess]

I'm migrating all of my Server.app services to open source per the macOS Server Service Migration Guide.

I have nice working up-to-date versions of all services except for a calDAV/cardDAV server, and am following the README.rst instructions on this repo.

When I follow these instructions, the ./bin/develop sh script throws numerous END OF LIFE warnings and errors.

I'd like to address these and get a working CalendarServer.

I have an up-to-date, working Python and pip environment installed using MacPorts, versions 3.7 and the deprecated 2.7.

First, check versions:

python --version
Python 3.7.3
pip --version
pip 19.1 from /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip (python 3.7)

Second, clone the repo:

git clone -b release/CalendarServer-9.3 https://github.com/apple/ccs-calendarserver.git
mkdir ccs-calendarserver.build
rsync -a --exclude '.git*' --delete-excluded --del ccs-calendarserver/ ccs-calendarserver.build
cd ccs-calendarserver.build

Third, follow the instructions:

./bin/develop
____________________________________________________________

Using system version of libffi.

____________________________________________________________

Using system version of OpenLDAP.

____________________________________________________________

Using system version of SASL.

____________________________________________________________

Using system version of memcached.

____________________________________________________________

Using system version of Postgres.

____________________________________________________________

Installing setuptools==18.5
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting setuptools==18.5
  Downloading https://files.pythonhosted.org/packages/69/9d/e6ff1608e4308ae7c544050b58926daa4bf7764e963c09cc9f9a0bcd3728/setuptools-18.5-py2.py3-none-any.whl (462kB)
     |████████████████████████████████| 471kB 2.1MB/s 
ERROR: markdown 3.1 has requirement setuptools>=36, but you'll have setuptools 18.5 which is incompatible.
Installing collected packages: setuptools
Successfully installed setuptools-18.5
____________________________________________________________

Installing pip==9.0.1
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting pip==9.0.1
  Downloading https://files.pythonhosted.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl (1.3MB)
     |████████████████████████████████| 1.3MB 2.1MB/s 
Installing collected packages: pip
Successfully installed pip-9.0.1
____________________________________________________________

Installing virtualenv==15.0.2
Collecting virtualenv==15.0.2
  Downloading https://files.pythonhosted.org/packages/bf/f4/43525ce41d4de0128460da3dce059c2c969f4d076bbdbac21b6bc1e996e8/virtualenv-15.0.2-py2.py3-none-any
…
Keychain init.
Could not find identity 'org.calendarserver.test'
No certificate 'localhost' found for identity 'org.calendarserver.test'
Imported certificate 'localhost'
Trusted certificate 'localhost'
Created identity 'org.calendarserver.test' for certificate 'localhost'

[essandess]

I've used the output from ./bin/develop to install all the necessary Python version 2.7 dependencies, and see that this repo appears too want to download specific dependencies into a v.2.7 virtualenv. So it's all right for the time being to overlook the deprecated version 2.7 issue.

I see from README.rst:

WARNING: these instructions are for running a server from the source tree, which is useful for development. These are not the correct steps for running the server in deployment or as part of an OS install.

What are the correct steps for running the server in deployment or as part of an OS install?

For example, the Python script ./bin/keychain_init.py installs an example certificate into my keychain:

identity_preference = "org.calendarserver.test"
certname_regex = re.compile(r'"alis"<blob>="(.*)"')

certificate_name = "localhost"
identity_file = "./twistedcaldav/test/data/server.pem"
certificate_file = "./twistedcaldav/test/data/cert.pem"

I am running on a macOS 10.14 system with Server.app 5.8 installed. It would be easy enough to point to my existing server certificates in /etc/certificates, but I do not wish to break my PKI or OD Master (although it appears that ./bin/keychain_init.py is coded not to overwrite existing certs).

A few pointers to get going in the right direction for running the server in deployment would be greatly appreciated.

[essandess]

Similarly, I've haven't yet done:

sudo python2 setup.py install clean

If that's that correct starting point, I can adjust all the necessary CalendarServer/Python environment/PostgreSQL/macOS configuration details as I go.

The objective is to deploy CalendarServer on macOS 10.14 running Server.app 5.8 with an OD Master, with a Python environment via either native macOS + pip or MacPorts, and my own postgres server configuration. I'm hoping that once it's configured correctly, there will be some launchd.plist that just starts it on boot. I'm also happy to see that this repo includes functions I used in Server.app like calendarserver_export (created after running ./bin/develop),

find . -name 'calendarserver_export'
./.develop/virtualenv/bin/calendarserver_export
./bin/calendarserver_export

and would like to get this part of the stack deployed as well.

[essandess]

Okay I see the python setup.py install in ./bin/package. And now also the rtfm in macOS Server Service Migration Guide, and this relevant issue: https://github.com/apple/ccs-calendarserver/issues/488. That should be all I need to make some progress.

Note to maintainers and others: the macOS Server Service Migration Guide says:

1. Use the “QuickStart” instructions on the main CalendarServer project page on Github to install, configure and run a calendar/contacts service with test users and data.

But the only QuickStart instructions in this repo point to README.rst, which is explicitly about development, not server deployment as mentioned above.

There appears to be sufficient deployment details in some combination of these links:

• doc/Admin/Guide.rst
• https://github.com/apple/ccs-calendarserver/pull/513
• https://github.com/apple/ccs-calendarserver/pull/489
• macOS Server Service Migration Guide

[essandess]

I also remain interested in the path forward now that Python 2.7 is EOL.

Migration to Python 3.7 and beyond shouldn't be too big of an issue.

I honestly looked at migrating to sabre/dav, inverse-inc/sogo, or Kozea/Radicale for a calDAV+cardDAV server. But AFAICT only apple/ccs-calendarserver and sabre/dav support iMIP, and I've been using apple/ccs-calendarserver's iMIP for years and know it works. And sabre/dav is PHP-based. So apple/ccs-calendarserver looks to be the main open source option of choice.

[essandess]

I've addressed the build issues in this PR: https://github.com/apple/ccs-calendarserver/pull/536