search

apple / ccs-calendarserver

The Calendar and Contacts Server.
https://www.calendarserver.org
Apache License 2.0
486 stars 136 forks source link

problems installing on centos7 #471

Open f1-outsourcing opened 7 years ago

f1-outsourcing commented 7 years ago

Obviously i would like to stay as close a possible to installing packages default supplied by rh / centos.

What am i doing wrong? Where is an install howto for rhel/centos? Why not an rpm repository?

Preparing Python requirements

Obtaining file:///usr/local/calendarserver (from -r /usr/local/calendarserver/requirements-cs.txt (line 1)) Obtaining twextpy from git+https://github.com/apple/ccs-twistedextensions.git@a8528bca42bd3529b5186bb6a979da4a764baf07#egg=twextpy (from -r /usr/local/calendarserver/requirements-cs.txt (line 2)) Updating ./.develop/virtualenv/src/twextpy clone (to a8528bca42bd3529b5186bb6a979da4a764baf07) Could not find a tag or branch 'a8528bca42bd3529b5186bb6a979da4a764baf07', assuming commit. Complete output from command python setup.py egg_info: Traceback (most recent call last): File "", line 20, in File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 331, in doSetup() File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 288, in doSetup version_string = version() File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 139, in version info = svn_info(source_root) File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 63, in svn_info stderr=subprocess.STDOUT, File "/usr/lib64/python2.7/subprocess.py", line 568, in check_output process = Popen(stdout=PIPE, *popenargs, **kwargs) File "/usr/lib64/python2.7/subprocess.py", line 711, in init errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 13] Permission denied

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /usr/local/calendarserver/.develop/virtualenv/src/twextpy

yum install python2-pip gcc bzip2 openssl-devel readline-devel zlib-devel flex python-devel python-ldap python2-pg8000 python-zope-interface python-pycparser python-cffi python-dateutil pytz python-psutil python-setproctitle python-virtualenv

pip install twextpy

useradd caldav

mkdir /usr/local/calendarserver/
chown caldav.caldav /usr/local/calendarserver/

cd /usr/local/
su caldav

git clone -b CalendarServer-9.0 https://github.com/apple/ccs-calendarserver.git calendarserver

cd /usr/local/calendarserver/

bin/develop
dreness commented 7 years ago

Hi,

It's quite likely that some part of CalendarServer, the python distribution plumbing, or the 'autoconf' style plumbing used by C deps is confused that you are putting the SVN checkout in /usr/local, given that this all ends with a 'permission denied' error. The ./bin/develop script is definitely intended to bootstrap a development sandbox, but not to 'install' anything. Get it working in the sandbox first (i.e. in a user's home directory, running as that user), then deal with role accounts and installation later.

-dre

On Feb 8, 2017, at 1:31 PM, F1 Outsourcing Development notifications@github.com wrote:

Obviously i would like to stay as close a possible to installing packages default supplied by rh / centos.

What am i doing wrong? Where is an install howto for rhel/centos? Why not an rpm repository?

Preparing Python requirements

Obtaining file:///usr/local/calendarserver (from -r /usr/local/calendarserver/requirements-cs.txt (line 1)) Obtaining twextpy from git+https://github.com/apple/ccs-twistedextensions.git@a8528bca42bd3529b5186bb6a979da4a764baf07#egg=twextpy https://github.com/apple/ccs-twistedextensions.git@a8528bca42bd3529b5186bb6a979da4a764baf07#egg=twextpy (from -r /usr/local/calendarserver/requirements-cs.txt (line 2)) Updating ./.develop/virtualenv/src/twextpy clone (to a8528bca42bd3529b5186bb6a979da4a764baf07) Could not find a tag or branch 'a8528bca42bd3529b5186bb6a979da4a764baf07', assuming commit. Complete output from command python setup.py egg_info: Traceback (most recent call last): File "", line 20, in File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 331, in doSetup() File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 288, in doSetup version_string = version() File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 139, in version info = svn_info(source_root) File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 63, in svn_info stderr=subprocess.STDOUT, File "/usr/lib64/python2.7/subprocess.py", line 568, in check_output process = Popen(stdout=PIPE, *popenargs, **kwargs) File "/usr/lib64/python2.7/subprocess.py", line 711, in init errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 13] Permission denied

Command "python setup.py egg_info" failed with error code 1 in /usr/local/calendarserver/.develop/virtualenv/src/twextpy

yum install python2-pip gcc bzip2 openssl-devel readline-devel zlib-devel flex python-devel python-ldap python2-pg8000 python-zope-interface python-pycparser python-cffi python-dateutil pytz python-psutil python-setproctitle python-virtualenv

pip install twextpy

useradd caldav

mkdir /usr/local/calendarserver/ chown caldav.caldav /usr/local/calendarserver/

cd /usr/local/ su caldav

git clone -b CalendarServer-9.0 https://github.com/apple/ccs-calendarserver.git calendarserver

cd /usr/local/calendarserver/

bin/develop — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/apple/ccs-calendarserver/issues/471, or mute the thread https://github.com/notifications/unsubscribe-auth/AE_8oBr4333Y22_QzavsGqqnljCxUjuBks5rajQxgaJpZM4L7Y4i.

dreness commented 7 years ago

Of course I meant git, not svn :)

-dre

On Feb 9, 2017, at 11:31 AM, Andre LaBranche dre@mac.com wrote:

Hi,

It's quite likely that some part of CalendarServer, the python distribution plumbing, or the 'autoconf' style plumbing used by C deps is confused that you are putting the SVN checkout in /usr/local, given that this all ends with a 'permission denied' error. The ./bin/develop script is definitely intended to bootstrap a development sandbox, but not to 'install' anything. Get it working in the sandbox first (i.e. in a user's home directory, running as that user), then deal with role accounts and installation later.

-dre

On Feb 8, 2017, at 1:31 PM, F1 Outsourcing Development <notifications@github.com mailto:notifications@github.com> wrote:

Obviously i would like to stay as close a possible to installing packages default supplied by rh / centos.

What am i doing wrong? Where is an install howto for rhel/centos? Why not an rpm repository?

Preparing Python requirements

Obtaining file:///usr/local/calendarserver file:///usr/local/calendarserver (from -r /usr/local/calendarserver/requirements-cs.txt (line 1)) Obtaining twextpy from git+https://github.com/apple/ccs-twistedextensions.git@a8528bca42bd3529b5186bb6a979da4a764baf07#egg=twextpy https://github.com/apple/ccs-twistedextensions.git@a8528bca42bd3529b5186bb6a979da4a764baf07#egg=twextpy (from -r /usr/local/calendarserver/requirements-cs.txt (line 2)) Updating ./.develop/virtualenv/src/twextpy clone (to a8528bca42bd3529b5186bb6a979da4a764baf07) Could not find a tag or branch 'a8528bca42bd3529b5186bb6a979da4a764baf07', assuming commit. Complete output from command python setup.py egg_info: Traceback (most recent call last): File "", line 20, in File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 331, in doSetup() File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 288, in doSetup version_string = version() File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 139, in version info = svn_info(source_root) File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 63, in svn_info stderr=subprocess.STDOUT, File "/usr/lib64/python2.7/subprocess.py", line 568, in check_output process = Popen(stdout=PIPE, *popenargs, **kwargs) File "/usr/lib64/python2.7/subprocess.py", line 711, in init errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 13] Permission denied

Command "python setup.py egg_info" failed with error code 1 in /usr/local/calendarserver/.develop/virtualenv/src/twextpy

yum install python2-pip gcc bzip2 openssl-devel readline-devel zlib-devel flex python-devel python-ldap python2-pg8000 python-zope-interface python-pycparser python-cffi python-dateutil pytz python-psutil python-setproctitle python-virtualenv

pip install twextpy

useradd caldav

mkdir /usr/local/calendarserver/ chown caldav.caldav /usr/local/calendarserver/

cd /usr/local/ su caldav

git clone -b CalendarServer-9.0 https://github.com/apple/ccs-calendarserver.git https://github.com/apple/ccs-calendarserver.git calendarserver

cd /usr/local/calendarserver/

bin/develop — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/apple/ccs-calendarserver/issues/471, or mute the thread https://github.com/notifications/unsubscribe-auth/AE_8oBr4333Y22_QzavsGqqnljCxUjuBks5rajQxgaJpZM4L7Y4i.

f1-outsourcing commented 7 years ago

this virtualenv / running under an unprivileged user installation is not working. If i run the install under root installs the python modules in the default location and then it is working.

I suggest to setup a vm with centos/rhel and investigate this.

dreness commented 7 years ago

Hello @f1-outsourcing

When running from a virtualenv, I found that something like this patch is needed to allow ./bin/develop to succeed. That change is part of the travis-ci integration PR that is not yet merged to master, although this change isn't specific to travis.

CalendarServer can run from any install location as long as PATH and PYTHONPATH are aware of that location. The same goes for any of CalendarServer's dependencies, with the addition of a small but platform-dependent cluster of environment vars: C_INCLUDE_PATH, LD_LIBRARY_PATH, CPPFLAGS, LDFLAGS, DYLD_LIBRARY_PATH - see https://github.com/apple/ccs-calendarserver/blob/master/bin/package#L154. These aren't all needed by any one platform, and some are used only at compile time, but the distinction between compile-time and run-time is often abstracted away, so we set them all in the environment.sh produced by ./bin/package.sh

The reason it worked for you in /usr/local is that this directory is typically included by default in all of the search paths configured by the above environment variables. The reason you needed to be root is likely that /usr/local and its descendants are only writable by root, by default. The ./bin/develop script doesn't expect to be run from anyplace where it doesn't have write access to the filesystem.

... and while trying to pin down exactly where it might be trying to write, other than /usr/local/calendarserver, I spotted a smoking gun of a different sort, from the traceback you posted:

File "/usr/local/calendarserver/.develop/virtualenv/src/twextpy/setup.py", line 63, in svn_info

We switched from svn to git several months back, and the string "svn_info" does not appear in the current version of twextpy (aka 'ccs-twistedextensions') - the svn_info function was replaced with git_info in this change. I guess this indicates the version you got from pip is not the most recent version... Anyway, I'm not exactly sure why this would manifest as a 'permissions denied' error - perhaps something was trying to execute one of the http redirect responses returned by svn.calendarserver.org, or maybe it's that the 'editable' flag is present on twextpy's entry in the requirements file, which maybe implies writability to the current twextpy install location. In any case, you always need the version of twextpy referenced by CalendarServer's requirements files - in this case, requirements.cs - that's the copy on master, but for a release we reference a specific version.

f1-outsourcing commented 7 years ago

Hmmmm, but if I do a

git clone -b CalendarServer-9.0 https://github.com/apple/ccs-calendarserver.git calendarserver

I expect to checkout a stable (working) version. That does not need patching or a lot of reading.

dreness commented 7 years ago

@f1-outsourcing that is a reasonable expectation. Unfortunately, even for our most supported platform targets (macOS and Oracle Linux), there is still at least some setup required prior to running ./bin/develop (such as installing pip / virtualenv if needed).

In this case, the ./bin/develop script will get twextpy for you if it's not already present on your system. You have manually installed a version of twextpy that's older than the one specified by CalendarServer, so remove that and re-run ./bin/develop.

Once ./bin/develop has completed successfully, you should be able to use ./bin/package to produce an installation dstroot. This dstroot includes an environment.sh file, which should be sourced by the shell of the user that will run caldavd.