Open macosforgebot opened 10 years ago
macosforgebot
commented
10 years ago @wsanchez originally submitted this as comment:1:ticket:846
trunk's has been getting reworked. try now.
macosforgebot
commented
9 years ago nejeoui@… originally submitted this as comment:61:ticket:846
macosforgebot
commented
9 years ago nejeoui@… originally submitted this as comment:62:ticket:846
Trunk branche
root@ns3240020:/home/calendarServer/trunk# ./bin/run -n Using /home/calendarServer/trunk/.develop/virtualenv/bin/python as Python
Starting server... Traceback (most recent call last):
File "/usr/bin/twistd", line 14, in <module>
run()
File "/usr/lib/python2.7/dist-packages/twisted/scripts/twistd.py", line 27, in run
app.run(runApp, ServerOptions)
File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 637, in run
config.parseOptions()
File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 604, in parseOptions
usage.Options.parseOptions(self, options)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 261, in parseOptions
for (cmd, short, parser, doc) in self.subCommands:
File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 621, in subCommands
for plug in sorted(plugins, key=attrgetter('tapname')):
File "/usr/lib/python2.7/dist-packages/twisted/plugins/caldav.py", line 31, in getProperty
return getattr(reflect.namedClass(self.serviceMakerClass), propname)
File "/usr/lib/python2.7/dist-packages/twisted/python/_reflectpy3.py", line 151, in namedObject
module = namedModule('.'.join(classSplit[:-1]))
File "/usr/lib/python2.7/dist-packages/twisted/python/_reflectpy3.py", line 137, in namedModule
topLevel = import(name)
File "/home/calendarServer/trunk/calendarserver/tap/caldav.py", line 41, in <module>
from twistedcaldav.stdconfig import config
File "/home/calendarServer/trunk/twistedcaldav/init.py", line 36, in <module>
import twistedcaldav.caldavxml
File "/home/calendarServer/trunk/twistedcaldav/caldavxml.py", line 28, in <module>
from pycalendar.datetime import DateTime
ImportError: cannot import name DateTime
fabacab
commented
7 years ago This seems to still be an issue. When I build and install CalendarServer 9.0 or CalendarServer 9.1 from source, the following occurs:
$ caldavd -X -L
Traceback (most recent call last):
[…SNIPPED FOR LENGTH…]
File "/tmp/ccs-calendarserver/ccsenv/local/lib/python2.7/site-packages/CalendarServer-9.0-py2.7.egg/twistedcaldav/caldavxml.py", line 28, in <module>
from pycalendar.datetime import DateTime
ImportError: cannot import name DateTimeNote the ImportError is a direct reference to the pycalendar egg, which gets installed at version 2.0:
$ pwd
/tmp/ccs-calendarserver/ccsenv/lib/python2.7/site-packages
(ccsenv)pi@raspberrypi:/tmp/ccs-calendarserver/ccsenv/lib/python2.7/site-packages $ ls pycalendar*
pycalendar-2.0-py2.7.eggI can resolve this issue by manually installing pycalendar-2.1 like so:
wget https://github.com/apple/ccs-pycalendar/archive/master.zip
unzip master.zip
easy_install ccs-pycalendar-masterI can then remove the older version of pycalendar from my site-packages folder. However, this results in a new ImportError and still will not work:
$ caldavd -X -L
Traceback (most recent call last):
[…SNIPPED FOR LENGTH…]
File "/tmp/ccs-calendarserver/ccsenv/local/lib/python2.7/site-packages/CalendarServer-9.0-py2.7.egg/calendarserver/push/util.py", line 19, in <module>
from twistedcaldav.util import getPasswordFromKeychain, KeychainAccessError, \
File "/tmp/ccs-calendarserver/ccsenv/local/lib/python2.7/site-packages/CalendarServer-9.0-py2.7.egg/twistedcaldav/util.py", line 34, in <module>
from twext.internet.ssl import simpleClientContextFactory
ImportError: cannot import name simpleClientContextFactoryThis, too, can be resolved by manually downloading the current master branch of the Twisted extensions (twextpy) python package as of this writing from the Apple repositories, here on GitHub:
wget https://github.com/apple/ccs-twistedextensions/archive/master.zip
unzip master.zip
easy_install ccs-twistedextensions-master/Unfortunately, this, too, is not enough to have a functioning CalendarServer, as now running caldavd -X -L again gives me a Python TypeError:
File "/tmp/ccs-calendarserver/ccsenv/local/lib/python2.7/site-packages/CalendarServer-9.0-py2.7.egg/txdav/common/datastore/sql_tables.py", line 62, in _populateSchema
return SchemaSyntax(schemaFromPath(pathObj))
File "/tmp/ccs-calendarserver/ccsenv/local/lib/python2.7/site-packages/twextpy-0.1a1_unknown-py2.7.egg/twext/enterprise/dal/parseschema.py", line 112, in schemaFromPath
addSQLToSchema(schema, schemaData)
File "/tmp/ccs-calendarserver/ccsenv/local/lib/python2.7/site-packages/twextpy-0.1a1_unknown-py2.7.egg/twext/enterprise/dal/parseschema.py", line 150, in addSQLToSchema
while stmt.tokens and not significant(stmt.tokens[0]):
File "/tmp/ccs-calendarserver/ccsenv/local/lib/python2.7/site-packages/twextpy-0.1a1_unknown-py2.7.egg/twext/enterprise/dal/parseschema.py", line 724, in significant
return (not isinstance(token, Comment) and not token.is_whitespace())
TypeError: 'bool' object is not callableAt this point I gave up, because unless I'm badly misunderstanding something, it's clear that the dependency chain in the "release" code does not seem to actually be release quality?
dreness
commented
7 years ago @meitar Hi; I suspect the outcome you're seeing is at least somewhat related to environmental factors, as this problem does not occur on macOS and at least some linux distributions. Can you share any details about your system setup such as OS version, python version, and any other python-related packages or modules that have been installed in the same scope as CalendarServer?
Also, it would be useful to know the specific steps you took to get to this point (mostly because we don't really have any docs on this, so I can't assume what you might have done).
As a temporary and not-for-production-use test, it would also be useful to see what happens if you try to run CalendarServer out of the git checkout directory using the ./bin/run script instead of directly calling caldavd - basically, follow the QuickStart steps in ReadMe.rst. If things are OK in this mode, then we should probably look at the differences between python-related stuff in the source directory compared to whatever is happening at the system level as the result of whatever was done to install CalendarServer.
As this does look like the same issue that was originally reported, re-opening this ticket.
fabacab
commented
7 years ago @dreness I'm attempting this on a RaspberryPi:
$ uname -a
Linux raspberrypi 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/LinuxThe Raspbian repositories have a package for CalendarServer 3.x, but this is so old that I wanted to see if the issues with certain clients (Thunderbird/Lightning, Apple Calendar on macOS 10.11.x, etc.) that I was having were due to an old server. However, I could not get a running instance of the new version working. :(
Can you share any details about your system setup such as OS version, python version, and any other python-related packages or modules that have been installed in the same scope as CalendarServer?
Yes. The steps were as follows:
cd /tmp
git clone -b CalendarServer-9.0 https://github.com/apple/ccs-calendarserver.git # also attempted 9.1, but same issues
cd ccs-calendarserver
virtualenv ccsenv
source ccsenv/bin/activate
# Install source dependencies; I tried building numerous times, each time failing due to missing certain `.h` header files. But finally, I ended up with this line:
sudo apt install python-dev libffi-dev libkrb5-dev krb5-config libssl-dev
./setup.py build
./setup.py install
# This first install failed with an error about pyasn1 not being installed (even though it was installed).
# So I just ran the install again:
./setup.py install
# This second installation succeeded, so at this point I ran:
caldavd -X -LI have also just now tried to use run script using bin/run -n, but receive a different error:
Starting server...
Traceback (most recent call last):
File "/usr/bin/twistd", line 14, in <module>
run()
File "/usr/lib/python2.7/dist-packages/twisted/scripts/twistd.py", line 27, in run
app.run(runApp, ServerOptions)
File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 637, in run
config.parseOptions()
File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 604, in parseOptions
usage.Options.parseOptions(self, options)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 261, in parseOptions
for (cmd, short, parser, doc) in self.subCommands:
File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 621, in subCommands
for plug in sorted(plugins, key=attrgetter('tapname')):
File "/usr/lib/python2.7/dist-packages/twisted/plugins/caldav.py", line 29, in getProperty
return getattr(reflect.namedClass(self.serviceMakerClass), propname)
File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 168, in namedObject
module = namedModule('.'.join(classSplit[:-1]))
File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 154, in namedModule
topLevel = __import__(name)
ImportError: No module named carddavI understand that the -n option avoids installing dependencies, so I tried again after creating a virtualenv in my home folder:
cd /home/pi/ccs-calendarserver
virtualenv ccsenv
source ccsenv/bin/activate
bin/runThis produces an enormous amount of compilation output, as it builds Postgres and so on, but ultimately results in the pip error while installing SetupTools:
InstallationError: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.I realize that maybe using virtualenv for the bin/run development script on a RaspberryPi is not the intended usage here, but again, I wasn't originally trying to build for development purposes. I was trying to build for deployment purposes and, as you said, there is zero documentation about how to do that. Perhaps there could/should be…?
dreness
commented
7 years ago @meitar Yes, there definitely could / should be documentation.
It is a vague memory, but I think the error about inability to do a --user install is caused by too many virtualenvs. Note that ./bin/run and ./bin/develop already use a virtualenv, as an internal implementation detail (which is why virtualenv is a dependency). Maybe retry without manually entering a venv first, so just: check out, ./bin/develop, ./bin/run. You can simply nuke the .develop directory as the equivalent of a 'make clean'.
After ./bin/develop, you can poke at the CardDAV-y parts by using the ./bin/python script (more bootstrapping 'magic' happens here), and then manually loading and exercising some of the modules custom to CalendarServer. Also, looking at the filesystem paths of the modules might be helpful:
╭─ andre@zomg ~/work/ccs-calendarserver
╰─ $ ./bin/python
Python 2.7.10 (default, Jul 15 2017, 17:16:57)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import twistedcaldav
>>> c = twistedcaldav.Card()
>>> type(c)
<class 'pycalendar.vcard.card.Card'>
>>> c.getEndDelimiter()
'END:VCARD'
>>> twistedcaldav.__file__
'twistedcaldav/__init__.pyc'
>>> import pycalendar
>>> pycalendar.vcard.card.__file__
'/Users/andre/work/ccs-calendarserver/.develop/virtualenv/src/pycalendar/src/pycalendar/vcard/card.pyc'If this works in the source directory, you can retry the same steps from outside the source directory, using the same python executable that caldavd is trying to use, instead of the ./bin/python script in the source dir.
CalendarServer uses a somewhat intricate network of patches, extensions, and modules, and tying it all together is challenging. If you can get it working in the git sandbox, you might try using ./bin/package
Sorry this is all so opaque / frustrating. If it's any consolation, this is by far the hardest part of using CalendarServer.
akanouras
commented
7 years ago Hello, in case it helps someone, the following should get CalendarServer-9.1's caldavd running in a Debian Stretch amd64 environment:
apt install libkrb5-dev python-pip python-virtualenv python-zope.interface python-setuptools python-cffi python-pycparser python-ldap python-six python-pg8000 python-dateutil python-psutil python-setproctitle python-xattr python-twisted python-service-identity python-characteristic python-pyasn1 python-pyasn1-modules python-crypto python-openssl python-cryptography python-idna python-enum34 python-ipaddress python-incremental python-constantlycd /optgit clone -b CalendarServer-9.1 https://github.com/apple/ccs-calendarserver calendarservercd calendarservervirtualenv --system-site-packages --no-setuptools --no-wheel .venv. .venv/bin/activate pip install pytz sqlparse==0.2.0grep -h editable requirements-cs.txt requirements-dev.txt | while read REPLY; do eval "pip install --no-deps $REPLY"; donecaldavd -X -L
Invalid configuration: Config file /etc/caldavd/caldavd.plist not found. Exiting.For deployment, /usr/bin/env PYTHON=/opt/calendarserver/.venv/bin/python /opt/calendarserver/.venv/bin/caldavd <args> is the command you want to use.
Don't forget to use a separate user for caldavd before deploying it!
While first trying to install the package, I tried to use sqlparse 0.2.2 from Debian and got the TypeError: 'bool' object is not callable mentioned above. pip install sqlparse==0.2.0 fixed it.
Luckily Debian Stretch's packaged Python modules are the same version or minor upgrades of CalendarServer-9.1's requirements, apart from sqlparse.
I will try to update this comment once I have done some more testing, hopefully with a link to a ready to install package.
vaab
commented
6 years ago @akanouras Thanks for your help on this. On my try (Docker using debian:stretch image...) caldavd still tries to compile things on its own when executed and thus required build-essential, python-dev... before getting to the same point than you reached.
_joyghosh001@… originally submitted this as ticket:846
I'm unable to run the server. I try to execute the following command and the following is what I get. Any help is appreciated.
sudo ./bin/run -n Using /Users/joyghosh/CalendarServer/trunk/.develop/roots/py_modules/bin/python as Python
Starting server... Traceback (most recent call last):
ImportError: cannot import name DateTime launchctl bsexec failed: No such file or directory