Closed mycroft closed 4 years ago
Won't this break twisted? https://twistedmatrix.com/documents/8.2.0/api/twisted.internet.reactor.html
Harold, obviously, my goal is to fix it, not to break it. Upgrading cassandra-driver to latest current version won't make bg-carbon-aggregator-cache and its peer to start anymore:
# /opt/graphite/pypy/bin/python /opt/graphite/pypy/bin/bg-carbon-aggregator-cache --debug --nodaemon start
Traceback (most recent call last):
File "/opt/graphite/pypy/bin/bg-carbon-aggregator-cache", line 12, in <module>
sys.exit(main())
File "/opt/graphite/pypy/site-packages/biggraphite/cli/bg_carbon_aggregator_cache.py", line 48, in main
carbon_util.run_twistd_plugin("carbon-aggregator-cache")
File "/opt/graphite/pypy/site-packages/carbon/util.py", line 138, in run_twistd_plugin
config.parseOptions(twistd_options)
File "/opt/graphite/pypy/site-packages/twisted/application/app.py", line 637, in parseOptions
usage.Options.parseOptions(self, options)
File "/opt/graphite/pypy/site-packages/twisted/python/usage.py", line 255, in parseOptions
self._dispatch[optMangled](optMangled, arg)
File "/opt/graphite/pypy/site-packages/twisted/python/usage.py", line 411, in <lambda>
fn = lambda name, value, m=method: m(value)
File "/opt/graphite/pypy/site-packages/twisted/application/app.py", line 549, in opt_reactor
raise usage.UsageError(msg)
UsageError: The specified reactor cannot be used, failed with error: stop.
See the list of available reactors with --help-reactors
The problem is that, for some reason, cassandra_driver will now load twisted reactor modules (cassandra_driver is load by biggraphite even before we start it), and loading twisted reactor will install it, even before we load the configuration. See this trace with a custom Exception I had to use to retrieve the stack trace of the initial installation:
# /opt/graphite/pypy/bin/python /opt/graphite/pypy/bin/bg-carbon-aggregator-cache --debug --nodaemon start
Traceback (most recent call last):
File "/opt/graphite/pypy/bin/bg-carbon-aggregator-cache", line 12, in <module>
sys.exit(main())
File "/opt/graphite/pypy/site-packages/biggraphite/cli/bg_carbon_aggregator_cache.py", line 34, in main
from biggraphite.plugins import carbon as unused_carbon # noqa
File "/opt/graphite/pypy/site-packages/biggraphite/plugins/carbon.py", line 38, in <module>
from biggraphite import settings as bg_settings
File "/opt/graphite/pypy/site-packages/biggraphite/settings.py", line 19, in <module>
from biggraphite import accessor_factory as bg_accessor_factory
File "/opt/graphite/pypy/site-packages/biggraphite/accessor_factory.py", line 17, in <module>
from biggraphite.drivers import cassandra as bg_cassandra
File "/opt/graphite/pypy/site-packages/biggraphite/drivers/cassandra.py", line 34, in <module>
from cassandra import cluster as c_cluster
File "/opt/graphite/pypy/site-packages/cassandra/cluster.py", line 94, in <module>
from cassandra.io.twistedreactor import TwistedConnection
File "/opt/graphite/pypy/site-packages/cassandra/io/twistedreactor.py", line 25, in <module>
from twisted.internet import reactor, protocol
File "/opt/graphite/pypy/site-packages/twisted/internet/reactor.py", line 39, in <module>
default.install()
File "/opt/graphite/pypy/site-packages/twisted/internet/epollreactor.py", line 245, in install
p = EPollReactor()
File "/opt/graphite/pypy/site-packages/twisted/internet/epollreactor.py", line 66, in __init__
raise Exception('stop')
The fix I'm proposing is to load biggraphite, which will load cassandra then firstly install the reactor module, and if before we start ours it exists in the loaded modules, to unregister it. It seems like a common practice, but if you want me to double check, I'll take a deeper look. However, I've tested this successfully on our testing environment (it connects & runs well against network & cassandra).
The recent cassandra_driver update triggered a reactor installation before starting to parse biggraphite configuration. This is triggering a failure because biggraphite can't initialize its own reactor, making the bg daemon fails.
Also, bumping to 0.14.20.