graphite-project / graphite-web

A highly scalable real-time graphing system
http://graphite.readthedocs.org/
Apache License 2.0
5.89k stars 1.26k forks source link

Metrics not browsable in web interface #2237

Closed rwyse closed 6 years ago

rwyse commented 6 years ago

So I am beating my head against the wall here. I have been at this for weeks trying to figure out what in the world is wrong or that I am missing. I will try to break this out as tersely as possible.

I am attaching my carbon.conf file and my local_settings.py which is slightly sanitized for this. I am using defaults for pretty much everything else based on what is out in the git repos for this deploy. Details:

Platform - RHEL 7.4 Django - 1.9 Twisted - 16.4.1 Whisper/Carbon/Graphite-Web - 1.1.2 Apache - 2.4.6-67.el7_4.5 Python - 2.7.5

I have ensured that all dependencies are installed for this and walked through the docs on the wiki with using a combination of installing from source and pip to get it installed. carbon-1.2.txt

local_settings-1.2.txt

Once all this is done, I have ensured that /opt/graphite is 755 and both web app and storage underneath recursively are owned by apache:apache. I have run the following as well:

cd /opt/graphite/webapp/graphite PYTHONPATH=/opt/graphite/webapp django-admin.py migrate --settings=graphite.settings --run-syncdb && chown apache:apache /opt/graphite/storage/graphite.db

All directories in the site-packages directory are mode 755 and all python files mode 644 while the whisper files are 644 and owned by apache.

Initially I had issues with the caches eventually returning timeout errors in the console logs, but I "think" changing the GRAPHITE_URL to point to http://localhost has resolved that issue. Time will tell.

Carbon will start up just fine and seems to run. There is data stored down in the /opt/graphite/storage/whisper directories with a bunch of metrics that I can interact with whisper commands from the CLI with no issues (whisper-info, whisper-fetch, etc).

However, I am now facing two issues that I am struggling to find the cause of. The first issue is that with no changes being made, all of a sudden apache is spamming the following in the error logs and web isn't working:

[Fri Feb 16 16:15:53.437449 2018] [:error] [pid 1015] ImportError: No module named datalib [Fri Feb 16 16:15:53.666847 2018] [:error] [pid 1016] Internal Server Error: /render/ [Fri Feb 16 16:15:53.666890 2018] [:error] [pid 1016] Traceback (most recent call last): [Fri Feb 16 16:15:53.666900 2018] [:error] [pid 1016] File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 134, in get_response [Fri Feb 16 16:15:53.666904 2018] [:error] [pid 1016] resolver_match = resolver.resolve(request.path_info) [Fri Feb 16 16:15:53.666906 2018] [:error] [pid 1016] File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 374, in resolve [Fri Feb 16 16:15:53.666909 2018] [:error] [pid 1016] for pattern in self.url_patterns: [Fri Feb 16 16:15:53.666911 2018] [:error] [pid 1016] File "/usr/lib/python2.7/site-packages/django/utils/functional.py", line 33, in get [Fri Feb 16 16:15:53.666914 2018] [:error] [pid 1016] res = instance.dict[self.name] = self.func(instance) [Fri Feb 16 16:15:53.666916 2018] [:error] [pid 1016] File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 417, in url_patterns [Fri Feb 16 16:15:53.666919 2018] [:error] [pid 1016] patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module) [Fri Feb 16 16:15:53.666921 2018] [:error] [pid 1016] File "/usr/lib/python2.7/site-packages/django/utils/functional.py", line 33, in get [Fri Feb 16 16:15:53.666924 2018] [:error] [pid 1016] res = instance.dict[self.name] = self.func(instance) [Fri Feb 16 16:15:53.666926 2018] [:error] [pid 1016] File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 410, in urlconf_module [Fri Feb 16 16:15:53.666929 2018] [:error] [pid 1016] return import_module(self.urlconf_name) [Fri Feb 16 16:15:53.666931 2018] [:error] [pid 1016] File "/usr/lib64/python2.7/importlib/init.py", line 37, in import_module [Fri Feb 16 16:15:53.666934 2018] [:error] [pid 1016] import(name) [Fri Feb 16 16:15:53.666936 2018] [:error] [pid 1016] File "/opt/graphite/webapp/graphite/urls.py", line 22, in [Fri Feb 16 16:15:53.666938 2018] [:error] [pid 1016] url('^render', include('graphite.render.urls')), [Fri Feb 16 16:15:53.666941 2018] [:error] [pid 1016] File "/usr/lib/python2.7/site-packages/django/conf/urls/init.py", line 52, in include [Fri Feb 16 16:15:53.666943 2018] [:error] [pid 1016] urlconf_module = import_module(urlconf_module) [Fri Feb 16 16:15:53.666946 2018] [:error] [pid 1016] File "/usr/lib64/python2.7/importlib/init.py", line 37, in import_module [Fri Feb 16 16:15:53.666948 2018] [:error] [pid 1016] import(name) [Fri Feb 16 16:15:53.666950 2018] [:error] [pid 1016] File "/opt/graphite/webapp/graphite/render/urls.py", line 16, in [Fri Feb 16 16:15:53.666953 2018] [:error] [pid 1016] from . import views [Fri Feb 16 16:15:53.666955 2018] [:error] [pid 1016] File "/opt/graphite/webapp/graphite/render/views.py", line 30, in [Fri Feb 16 16:15:53.666957 2018] [:error] [pid 1016] from graphite.render.evaluator import evaluateTarget [Fri Feb 16 16:15:53.666960 2018] [:error] [pid 1016] File "/opt/graphite/webapp/graphite/render/evaluator.py", line 7, in [Fri Feb 16 16:15:53.666962 2018] [:error] [pid 1016] from graphite.render.datalib import fetchData, TimeSeries, prefetchData

Issue 2: even when that wasn't happening, I could get the web interface but when I clicked on metrics to expand the tree in the web interface it would sit and then think for a moment and then return nothing. Just blank and no folders as I can see on the underlying filesystem or that I am able to browse on a system that I am running the .10 code on and trying to update to the latest releases.

At this point I have tried monkeying around with the GRAPHITE_URL setting and just about anything else to attempt to get the metrics viewable. At one point they were working absolutely fine. Then overnight it just disappeared and went back to the behavior that I am seeing now.

For grins I made sure that I am using the proper .py files that are out in git to ensure that there wasn't something that I was missing, and I feel like this is a config issue that I am possibly missing but just can't get my head around. I also have another instance running just graphite-web that has an identical local settings config with the exception of adding in the cluster servers values to point to this node and it is working just fine.

I have also used netstat and telnet to confirm that the connections are listening on the ports and can be connected to. Again, I am running out of ideas as to what in the world it is that is happening here and feel like I am going a bit insane trying to get it to do what I think it should, so any help someone can provide would be great.

Thanks in advance for any guidance or direction on this and please let me know if there is any other information that I can provide to assist in figuring this one out.

-Ryan

deniszh commented 6 years ago

@rwyse - could you please show your apache config? Did it base on https://github.com/graphite-project/graphite-web/blob/master/examples/example-graphite-vhost.conf ?

rwyse commented 6 years ago

@deniszh, yes it is based off of that example. I am including it attached now. graphite-vhost.txt

deniszh commented 6 years ago

@rwyse, Could you please show me `opt/graphite/conf/graphite.wsgi'? Does it looks like https://github.com/graphite-project/graphite-web/blob/master/conf/graphite.wsgi.example ?

rwyse commented 6 years ago

@deniszh it is exactly that. As I said, I tried to leave things as close to what is documented in the wiki to ensure that this would function as expected. I am attaching the file though. graphite-wsgi.txt

For what it is worth, I can run a python shell and run the commands just fine as well. Also, I can run the syncdb commands with Django-admin as well. Let me know if there is any other information or files I can provide and thanks for looking into this.

deniszh commented 6 years ago

@rwyse Well. Something is really wrong with your system, I'm just trying to find what exactly. How do you install graphite-web - using pip install? did you use virtualenv? Please show output of ls -al /opt/graphite/webapp/graphite/render/

rwyse commented 6 years ago

@deniszh I am installing graphite by grabbing the tarball, extracting it into a temp location, running a pip install on requirements.txt and then once those are done running a python setup.py install. It is being done with chef and all I did was grab the new tar balls from the git repo and then using them in the cookbooks that have been running and working without issue for a long time now. The output requested is below:

ls -al /opt/graphite/webapp/graphite/render/ total 524 drwxr-xr-x 2 apache apache 4096 Feb 16 16:23 . drwxr-xr-x 18 apache apache 4096 Feb 16 16:06 .. -rw-r--r-- 1 apache apache 5530 Jan 9 07:34 attime.py -rw-r--r-- 1 apache apache 5967 Feb 5 23:30 attime.pyc -rw------- 1 root root 9936 Feb 13 16:39 datalib.py -rw------- 1 root root 8620 Feb 16 15:01 datalib.pyc -rw-r--r-- 1 apache apache 5893 Jan 9 07:34 evaluator.py -rw-r--r-- 1 apache apache 4615 Feb 5 23:30 evaluator.pyc -rw-r--r-- 1 apache apache 125506 Feb 13 17:00 functions.py -rw-r--r-- 1 apache apache 124346 Feb 13 17:01 functions.pyc -rw-r--r-- 1 apache apache 72586 Jan 9 07:34 glyph.py -rw-r--r-- 1 apache apache 59462 Feb 5 23:30 glyph.pyc -rw-r--r-- 1 apache apache 3547 Jan 9 07:34 grammar.py -rw-r--r-- 1 apache apache 3753 Feb 5 23:30 grammar.pyc -rw-r--r-- 1 apache apache 4722 Jan 9 07:34 hashing.py -rw-r--r-- 1 apache apache 6073 Feb 5 23:30 hashing.pyc -rw-r--r-- 1 apache apache 0 Jan 9 07:34 init.py -rw-r--r-- 1 apache apache 135 Feb 5 23:30 init.pyc -rw-r--r-- 1 apache apache 871 Jan 9 07:34 urls.py -rw-r--r-- 1 apache apache 1101 Feb 5 23:30 urls.pyc -rw-r--r-- 1 apache apache 21430 Jan 9 07:34 views.py -rw-r--r-- 1 apache apache 18569 Feb 5 23:30 views.pyc

Thanks again and looking forward to trying to figure out what the heck is going on with this.

Ryan

rwyse commented 6 years ago

Alright so after seeing that I chown'd the web app dir correctly and that got the data lib error resolved although I am not entirely certain how the permissions on those files would have changed anyhow considering chef sets them to owned by apache and I have done it many times. Regardless here is the new output:

ls -al /opt/graphite/webapp/graphite/render/ total 524 drwxr-xr-x 2 apache apache 4096 Feb 16 16:23 . drwxr-xr-x 18 apache apache 4096 Feb 16 16:06 .. -rw-r--r-- 1 apache apache 5530 Jan 9 07:34 attime.py -rw-r--r-- 1 apache apache 5967 Feb 5 23:30 attime.pyc -rw------- 1 apache apache 9936 Feb 13 16:39 datalib.py -rw------- 1 apache apache 8620 Feb 16 15:01 datalib.pyc -rw-r--r-- 1 apache apache 5893 Jan 9 07:34 evaluator.py -rw-r--r-- 1 apache apache 4615 Feb 5 23:30 evaluator.pyc -rw-r--r-- 1 apache apache 125506 Feb 13 17:00 functions.py -rw-r--r-- 1 apache apache 124346 Feb 13 17:01 functions.pyc -rw-r--r-- 1 apache apache 72586 Jan 9 07:34 glyph.py -rw-r--r-- 1 apache apache 59462 Feb 5 23:30 glyph.pyc -rw-r--r-- 1 apache apache 3547 Jan 9 07:34 grammar.py -rw-r--r-- 1 apache apache 3753 Feb 5 23:30 grammar.pyc -rw-r--r-- 1 apache apache 4722 Jan 9 07:34 hashing.py -rw-r--r-- 1 apache apache 6073 Feb 5 23:30 hashing.pyc -rw-r--r-- 1 apache apache 0 Jan 9 07:34 init.py -rw-r--r-- 1 apache apache 135 Feb 5 23:30 init.pyc -rw-r--r-- 1 apache apache 871 Jan 9 07:34 urls.py -rw-r--r-- 1 apache apache 1101 Feb 5 23:30 urls.pyc -rw-r--r-- 1 apache apache 21430 Jan 9 07:34 views.py -rw-r--r-- 1 apache apache 18569 Feb 5 23:30 views.pyc

Web interface is working properly however I am still not able to expand metrics and see any thing in the tree. Thanks.

Ryan

deniszh commented 6 years ago

Graphite's file permissions are quite simple - files under /opt/graphite/storage/whisper should be writable by carbon process, and readable by apache process. If you using sqlite database - it should be readable and writable by apache process. Simplest solution - run carbon process with apache permissions too. Files under /opt/graphite/webapp/ can be only readable by apache process, write permission is not mandatory.

Web interface is working properly however I am still not able to expand metrics and see any thing in the tree.

What's in the error logs now?

rwyse commented 6 years ago

@deniszh So the apache logs are looking a little better and the previous error is gone; however in the carbon cache logs I am getting the following repeatedly:

18/02/2018 17:39:33 :: Unhandled error in Deferred: 18/02/2018 17:39:33 :: Unhandled Error Traceback (most recent call last): Failure: twisted.internet.error.TimeoutError: User timeout caused connection failure.

18/02/2018 17:45:33 :: Unhandled error in Deferred: 18/02/2018 17:45:33 :: Unhandled Error Traceback (most recent call last): Failure: twisted.internet.error.TimeoutError: User timeout caused connection failure.

Along with that I am seeing that the relays have all stopped running at this point as well. The only thing that I am seeing in the logs for the relays is the following (I am leaving out a lot due to letting you assume it is the same for any connection from relay to cache):

17/02/2018 11:42:25 :: CarbonClientFactory(127.0.0.1:2214:k)::startedConnecting (127.0.0.1:2214) 17/02/2018 11:42:25 :: CarbonClientFactory(127.0.0.1:2504:e)::startedConnecting (127.0.0.1:2504) 17/02/2018 11:42:27 :: CarbonClientFactory(127.0.0.1:2204:b)::startedConnecting (127.0.0.1:2204) 17/02/2018 11:42:27 :: CarbonClientFactory(127.0.0.1:2804:h)::startedConnecting (127.0.0.1:2804) 17/02/2018 11:42:29 :: CarbonClientFactory(127.0.0.1:2214:k)::clientConnectionFailed (127.0.0.1:2214) Connection was refused by other side: 111: Connection refused. 17/02/2018 11:42:29 :: Destination is down: 127.0.0.1:2214:k (103/5) 17/02/2018 11:42:29 :: CarbonClientFactory(127.0.0.1:2504:e)::clientConnectionFailed (127.0.0.1:2504) Connection was refused by other side: 111: Connection refused. 17/02/2018 11:42:29 :: Destination is down: 127.0.0.1:2504:e (188/5) 17/02/2018 11:42:29 :: CarbonClientFactory(127.0.0.1:2204:b)::clientConnectionFailed (127.0.0.1:2204) Connection was refused by other side: 111: Connection refused. 17/02/2018 11:42:29 :: Destination is down: 127.0.0.1:2204:b (282/5) 17/02/2018 11:42:29 :: CarbonClientFactory(127.0.0.1:2804:h)::clientConnectionFailed (127.0.0.1:2804) Connection was refused by other side: 111: Connection refused. 17/02/2018 11:42:29 :: Destination is down: 127.0.0.1:2804:h (680/5) 17/02/2018 11:42:34 :: CarbonClientFactory(127.0.0.1:2214:k)::startedConnecting (127.0.0.1:2214)

However if I stop carbon and start it up, I get connections establishing just fine....:

16/02/2018 16:53:36 :: CarbonClientProtocol(127.0.0.1:2224:l)::connectionMade 16/02/2018 16:53:36 :: CarbonClientFactory(127.0.0.1:2224:l)::connectionMade (CarbonClientProtocol(127.0.0.1:2224:l)) 16/02/2018 16:53:36 :: Destination is up: 127.0.0.1:2224:l 16/02/2018 16:53:36 :: CarbonClientProtocol(127.0.0.1:2404:d)::connectionMade 16/02/2018 16:53:36 :: CarbonClientFactory(127.0.0.1:2404:d)::connectionMade (CarbonClientProtocol(127.0.0.1:2404:d)) 16/02/2018 16:53:36 :: Destination is up: 127.0.0.1:2404:d

The only thing that is throwing errors in the apache logs is in the webapp/exception.log I am seeing the following, but I suspect it is unrelated to the issue:

2018-02-18,17:57:24.018 :: Exception encountered in <GET http://10.1.10.30/render/?format=pickle&local=1&noCache=1&from=1518911843&until=1518998243&target=browse.collectd.mexico-web.i-%2A.load.load.midterm&target=browse.collectd.mexico-web.i-%2A.load.load.shortterm&target=browse.collectd.mexico-web.i-%2A.load.load.longterm&now=1518998243> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 147, in get_response response = wrapped_callback(request, *callback_args, *callback_kwargs) File "/opt/graphite/webapp/graphite/render/views.py", line 117, in renderView data.extend(evaluateTarget(requestContext, targets)) File "/opt/graphite/webapp/graphite/render/evaluator.py", line 15, in evaluateTarget prefetchData(requestContext, pathExpressions) File "/opt/graphite/webapp/graphite/render/datalib.py", line 253, in prefetchData for result in STORE.fetch(pathExpressions, startTime, endTime, now, requestContext): File "/opt/graphite/webapp/graphite/storage.py", line 179, in fetch 'fetch for %s' % str(patterns)) File "/opt/graphite/webapp/graphite/storage.py", line 130, in wait_jobs '\n'.join(traceback.format_exception(job.exception_info)) AttributeError: 'Job' object has no attribute 'exception_info'

This is also mirrored in the apache error log itself, but the web UI is up and I am able to interact with it. I am not sure how those permissions would have changed considering that at the very least chef should ensure that the owner and mode are set correctly as it has done for the past few years we have run this....Regardless, I still cannot expand the metrics tree in the browser and see any data....

Thanks again for the assist and let me know what the next steps are...If it helps, I have rebuilt from scratch and reproduce the issue every time so I suspect either there is a config that is incorrect that I am not understanding properly or something entirely different that I am missing altogether.

-Ryan

deniszh commented 6 years ago

@rwyse: Sorry, but it looks like your installation is severely broken, maybe recreating from scratch is a good idea. E.g. I'm very puzzled how you can get AttributeError: 'Job' object has no attribute 'exception_info' if Job object definitely has this attribute. Maybe permissions is bad again and it just could not import it?

rwyse commented 6 years ago

@deniszh the permissions look to be set properly; however I will double check on those tomorrow when I am back to my workstation. As for the installation, I have reinstalled this many times now with the exact same results. I suppose it is possible that I grabbed tar balls that are problematic and I need to grab new ones to attempt the installation, but as it stands, I get this same result any time that I install the application on any new instance or current. Repeatable every time. If its an issue with the source that I am using then do you have known good or stable tar balls that you could point me to so that I can attempt it with them?

Again, I really appreciate the help on it. Redoing things from scratch yields the same results I currently have so I am thinking at this point it has to be something code related.....

Thanks and let me know what you think or what you think I should use, -Ryan

deniszh commented 6 years ago

Hello Ryan, Do you still have AttributeError: 'Job' object has no attribute 'exception_info' error? If yes - could you please show me output of ls -al /opt/graphite/webapp/graphite ls -al /opt/graphite/webapp/graphite/worker_pool commands. And if you using some manual for installation - it would be nice to have a link, maybe it's outdated.

rwyse commented 6 years ago

Yes, here is the output requested:

ls -al /opt/graphite/webapp/graphite total 348 drwxr-xr-x 18 apache apache 4096 Feb 16 16:06 . drwxr-xr-x 4 apache apache 78 Feb 5 23:30 .. drwxr-xr-x 3 apache apache 240 Feb 5 23:30 account -rw-r--r-- 1 apache apache 3336 Jan 9 07:34 app_settings.py -rw-r--r-- 1 apache apache 3764 Feb 5 23:30 app_settings.pyc drwxr-xr-x 2 apache apache 109 Feb 5 23:30 browser -rw-r--r-- 1 apache apache 7059 Jan 9 07:34 carbonlink.py -rw-r--r-- 1 apache apache 8880 Feb 5 23:30 carbonlink.pyc -rw-r--r-- 1 apache apache 539 Jan 9 07:34 compat.py -rw-r--r-- 1 apache apache 1156 Feb 5 23:30 compat.pyc drwxr-xr-x 2 apache apache 109 Feb 5 23:30 composer drwxr-xr-x 3 apache apache 238 Feb 5 23:30 dashboard -rw-r--r-- 1 apache apache 94 Jan 9 07:34 errors.py -rw-r--r-- 1 apache apache 340 Feb 5 23:30 errors.pyc drwxr-xr-x 3 apache apache 230 Feb 5 23:30 events drwxr-xr-x 2 apache apache 185 Feb 5 23:30 finders drwxr-xr-x 3 apache apache 158 Feb 5 23:30 functions -rw-r--r-- 1 apache apache 95 Jan 9 07:34 http_pool.py -rw-r--r-- 1 apache apache 290 Feb 5 23:30 http_pool.pyc -rw-r--r-- 1 apache apache 53 Jan 9 07:34 __init__.py -rw-r--r-- 1 apache apache 128 Feb 5 23:30 __init__.pyc -rw-r--r-- 1 apache apache 4223 Jan 9 07:34 intervals.py -rw-r--r-- 1 apache apache 7251 Feb 5 23:30 intervals.pyc -rw-r--r-- 1 apache apache 15351 Feb 16 16:06 local_settings.py -rw-r--r-- 1 apache apache 2131 Feb 16 16:06 local_settings.pyc -rw-r--r-- 1 apache apache 15574 Jan 9 07:34 local_settings.py.example -rw-r--r-- 1 apache apache 3644 Jan 9 07:34 logger.py -rw-r--r-- 1 apache apache 4438 Feb 5 23:30 logger.pyc drwxr-xr-x 2 apache apache 109 Feb 5 23:30 metrics -rw-r--r-- 1 apache apache 396 Jan 9 07:34 middleware.py -rw-r--r-- 1 apache apache 854 Feb 5 23:30 middleware.pyc -rw-r--r-- 1 apache apache 971 Jan 9 07:34 node.py -rw-r--r-- 1 apache apache 2179 Feb 5 23:30 node.pyc drwxr-xr-x 2 apache apache 245 Feb 5 23:30 readers drwxr-xr-x 2 apache apache 4096 Feb 16 16:23 render -rw-r--r-- 1 apache apache 10186 Feb 6 11:47 settings.py -rw------- 1 apache apache 7892 Feb 6 11:51 settings.pyc -rw-r--r-- 1 apache apache 2703 Jan 9 07:34 singleton.py -rw-r--r-- 1 apache apache 4089 Feb 5 23:30 singleton.pyc -rw-r--r-- 1 apache apache 18295 Feb 13 09:44 storage.py -rw-r--r-- 1 apache apache 14269 Feb 13 09:45 storage.pyc drwxr-xr-x 3 apache apache 4096 Feb 5 23:30 tags drwxr-xr-x 2 apache apache 236 Feb 5 23:30 templates -rw-r--r-- 1 apache apache 35513 Jan 9 07:34 umsgpack.py -rw-r--r-- 1 apache apache 33021 Feb 5 23:30 umsgpack.pyc drwxr-xr-x 3 apache apache 170 Feb 5 23:30 url_shortener -rw-r--r-- 1 apache apache 1826 Jan 9 07:34 urls.py -rw-r--r-- 1 apache apache 2137 Feb 5 23:30 urls.pyc -rw-r--r-- 1 apache apache 1653 Jan 9 07:34 user_util.py -rw-r--r-- 1 apache apache 2034 Feb 5 23:30 user_util.pyc -rw-r--r-- 1 apache apache 10075 Jan 9 07:34 util.py -rw-r--r-- 1 apache apache 13839 Feb 5 23:30 util.pyc drwxr-xr-x 2 apache apache 109 Feb 5 23:30 version -rw-r--r-- 1 apache apache 314 Jan 9 07:34 views.py -rw-r--r-- 1 apache apache 623 Feb 5 23:30 views.pyc drwxr-xr-x 2 apache apache 109 Feb 5 23:30 whitelist drwxr-xr-x 2 apache apache 76 Feb 5 23:30 worker_pool -rw-r--r-- 1 apache apache 1241 Jan 9 07:34 wsgi.py -rw-r--r-- 1 apache apache 1410 Feb 5 23:30 wsgi.pyc

ls -al /opt/graphite/webapp/graphite/worker_pool total 16 drwxr-xr-x 2 apache apache 76 Feb 5 23:30 . drwxr-xr-x 18 apache apache 4096 Feb 16 16:06 .. -rw-r--r-- 1 apache apache 0 Jan 9 07:34 __init__.py -rw-r--r-- 1 apache apache 140 Feb 5 23:30 __init__.pyc -rw-r--r-- 1 apache apache 2686 Jan 9 07:34 pool.py -rw-r--r-- 1 apache apache 4078 Feb 5 23:30 pool.pyc

Thanks. I am currently testing what I think to be a more stable download that I grabbed out of the git repo upstream as well to see if that resolves the issue, but the install manual that I have been using is the one on the graphite read the docs wiki:

http://graphite.readthedocs.io/en/stable/releases/1_1_2.html

Ryan

rwyse commented 6 years ago

@deniszh I downloaded new tar files and loaded them into what I was doing yesterday. That seems to have corrected the issue. I am not sure why the original files I downloaded were problematic, but the new ones I have running in a test environment now and am looking to put it into stage shortly. If there is anything else that you can tell me on this I would be curious else I would think I can close this issue since I have it resolved with the new installation files.

Thanks, -Ryan