search

graphite-project / graphite-web

A highly scalable real-time graphing system
http://graphite.readthedocs.org/
Apache License 2.0
5.88k stars 1.26k forks source link

go-carbon stopping to pull metrics and showing below erroe #2770

Closed Prasadevops0710 closed 1 year ago

Prasadevops0710 commented 1 year ago

go-carbon stopped to show metrics and showing below error everytime I need to restart go-carbon service to work fine can u please help regarding this @deniszh

Jul 28 01:06:03 ip-10-0-250-82 sh[16094]: os.rename(self.baseFilename, dfn) Jul 28 01:06:03 ip-10-0-250-82 sh[16094]: OSError: [Errno 2] No such file or directory Jul 28 01:06:03 ip-10-0-250-82 sh[16094]: Logged from file logger.py, line 88 Jul 28 01:56:37 ip-10-0-250-82 sh[16094]: Traceback (most recent call last): Jul 28 01:56:37 ip-10-0-250-82 sh[16094]: File "/usr/lib/python2.7/logging/handlers.py", line 77, in emit Jul 28 01:56:37 ip-10-0-250-82 sh[16094]: self.doRollover() Jul 28 01:56:37 ip-10-0-250-82 sh[16094]: File "/usr/lib/python2.7/logging/handlers.py", line 350, in doRollover Jul 28 01:56:37 ip-10-0-250-82 sh[16094]: os.rename(self.baseFilename, dfn) Jul 28 01:56:37 ip-10-0-250-82 sh[16094]: OSError: [Errno 2] No such file or directory Jul 28 01:56:37 ip-10-0-250-82 sh[16094]: Logged from file logger.py, line 88

Prasadevops0710 commented 1 year ago

please help @deniszh its production server

Prasadevops0710 commented 1 year ago

Traceback (most recent call last): File "/data/graphite/webapp/graphite/finders/remote.py", line 255, in request preload_content=False) File "/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 76, in request method, url, fields=fields, headers=headers, urlopen_kw File "/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 97, in request_encode_url return self.urlopen(method, url, extra_kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 330, in urlopen response = conn.urlopen(method, u.request_uri, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 760, in urlopen response_kw File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 760, in urlopen response_kw File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 760, in urlopen response_kw File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) MaxRetryError: HTTPConnectionPool(host='graphite-sea.knorex-internal.com', port=80): Max retries exceeded with url: /render/?format=pickle&local=1&noCache=1&from=1658993763&until=1658997363&target=rtb-test.st.aggregated.accounts.c4689c2a_bc8b_4d50_9d91_7dcd51beeb4a.2f97f262_b5c8_40d0_a14e_aad8f8b90dab.%2A&now=1658997363 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fafd8631fd0>: Failed to establish a new connection: [Errno -2] Name or service not known',)) 2022-07-28,08:36:03.796 :: Failed after 2 attempts! Root cause: Traceback (most recent call last): File "/data/graphite/webapp/graphite/readers/remote.py", line 63, in fetch_multi timeout=settings.FETCH_TIMEOUT, File "/data/graphite/webapp/graphite/finders/remote.py", line 259, in request raise Exception("Error requesting %s: %s" % (url_full, err)) Exception: Error requesting http://graphite-sea.knorex-internal.com:80/render/?format=pickle&local=1&noCache=1&from=1658993763&until=1658997363&target=rtb-test.st.aggregated.accounts.c4689c2a_bc8b_4d50_9d91_7dcd51beeb4a.2f97f262_b5c8_40d0_a14e_aad8f8b90dab.%2A&now=1658997363: HTTPConnectionPool(host='graphite-sea.knorex-internal.com', port=80): Max retries exceeded with url: /render/?format=pickle&local=1&noCache=1&from=1658993763&until=1658997363&target=rtb-test.st.aggregated.accounts.c4689c2a_bc8b_4d50_9d91_7dcd51beeb4a.2f97f262_b5c8_40d0_a14e_aad8f8b90dab.%2A&now=1658997363 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fafd8631fd0>: Failed to establish a new connection: [Errno -2] Name or service not known',)) Traceback (most recent call last): File "/data/graphite/webapp/graphite/readers/remote.py", line 63, in fetch_multi timeout=settings.FETCH_TIMEOUT, File "/data/graphite/webapp/graphite/finders/remote.py", line 259, in request raise Exception("Error requesting %s: %s" % (url_full, err)) Exception: Error requesting http://graphite-sea.knorex-internal.com:80/render/?format=pickle&local=1&noCache=1&from=1658993763&until=1658997363&target=rtb-test.st.aggregated.accounts.c4689c2a_bc8b_4d50_9d91_7dcd51beeb4a.2f97f262_b5c8_40d0_a14e_aad8f8b90dab.%2A&now=1658997363: HTTPConnectionPool(host='graphite-sea.knorex-internal.com', port=80): Max retries exceeded with url: /render/?format=pickle&local=1&noCache=1&from=1658993763&until=1658997363&target=rtb-test.st.aggregated.accounts.c4689c2a_bc8b_4d50_9d91_7dcd51beeb4a.2f97f262_b5c8_40d0_a14e_aad8f8b90dab.%2A&now=1658997363 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fafd8631fd0>: Failed to establish a new connection: [Errno -2] Name or service not

Prasadevops0710 commented 1 year ago

s