[carbon-cache] Error: Connection refused by other side: 111

[qops1981]

Greetings,

I am building out a new Graphite Box since or other one is using a much older version and not appropriately configured. That being said I am trying to set up using two carbon-cache processes with 1 carbon-relay. It basically works, but reviewing the logs for carbon-cache I see many of these messages.

11/10/2017 12:04:50 :: Tagging carbon.agents.ubuntu-16-ctk-b.committedPoints
11/10/2017 12:04:50 :: Starting factory _HTTP11ClientFactory(<function quiescentCallback at 0x7f6980d56ed8>, <twisted.internet.endpoints.HostnameEndpoint object at 0x7f6980d3b990>)
11/10/2017 12:04:50 :: Stopping factory _HTTP11ClientFactory(<function quiescentCallback at 0x7f6980d56ed8>, <twisted.internet.endpoints.HostnameEndpoint object at 0x7f6980d3b990>)
11/10/2017 12:04:50 :: Error tagging carbon.agents.ubuntu-16-ctk-b.committedPoints: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionRefusedError'>: Connection was refused by other side: 111: Connection refused.
]
11/10/2017 12:05:21 :: Tagging ubuntu-16-ctk.disktotals.writekbs
11/10/2017 12:05:21 :: Starting factory _HTTP11ClientFactory(<function quiescentCallback at 0x7f6980f80410>, <twisted.internet.endpoints.HostnameEndpoint object at 0x7f6980d25090>)
11/10/2017 12:05:21 :: Stopping factory _HTTP11ClientFactory(<function quiescentCallback at 0x7f6980f80410>, <twisted.internet.endpoints.HostnameEndpoint object at 0x7f6980d25090>)
11/10/2017 12:05:21 :: Error tagging ubuntu-16-ctk.disktotals.writekbs: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionRefusedError'>: Connection was refused by other side: 111: Connection refused.
]

I do see metrics getting recorded, but this seems to indicate that not all of them are. What am I doing wrong?

Besides Carbon Metrics I was also using Callectl to test metric ingestion. sudo collectl --export graphite,127.0.0.1:2013

carbon.conf

[cache]
DATABASE = whisper
ENABLE_LOGROTATION = True
USER =
MAX_CACHE_SIZE = inf
MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 50
MIN_TIMESTAMP_RESOLUTION = 1
LINE_RECEIVER_INTERFACE = 0.0.0.0
LINE_RECEIVER_PORT = 2003
ENABLE_UDP_LISTENER = False
UDP_RECEIVER_INTERFACE = 0.0.0.0
UDP_RECEIVER_PORT = 2003
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
PICKLE_RECEIVER_PORT = 2004
USE_INSECURE_UNPICKLER = False
CACHE_QUERY_INTERFACE = 0.0.0.0
CACHE_QUERY_PORT = 7002
USE_FLOW_CONTROL = True
LOG_UPDATES = False
LOG_CREATES = False
LOG_CACHE_HITS = False
LOG_CACHE_QUEUE_SORTS = False
CACHE_WRITE_STRATEGY = sorted
WHISPER_AUTOFLUSH = False
WHISPER_FALLOCATE_CREATE = True

[cache:b]
LINE_RECEIVER_PORT = 2103
PICKLE_RECEIVER_PORT = 2104
CACHE_QUERY_PORT = 7102

[relay]
LINE_RECEIVER_INTERFACE = 0.0.0.0
LINE_RECEIVER_PORT = 2013
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
PICKLE_RECEIVER_PORT = 2014

RELAY_METHOD = consistent-hashing
DESTINATIONS = 127.0.0.1:2004:a, 127.0.0.1:2104:b
MAX_QUEUE_SIZE = 10000
MAX_DATAPOINTS_PER_MESSAGE = 500
QUEUE_LOW_WATERMARK_PCT = 0.8
TIME_TO_DEFER_SENDING = 0.0001
USE_FLOW_CONTROL = True
USE_RATIO_RESET=False
MIN_RESET_STAT_FLOW=1000
MIN_RESET_RATIO=0.9
MIN_RESET_INTERVAL=121
[aggregator]
LINE_RECEIVER_INTERFACE = 0.0.0.0
LINE_RECEIVER_PORT = 2023
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
PICKLE_RECEIVER_PORT = 2024
FORWARD_ALL = True
DESTINATIONS = 127.0.0.1:2004
REPLICATION_FACTOR = 1
MAX_QUEUE_SIZE = 10000
USE_FLOW_CONTROL = True
MAX_DATAPOINTS_PER_MESSAGE = 500
MAX_AGGREGATION_INTERVALS = 5

Any help would be appricated. Thank you.

[DanCech]

You need to configure GRAPHITE_URL = http://your.graphite in your [cache] config to point to your graphite-web install if it's not available on localhost port 8000

[qops1981]

Woot, thank you. You saved me the rest of my day.. Probably a couple of days, lol!