Graphite-api sends requests for multiple metrics but cyanite responds with one

[dancb10]

We have created a setup of 3 Cassandra nodes in a cluster and two cyanite+graphite-api nodes. We have created all infrastructure with elbs and such and we have added the data source the graphite-api elb in grafana. Everything is connected and all flows are opened. But there is one problem when trying to build dashboards. We are trying to get multiple metrics visible on the same dashboard in grafana. While we can see data from each metric individually, when asking for more metrics, we receive only one:

DEBUG [2017-12-06 17:40:18,696] epollEventLoopGroup-3-1 - io.cyanite.api got request:  {:uri "/paths", :get-params {:query "servers.X.disk.xvda1.disk_octets.read"}, :params {:query "servers.X.disk.xvda1.disk_octets.read"}, :request-method :get, :version "HTTP/1.1", :headers {:host "127.0.0.1:8080", :connection "keep-alive", :accept-encoding "gzip, deflate", :accept "*/*", :user-agent "python-requests/2.18.4"}, :body #object[clojure.core.async.impl.channels.ManyToManyChannel 0x52ef24df "clojure.core.async.impl.channels.ManyToManyChannel@52ef24df"]}
DEBUG [2017-12-06 17:40:18,696] epollEventLoopGroup-3-1 - io.cyanite.api path fetch request for: "servers.X.disk.xvda1.disk_octets.read"
DEBUG [2017-12-06 17:40:18,702] epollEventLoopGroup-3-1 - io.cyanite.api got request:  {:uri "/paths", :get-params {:query "servers.X.load.load.shortterm"}, :params {:query "servers.X.load.load.shortterm"}, :request-method :get, :version "HTTP/1.1", :headers {:host "127.0.0.1:8080", :connection "keep-alive", :accept-encoding "gzip, deflate", :accept "*/*", :user-agent "python-requests/2.18.4"}, :body #object[clojure.core.async.impl.channels.ManyToManyChannel 0x63e298b4 "clojure.core.async.impl.channels.ManyToManyChannel@63e298b4"]}
DEBUG [2017-12-06 17:40:18,702] epollEventLoopGroup-3-1 - io.cyanite.api path fetch request for: "servers.X.load.load.shortterm"
DEBUG [2017-12-06 17:40:18,706] epollEventLoopGroup-3-1 - io.cyanite.api got request:  {:uri "/metrics", :get-params {:path "servers.X.disk.xvda1.disk_octets.read", :from "1512578418", :to "1512582018"}, :params {:path "servers.X.xvda1.disk_octets.read", :from "1512578418", :to "1512582018"}, :request-method :get, :version "HTTP/1.1", :headers {:host "127.0.0.1:8080", :connection "keep-alive", :accept-encoding "gzip, deflate", :accept "*/*", :user-agent "python-requests/2.18.4"}, :body #object[clojure.core.async.impl.channels.ManyToManyChannel 0x490637d8 "clojure.core.async.impl.channels.ManyToManyChannel@490637d8"]}
DEBUG [2017-12-06 17:40:18,707] epollEventLoopGroup-3-1 - io.cyanite.api metric fetch request for: "servers.X.disk.xvda1.disk_octets.read"

So basically we query for disk.xvda1.disk_octets.read and load.load.shortterm and we receive only disk_octets.read. This issue is blocking us from making complex dashboards with multiple metrics.

We have tried also with curl and we get the same behavior:

curl -X POST https://X/render --data ‘target=servers.X.disk.xvda1.disk_octets.read&target=servers.X.load.load.shortterm&from=-1h&until=now&format=json&maxDataPoints=1311’

How can we fix this issue? (we are using Graphite-Api 1.1.2, latest cyanite version and grafana 4.6.x) Note that I have replaced our node name with X

[dancb10]

@pyr seems like this issue is related to another opened issue https://github.com/pyr/cyanite/issues/279

[dancb10]

I have compiled the project after this commit https://github.com/pyr/cyanite/pull/265 and it seems that now we can see data but the log file of cyanite is spamming with the following message: INFO [2017-12-07 13:11:24,242] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:25,243] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:26,242] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:27,243] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:28,242] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:29,242] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:30,243] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:31,243] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:32,243] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:33,243] cyanite-snapshot - io.cyanite.engine.writer starting snapshot. INFO [2017-12-07 13:11:34,243] cyanite-snapshot - io.cyanite.engine.writer starting snapshot.

It also seems like we are loosing data somehow. We are still debugging