[Questions] carbonapi have different tags object in response

[evdevk]

Problem description Have been used carbonapi for requests from grafana to clickhouse server (graphite-clickhouse inbetween). After update to newer version noticed what response json become different

Old version request (working as inteded):

request:Object
    method:"POST"
    url:"api/datasources/proxy/19/render"
   data:"target=seriesByTag('name%3Dsystem.diskio.reads'%2C%20'host%3D~server'%2C%20'_name%3Dsda')&from=-5min&until=now&format=json&maxDataPoints=2503"
    hideFromInspector:false

Old version response (working as inteded):

response:Array[1]
0:Object
    target:"system.diskio.reads;_name=sda;contur=prod;host=server1;interval=15s"
    datapoints:Array[21]
    tags:Object
        _name:"sda"
        host:"server1"
        interval:"15s"
        name:"system.diskio.reads"
    title:"system.diskio.reads;_name=sda;contur=prod;host=server1;interval=15s"

New version request is the same as older version. New version response also the same. BUT if I put to a request aliasByTags function - tags object in response are different for old and new versions.

Request:

request:Object
    method:"POST"
    url:"api/datasources/proxy/11/render"
    data:"target=aliasByTags(seriesByTag('name%3Dsystem.diskio.reads'%2C%20'host%3D~server1'%2C%20'_name%3Dsda')%2C%20'name')&from=-5min&until=now&format=json&maxDataPoints=2503"
    hideFromInspector:false

Old version response:

response:Array[1]
0:Object
    target:"system.diskio.reads"
    datapoints:Array[20]
    tags:Object
        _name:"sda"
        host:"server1"
        interval:"15s"
        name:"system.diskio.reads"

New version response:

response:Array[1]
0:Object
    target:"system.diskio.reads"
    datapoints:Array[20]
    tags:Object
        name:"system.diskio.reads"

in my case tags list are matter. How can I return back old tags oject result? any config options?

carbonapi's version v 0.15.6 and later - working with issue. tried different versions, newest included v 0.15.5 and less - working as intended

Does this happened before No. After

carbonapi's config Both versions have the same config file:

upstreams:
    doMultipleRequestsIfSplit: false
    tldCacheDisabled: false

    graphite09compat: false
    buckets: 10

    concurrencyLimitPerServer: 0
    keepAliveInterval: "30s"
    maxIdleConnsPerHost: 100
    timeouts:
        find: "3s"
        render: "10s"
        connect: "200ms"
    backendsv2:
        backends:
            - 
                groupName: "clickhouse-cluster"
                protocol: "carbonapi_v3_pb"
                lbMethod: "roundrobin"
                maxTries: 1
                maxBatchSize: 0
                keepAliveInterval: "10s"
                concurrencyLimit: 0
                maxIdleConnsPerHost: 100
                forceAttemptHTTP2: false
                doMultipleRequestsIfSplit: true 

                timeouts:
                    find: "2s"
                    render: "50s"
                    connect: "1s"
                servers:
                    - "http://server1:7090"
                    - "http://server2:7090"

listen: "0.0.0.0:8091"
concurency: 1000
cache:
   type: "mem"
   size_mb: 2048 
   defaultTimeoutSec: 10
cpus: 8
tz: ""
unicodeRangeTables:
  - "all"
idleConnections: 1000
pidFile: ""

functionsConfig:
    graphiteWeb: /etc/carbonapi/graphiteWeb.yaml
maxBatchSize: 0
expireDelaySec: 300
graphTemplates: /etc/carbonapi/graphTemplates.yaml
prefix: ""
useCachingDNSResolver: false
cachingDNSRefreshTime: "1m"
expvar:
  enabled: true
  pprofEnabled: false
  listen: ""
headersToPass:
  - "X-Dashboard-Id"
  - "X-Grafana-Org-Id"
  - "X-Panel-Id"
headersToLog:
  - "X-Dashboard-Id"
  - "X-Grafana-Org-Id"
  - "X-Panel-Id"
notFoundStatusCode: 200
logger:
    - logger: ""
      file: "/var/log/carbonapi/carbonapi.log"
      level: "info"
      encoding: "json"
graphite:
    host: ""
    interval: "10s"
    prefix: "carbonapi"

backend software and config graphite-clickhouse 0.12.0 clickhouse-server 22.8.4.7 (official build)

carbonapi performance metrics

Query that causes problems

Additional context

[npazosmendez]

This might be a bug, potentially introduced here https://github.com/go-graphite/carbonapi/commit/d77e3ee963f7262ee852034f6fdcfa27663bca92#diff-fce0198985d6dad323115ac9f864bab245cb25e6b06dd228c901e92e6a590b71

The CopyTag call is overriding the tags

https://github.com/go-graphite/carbonapi/blob/236726cbe92f5aa1e478544a2e57bcffef5c4219/expr/functions/aliasByNode/function.go#L47-L48

However, it looks like that was introduced in v0.16.0-patch1. You say this happens in v 0.15.6 and later ?

[evdevk]

However, it looks like that was introduced in v0.16.0-patch1. You say this happens in v 0.15.6 and later ?

I have a build which say on the start in journalctl v0.15.6-296-g0bb2. but is there already. v 0.15.5 working as intended. I tried also latest versions 0.16.X - bug was there also.

[npazosmendez]

I've submitted a PR that should solve the issue.

[evdevk]

this bug is still around in latest 0.16.1 https://packagecloud.io/go-graphite/autobuilds/packages/el/8/carbonapi-0.16.1~10+sha.gea3cf7-1.x86_64.rpm?distro_version_id=205

additional info: typical request without alias like: seriesByTag('name=system.system.load5', 'host=server1234') works ok. response tags array looks like:

tags:Object
    contur:"prod"
    host:"server1234"
    interval:"15s"
    ostype:"linux"

this one are without tags: groupByTags(seriesByTag('name=system.system.load5', 'host=server1234'), 'max', 'host')

tags:Object
   host:"server1234"
   name:"max"

with older versions 0.15.5 works ok.

requested new bug here https://github.com/go-graphite/carbonapi/issues/808