ravishankar15
commented
1 month ago Hi ! I am not sure if this is still reproducible, I tried to push the log, and fetch it using query and it seems to be working,
Push,
Curl
curl --location 'http://localhost:3100/loki/api/v1/push' \
--header 'Content-Type: application/json' \
--data-raw '{
"streams": [
{
"stream": {
"apipush": "helloss"
},
"values": [
[
"1722029400000000000",
"{\"geoip_city_country_code\":\"FR\",\"request_method\":\"GET\",\"request_id\":\"XXXXXXXXXXXX\",\"@timestamp\":\"2024-03-22T08:42:55.639\",\"http2\":\"h2\",\"status\":200,\"level\":\"INFO\",\"brotli_ratio\":6.35,\"iiif_is_internal_user\":false,\"upstream_addr\":[\"XXXXXXXX\"],\"vhost\":\"somevhost\",\"upstream_http_content_type\":\"text\\/html\",\"http_user_agent\":\"someAgent\",\"body_bytes_sent\":2153,\"pipe\":false,\"request_uri\":\"/some_uri\",\"geoip_point\":[0,0],\"upstream_status\":[200],\"remote_addr\":\"0.0.0.0\",\"remote_hash\":\"XXXXXXXXX\"}"
]
]
}
]
}'Result
{
"streams": [
{
"stream": {
"apipush": "helloss"
},
"values": [
[
"1722029400000000000",
"{\"geoip_city_country_code\":\"FR\",\"request_method\":\"GET\",\"request_id\":\"XXXXXXXXXXXX\",\"@timestamp\":\"2024-03-22T08:42:55.639\",\"http2\":\"h2\",\"status\":200,\"level\":\"INFO\",\"brotli_ratio\":6.35,\"iiif_is_internal_user\":false,\"upstream_addr\":[\"XXXXXXXX\"],\"vhost\":\"somevhost\",\"upstream_http_content_type\":\"text\\/html\",\"http_user_agent\":\"someAgent\",\"body_bytes_sent\":2153,\"pipe\":false,\"request_uri\":\"/some_uri\",\"geoip_point\":[0,0],\"upstream_status\":[200],\"remote_addr\":\"0.0.0.0\",\"remote_hash\":\"XXXXXXXXX\"}"
]
]
}
]
}Query
Curl
curl --location --globoff 'localhost:3100/loki/api/v1/query_range?query={apipush%3D%22helloss%22}%20%7C%20json%20request_uri&start=1721022200000000000&end=1723025800000000000'Result
{
"status": "success",
"data": {
"resultType": "streams",
"result": [
{
"stream": {
"apipush": "helloss",
"detected_level": "info",
**"request_uri": "/some_uri",** Note here the label is extracted without any errors
"service_name": "unknown_service"
},
"values": [
[
"1722029400000000000",
"{\"geoip_city_country_code\":\"FR\",\"request_method\":\"GET\",\"request_id\":\"XXXXXXXXXXXX\",\"@timestamp\":\"2024-03-22T08:42:55.639\",\"http2\":\"h2\",\"status\":200,\"level\":\"INFO\",\"brotli_ratio\":6.35,\"iiif_is_internal_user\":false,\"upstream_addr\":[\"XXXXXXXX\"],\"vhost\":\"somevhost\",\"upstream_http_content_type\":\"text\/html\",\"http_user_agent\":\"someAgent\",\"body_bytes_sent\":2153,\"pipe\":false,\"request_uri\":\"/some_uri\",\"geoip_point\":[0,0],\"upstream_status\":[200],\"remote_addr\":\"0.0.0.0\",\"remote_hash\":\"XXXXXXXXX\"}"
]
]
}
],
"stats": {
"summary": {
"bytesProcessedPerSecond": 1215,
"linesProcessedPerSecond": 2,
"totalBytesProcessed": 540,
"totalLinesProcessed": 1,
"execTime": 0.444164,
"queueTime": 0.017401,
"subqueries": 0,
"totalEntriesReturned": 1,
"splits": 558,
"shards": 558,
"totalPostFilterLines": 1,
"totalStructuredMetadataBytesProcessed": 30
},
"querier": {
"store": {
"totalChunksRef": 1,
"totalChunksDownloaded": 1,
"chunksDownloadTime": 102125,
"queryReferencedStructuredMetadata": false,
"chunk": {
"headChunkBytes": 0,
"headChunkLines": 0,
"decompressedBytes": 540,
"decompressedLines": 1,
"compressedBytes": 345,
"totalDuplicates": 0,
"postFilterLines": 1,
"headChunkStructuredMetadataBytes": 0,
"decompressedStructuredMetadataBytes": 30
},
"chunkRefsFetchTime": 15716203,
"congestionControlLatency": 0,
"pipelineWrapperFilteredLines": 0
}
},
"ingester": {
"totalReached": 214,
"totalChunksMatched": 0,
"totalBatches": 214,
"totalLinesSent": 0,
"store": {
"totalChunksRef": 0,
"totalChunksDownloaded": 0,
"chunksDownloadTime": 0,
"queryReferencedStructuredMetadata": false,
"chunk": {
"headChunkBytes": 0,
"headChunkLines": 0,
"decompressedBytes": 0,
"decompressedLines": 0,
"compressedBytes": 0,
"totalDuplicates": 0,
"postFilterLines": 0,
"headChunkStructuredMetadataBytes": 0,
"decompressedStructuredMetadataBytes": 0
},
"chunkRefsFetchTime": 7132329,
"congestionControlLatency": 0,
"pipelineWrapperFilteredLines": 0
}
},
"cache": {
"chunk": {
"entriesFound": 1,
"entriesRequested": 1,
"entriesStored": 0,
"bytesReceived": 680,
"bytesSent": 0,
"requests": 2,
"downloadTime": 4083,
"queryLengthServed": 0
},
"index": {
"entriesFound": 0,
"entriesRequested": 0,
"entriesStored": 0,
"bytesReceived": 0,
"bytesSent": 0,
"requests": 0,
"downloadTime": 0,
"queryLengthServed": 0
},
"result": {
"entriesFound": 0,
"entriesRequested": 0,
"entriesStored": 0,
"bytesReceived": 0,
"bytesSent": 0,
"requests": 0,
"downloadTime": 0,
"queryLengthServed": 0
},
"statsResult": {
"entriesFound": 360,
"entriesRequested": 360,
"entriesStored": 0,
"bytesReceived": 54585,
"bytesSent": 0,
"requests": 360,
"downloadTime": 997831,
"queryLengthServed": 1253366000000000
},
"volumeResult": {
"entriesFound": 0,
"entriesRequested": 0,
"entriesStored": 0,
"bytesReceived": 0,
"bytesSent": 0,
"requests": 0,
"downloadTime": 0,
"queryLengthServed": 0
},
"seriesResult": {
"entriesFound": 0,
"entriesRequested": 0,
"entriesStored": 0,
"bytesReceived": 0,
"bytesSent": 0,
"requests": 0,
"downloadTime": 0,
"queryLengthServed": 0
},
"labelResult": {
"entriesFound": 0,
"entriesRequested": 0,
"entriesStored": 0,
"bytesReceived": 0,
"bytesSent": 0,
"requests": 0,
"downloadTime": 0,
"queryLengthServed": 0
},
"instantMetricResult": {
"entriesFound": 0,
"entriesRequested": 0,
"entriesStored": 0,
"bytesReceived": 0,
"bytesSent": 0,
"requests": 0,
"downloadTime": 0,
"queryLengthServed": 0
}
},
"index": {
"totalChunks": 0,
"postFilterChunks": 0,
"shardsDuration": 0
}
}
}
}
Describe the bug When extracting a label from a JSON log line via
... |json mylabel="mykey"or variants thereof, aJSONParserErris raised and no label is created.To Reproduce Steps to reproduce the behavior:
{env="prod", app="nginx"} |json request_uri{env="prod", app="nginx"} |json request_uri="request_uri"Expected behavior A label
request_uriis created that contains the value/some_uri. The label is created without error along with all other json labels if thejsonparser is called without any arguments. I was expecting this to work since it's pretty much 1:1 the example from the 2.9.x Loki online documentation and the LogQL Analyzer in the documentation correctly parses the query.Environment:
Screenshots, Promtail config, or terminal output