search

noi-techpark / bdp-commons

Open Data Hub time series data collectors (legacy)
GNU Affero General Public License v3.0
2 stars 12 forks source link

Meteorology-eurac: High number of warnings in logs #611

Closed clezag closed 1 year ago

clezag commented 1 year ago

There's a lot (~500k per day) warning logs generated by the mobility-writer API, all related to calls coming from the meteoroloy-eurac data collector.

At first glance, these all seem to be about an invalid '(null') value, maybe this could be solved on the data collector side.

Example here&_a=(columns:!(container.labels.com_docker_compose_project),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:b1f70440-79a2-11ea-b0b8-c5ba8b8d5550,key:json.api_request_info.provenance_name,negate:!f,params:(query:odh-mobility-dc-meteorology-eurac),type:phrase),query:(match_phrase:(json.api_request_info.provenance_name:odh-mobility-dc-meteorology-eurac)))),index:b1f70440-79a2-11ea-b0b8-c5ba8b8d5550,interval:auto,query:(language:kuery,query:''),sort:!(!('@timestamp',desc))))

clezag commented 1 year ago

0b6e40b prevents pushing null values.

clezag commented 1 year ago

Seems to be working: https://logs.opendatahub.com/goto/38bc74c0-0081-11ee-b69a-a15ab3b1f0cb

Logs have been drastically reduced