Error in logs 'missing tag value'

[Luukth]

Hi, I'm see this error when typing docker-compose logs -f. Some failed login attempts show up in grafana. But a lot of them don't and then this error appears. If you need more information please let me know. Cool project!!

I'm using the docker-compose.standalone.yml file.

version: "3.7"

services:
  ssh-log-to-influx:
    image: acouvreur/ssh-log-to-influx
    build: ./
    restart: always
    environment: 
      - INFLUX_URL=influxdb
      - INFLUX_DB=telegraf
    ports: 
      - 7070:7070/tcp

  #----------------------------------------------#
  # InfluxDB : time series database
  #----------------------------------------------#
  influxdb:
    image: influxdb:latest
    environment: 
      - INFLUXDB_DB=telegraf
      - INFLUXDB_ADMIN_USER=telegraf
      - INFLUXDB_ADMIN_PASSWORD=grafana
    volumes:
      - influxdb_data:/var/lib/influxdb

  #----------------------------------------------#
  # Grafana : analytics and monitoring
  #----------------------------------------------#
  grafana:
    image: "grafana/grafana:master"
    environment: 
      - GF_INSTALL_PLUGINS=grafana-worldmap-panel
    ports: 
      - 3000:3000
    volumes:
      - grafana_data:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning
      - ./grafana/config.ini:/etc/grafana/config.ini
      - ./grafana/dashboards:/var/lib/grafana/dashboards

volumes:
  influxdb_data:
  grafana_data:

ssh-log-to-influx_1  | [2020-05-09T13:30:05.272] [INFO] default - CLOSED: ::ffff:172.21.0.1:37480
ssh-log-to-influx_1  | (node:30) UnhandledPromiseRejectionWarning: Error: A 400 Bad Request error occurred: {"error":"unable to parse 'geossh,geohash=wtw2de99s,username=root,port=22010,ip=222.186.175.202,location=Shanghai\\,\\ Shanghai,status=success,country=China,countryCode=CN,region=SH,regionName=Shanghai,city=Shanghai,zip=,timezone=Asia/Shanghai,isp=Chinanet\\ Jiangsu,org=Chinanet\\ JS,as=AS23650\\ AS\\ Number\\ for\\ CHINANET\\ jiangsu\\ province\\ backbone,query=222.186.175.202 value=1': missing tag value"}
ssh-log-to-influx_1  | 
ssh-log-to-influx_1  |     at IncomingMessage.<anonymous> (/app/node_modules/influx/lib/src/pool.js:49:38)
ssh-log-to-influx_1  |     at IncomingMessage.emit (events.js:322:22)
ssh-log-to-influx_1  |     at IncomingMessage.EventEmitter.emit (domain.js:482:12)
ssh-log-to-influx_1  |     at endReadableNT (_stream_readable.js:1187:12)
ssh-log-to-influx_1  |     at processTicksAndRejections (internal/process/task_queues.js:84:21)
ssh-log-to-influx_1  | (node:30) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 4)```

[acouvreur]

It looks like the parser is trying to parse an already parsed message ? Are you testing via netcat ? In this case you should see the parser specs https://github.com/acouvreur/ssh-log-to-influx/blob/master/src/parser.spec.js

As the format are raw data from rsyslog

[mahood73]

I get the same error - I removed line 87 (others...) from src/index.js and it seems to stop it erroring (although then Grafana misses a lot of data it wants).

I think it's actually an error returned by Influx for certain locations.

I believe it's the missing 'org' field in my case, and the missing 'zip' field in Luukth's example.

(node:56) UnhandledPromiseRejectionWarning: Error: A 400 Bad Request error occurred: {"error":"unable to parse 'geossh,geohash=gcq1dq86n,username=mark,port=49520,ip=86.140.12.9,location=England\,\ Ludlow,status=success,country=United\ Kingdom,countryCode=GB,region=ENG,regionName=England,city=Ludlow,zip=SY8,timezone=Europe/London,isp=BT\ Public\ Internet\ Service,org=,as=AS2856\ British\ Telecommunications\ PLC,query=86.140.12.9 value=1': missing tag value"}

ssh-log-to-influx_1 | (node:30) UnhandledPromiseRejectionWarning: Error: A 400 Bad Request error occurred: {"error":"unable to parse 'geossh,geohash=wtw2de99s,username=root,port=22010,ip=222.186.175.202,location=Shanghai\,\ Shanghai,status=success,country=China,countryCode=CN,region=SH,regionName=Shanghai,city=Shanghai,zip=,timezone=Asia/Shanghai,isp=Chinanet\ Jiangsu,org=Chinanet\ JS,as=AS23650\ AS\ Number\ for\ CHINANET\ jiangsu\ province\ backbone,query=222.186.175.202 value=1': missing tag value"}

[acouvreur]

Okay so the API might be returning empty values for some locations indeed. Which means that some tags are blank. By removing line 87 you don't save country, timezone, region, etc.

Thanks for your comment, I'll make a change to initialize every value to a default "none" value and overwrite it with the API call if it exist. It should be more consistent. But I find it weird that the API does not always retrieve all the values.

[acouvreur]

Can you provide me the log before the crash ? There should be something like

Received data from API ...

And it should shows the details from the data received from the API.

[mahood73]

Many thanks - I can see that removing that line is a bad idea! This is what I see - 'org' is blank:

[2020-05-10T13:02:07.360] [DEBUG] default - Not making an API Call for 86.140.12.9, using in memory from previous calls { status: 'success', country: 'United Kingdom', countryCode: 'GB', region: 'ENG', regionName: 'England', city: 'Ludlow', zip: 'SY8', lat: 52.331, lon: -2.713, timezone: 'Europe/London', isp: 'BT Public Internet Service', org: '', as: 'AS2856 British Telecommunications PLC', query: '86.140.12.9' } [2020-05-10T13:02:07.361] [DEBUG] default - Geohashing with lat: 52.331, lon: -2.713: gcq1dq86n [2020-05-10T13:02:07.362] [INFO] default - CLOSED: ::ffff:127.0.0.1:39712 (node:56) UnhandledPromiseRejectionWarning: Error: A 400 Bad Request error occurred: {"error":"unable to parse 'geossh,geohash=gcq1dq86n,username=mark,port=47680,ip=86.140.12.9,location=England\,\ Ludlow,status=success,country=United\ Kingdom,countryCode=GB,region=ENG,regionName=England,city=Ludlow,zip=SY8,timezone=Europe/London,isp=BT\ Public\ Internet\ Service,org=,as=AS2856\ British\ Telecommunications\ PLC,query=86.140.12.9 value=1': missing tag value"}

at IncomingMessage.<anonymous> (/app/node_modules/influx/lib/src/pool.js:49:38)
at IncomingMessage.emit (events.js:322:22)
at IncomingMessage.EventEmitter.emit (domain.js:482:12)
at endReadableNT (_stream_readable.js:1187:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)

(node:56) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)

[acouvreur]

I will change every empty string values into 'none' string. This should do the trick

[acouvreur]

Should be fixed now. See specs : https://github.com/acouvreur/ssh-log-to-influx/blob/master/src/api.spec.js