ERROR: Registration failed: Invalid source: * The json file /etc/sumo-sources.json contains invalid value.

[fgarsombke]

I would like to use the source replacement documented here https://github.com/SumoLogic/sumologic-collector-docker#using-source-templates but receive the error "ERROR: Registration failed: Invalid source: * The json file /etc/sumo-sources.json contains invalid value" upon container startup.

I am creating a new docker image using the SumoLogic docker image with the embedded source file that includes a replacement variable.

{
"api.version": "v1",
"sources": [{
"sourceType": "LocalFile",
"name": "${SERVICE_NAME} log",
"pathExpression": "/log/service.log",
"multilineProcessingEnabled": true,
"category": "${SERVICE_NAME}",
"encoding": "UTF-8"
}]
}

Create the docker file...

FROM sumologic/collector:latest
MAINTAINER Rachio
ADD sumo-sources.json.tmpl /etc/sumo.json.tmpl

Build the image...

docker build --tag="sumocollector" .

Then run the container...

docker run -e SUMO_ACCESS_ID=[your id] -e SUMO_ACCESS_KEY=[your key] -e SERVICE_NAME=foobar -e SUMO_SOURCES_JSON=/etc/sumo.json -t sumocollector

Even though I see this in the startup output when I ssh into the container there is no file called /etc/sumo.json

INFO: Replacing environment variables from /etc/sumo.json.tmpl into /etc/sumo.json
Running SumoLogic Collector...
wrapper | --> Wrapper Started as Console

It's almost like it can't write to that volume or something.

Here are the startup errors. Thoughts on a workaround for this? Our goal is to dynamically set the source category in a docker image that we will share across our micro services.

Thanks!

2021-12-15 04:45:46jvm 1 | oslib (Utils.readProcess): Cannot run program "lsb_release": error=2, No such file or directory
2021-12-15 04:45:46jvm 1 | (oslib.linux.CommandContainsType) cmd=lsb_release -sd > null
2021-12-15 04:45:46jvm 1 | * Connecting to https://collectors.sumologic.com.
2021-12-15 04:45:46jvm 1 | * ERROR: Registration failed: Invalid source: * The json file /etc/sumo-sources.json contains invalid value.
2021-12-15 04:45:46jvm 1 | Collector exiting...
2021-12-15 04:45:46wrapper | <-- Wrapper Stopped
2021-12-15 04:45:46jvm 1 | current folder:/opt/SumoCollector
2021-12-15 04:45:46jvm 1 | * See /opt/SumoCollector/./logs for more details.
2021-12-15 04:45:46jvm 1 | 2021-12-15 11:44:46,723 WrapperSimpleAppMain ERROR Appender Console cannot be located. Route ignored
2021-12-15 04:45:46jvm 1 | 2021-12-15 11:44:46,729 WrapperSimpleAppMain ERROR Appender Console cannot be located. Route ignored
2021-12-15 04:45:46jvm 1 | oslib (Utils.readProcess): Cannot run program "lsb_release": error=2, No such file or directory
2021-12-15 04:45:46jvm 1 | Sumo Logic Collector Version 19.361-12
2021-12-15 04:45:46jvm 1 | Sumo Logic Build Hash 994058cc4449
2021-12-15 04:44:18Running SumoLogic Collector...
2021-12-15 04:44:17INFO: Replacing environment variables from /etc/sumo-sources.json.tmpl into /etc/sumo-sources.json

[maimaisie]

Hi, I'm not sure if this is where the issue is but it seems you are using the file name sumo-sources in some places and sumo in others. Can you try using the same name everywhere? Eg. in your Dockerfile, use

ADD sumo-sources.json.tmpl /etc/sumo-sources.json.tmpl

And when you run the container, use -e SUMO_SOURCES_JSON=/etc/sumo-sources.json

[fgarsombke]

Thank you for the quick response @maimaisie. I tried your suggestion with the same results. I noticed in the example it has an explicit volume set which is different. Unfortunately I can not set that using Fargate.

I tested without the manualPrefixRegexp just to make sure it's not that. Same results with or without.

sumo-sources.json.tmpl

{
 "api.version": "v1",
 "sources": [{
  "sourceType": "LocalFile",
  "name": "${SERVICE_NAME} log",
  "pathExpression": "/log/service.log",
  "multilineProcessingEnabled": true,
  "useAutolineMatching": false,
  "manualPrefixRegexp": "\\d{4}\\-\\d{2}\\-\\d{2}\\s\\d{2}\\:\\d{2}\\:\\d{2}\\,\\d{3}\\s+.*",
  "automaticDateParsing": true,
  "category": "${SERVICE_NAME}",
  "encoding": "UTF-8"
 }]
}

FROM sumologic/collector:latest
MAINTAINER Rachio
ADD sumo-sources.json.tmpl /etc/sumo-sources.json.tmpl

docker build --tag="sumocollector" .

docker run -e SUMO_ACCESS_ID=[] -e SUMO_ACCESS_KEY=[] -e SERVICE_NAME=foobar -e SUMO_SOURCES_JSON=/etc/sumo-sources.json -t sumocollector

INFO: Replacing environment variables from /etc/sumo-sources.json.tmpl into /etc/sumo-sources.json
Running SumoLogic Collector...
wrapper  | --> Wrapper Started as Console
wrapper  | Java Service Wrapper Standard Edition 64-bit 3.5.33
wrapper  |   Copyright (C) 1999-2017 Tanuki Software, Ltd. All Rights Reserved.
wrapper  |     http://wrapper.tanukisoftware.com
wrapper  |   Licensed to Sumo Logic Inc. for Collector
wrapper  | 
wrapper  | Launching a JVM...
jvm 1    | WrapperManager: Initializing...
jvm 1    |    . .       . .       . .    .       . .
jvm 1    | .+'|=|`+. .+'| |`+. .+'|=|`+.=|`+. .+'|=|`+.
jvm 1    | |  | `+.| |  | |  | |  | `+ | `+ | |  | |  |
jvm 1    | |  | .    |  | |  | |  |  | |  | | |  | |  |
jvm 1    | `+.|=|`+. |  | |  | |  |  | |  | | |  | |  |
jvm 1    | .    |  | |  | |  | |  |  | |  | | |  | |  |
jvm 1    | |`+. |  | |  | |  | |  |  | |  | | |  | |  |
jvm 1    | `+.|=|.+' `+.|=|.+' `+.|  |.|  |+' `+.|=|.+'
jvm 1    | Sumo Logic Collector Version 19.361-12
jvm 1    | Sumo Logic Build Hash 994058cc4449
jvm 1    | current folder:/opt/SumoCollector
jvm 1    |   * See /opt/SumoCollector/./logs for more details.
jvm 1    | 2021-12-23 00:30:53,094 WrapperSimpleAppMain ERROR Appender Console cannot be located. Route ignored
jvm 1    | 2021-12-23 00:30:53,094 WrapperSimpleAppMain ERROR Appender Console cannot be located. Route ignored
jvm 1    | oslib (Utils.readProcess): Cannot run program "lsb_release": error=2, No such file or directory
jvm 1    | oslib (Utils.readProcess): Cannot run program "lsb_release": error=2, No such file or directory
jvm 1    | (oslib.linux.CommandContainsType) cmd=lsb_release -sd > null
jvm 1    |   * Connecting to https://collectors.sumologic.com.
jvm 1    |  * ERROR: Registration failed: Invalid source:  * The json file /etc/sumo-sources.json contains invalid value.
jvm 1    | Collector exiting...
wrapper  | <-- Wrapper Stopped

[fgarsombke]

I lied! It seems to be the regexp was the culprit (along with your suggestion). Changing to this fixed it. Thank you for the inspiration!

"manualPrefixRegexp": "\\d{4}\\-\\d{2}\\-\\d{2}\\s\\d{2}\\:\\d{2}\\:\\d{2}\\,\\d{3}\\s+.*",