OpenWB subscribe

[Bamfax]

Hi,

when trying to pull a topic from a OpenWB MQTT Broker (openwb.de) and forward it to Splunk, it dies with the error below. The MQTT pull works fine, but when connecting to the HEC endpoint it errors. I was not not able to see a call out to :8088 via tcpdump. I tried to configure the hec_url with both hostname (https:.//x.y.z) and ip only, also switched insecure_skip_verify to false. The HEC endpoint uses an unsigned certificate. A curl to the endpoint worked. Is there anything I am missing here or I could do to analyse further? Many thanks.

... subscribed to: openWB/global/ChargeMode Message could not be parsed (3): json: cannot unmarshal number into Go value of type map[string]interface {}received message: 3 panic: assignment to entry in nil map

goroutine 34 [running]: main.(handler).handle(0xc000013260, {0x1, 0x775030}, {0x777dc0, 0xc000296000}) /home/runner/work/splunk-mqtt/splunk-mqtt/main.go:98 +0x1fc github.com/eclipse/paho%2emqtt%2egolang.(router).matchAndDispatch.func2.1() /home/runner/go/pkg/mod/github.com/eclipse/paho.mqtt.golang@v1.3.5/router.go:184 +0x44 created by github.com/eclipse/paho%2emqtt%2egolang.(*router).matchAndDispatch.func2 /home/runner/go/pkg/mod/github.com/eclipse/paho.mqtt.golang@v1.3.5/router.go:183 +0x805

This was the last config:

---

broker: tcp://192.168.123.123:1883 mqtt_username: OpenWb mqtt_password: SomePW hec_url : https://127.0.0.1:8088/services/collector hec_token: a-b-c-d client_id: mqtt_subscribe write_to_console: true write_to_splunk: true topics:

• openWB/global/ChargeMode insecure_skip_verify: true

[aSauerwein]

Hi

My program assumes that all MQTT messages can be parsed as JSON but it looks like you receive just a plain Number.

Is there also a log line like 'received message:' ? Could you give me that output as well ?

Or do you know what exactly is being sent by the broker ? Maybe you could find this out as well using tcpdump.

PS: this explains why you see nothing on Hec. Because the program panics before sending to HEC

[Bamfax]

Hi,

thanks and understood, this explains it.

Yes, the OpenWB broker definitely emits only plain values (on all published topics), I checked with MQTT-Explorer.

Here is some more dumps.

This is the config (pulling a different topic now):

broker: tcp://192.168.1.2:1883
mqtt_username: user
mqtt_password: pw
hec_url : https://127.0.0.1:8088/services/collector
hec_token: a-b-c-d-e
client_id: mqtt_subscribe
write_to_console: true
write_to_splunk: false
topics:
  - openWB/global/WHouseConsumption
insecure_skip_verify: true

splunk-mqtt writes the following to stdout:

Connection is up
connection established
subscribed to:  openWB/global/WHouseConsumption
Message could not be parsed (703): json: cannot unmarshal number into Go value of type map[string]interface {}received message: 703
Message could not be parsed (1218): json: cannot unmarshal number into Go value of type map[string]interface {}received message: 1218
Message could not be parsed (646): json: cannot unmarshal number into Go value of type map[string]interface {}received message: 646
Message could not be parsed (653): json: cannot unmarshal number into Go value of type map[string]interface {}received message: 653

A quick tcpdump confirms that:

12:48:26.675337 IP 192.168.1.2.1883 > 192.168.10.1.54168: Flags [P.], seq 48:86, ack 100, win 216, options [nop,nop,TS val 198917621 ecr 1954098558], length 38
E..<unprintable ascii>..openWB/global/WHouseConsumption686

Do you see the possibility to let splunk-mqtt the understand and handle unformatted plain outputs?

Many thanks for your answer!

[aSauerwein]

Hi,

yes that is certainly doable. There are some options I'm considering

• try json and if that fails use string
• find a way to enumerate variable type received

I'll update you once implemented

[Bamfax]

Sounds great, many thanks. Looking forward to it!

[aSauerwein]

see b3bcaa3da8d00cf7088195d640f618a1846af317 docker image: asauerwein/splunk-mqtt:dev

my current approach is to

• check if msg is number
• try json parsing
• just convert to string

please try and tell me if that fixes your issues.

but this is not finished. I need to implement some debug flags to not show the "failed to parse json" message every time a plain string is received.

[Bamfax]

Many thanks, looking forward to test it. Allow me a little to setup a go env to build it. Or, if you could put up an interim release, I could give that a testdrive tomorrow.

[aSauerwein]

https://github.com/aSauerwein/splunk-mqtt/releases/tag/v0.1.1-alpha

[Bamfax]

Thank you for the interim build, that was a timesaver. Just tested it, it runs nicely, accepts the plain value without output to stderr and loads it via HEC into splunk as json with topic and, of course, the value. Many thanks for this adaption. This is quite a nice way to bridge mqtt into splunk with just a small footprint and using HEC as intended. Awesome, kudos!