cloudfoundry-community / firehose-to-syslog

Send firehose events from Cloud Foundry to syslog.
MIT License
44 stars 58 forks source link

HttpStartStop invalid app id #219

Closed sba30 closed 4 years ago

sba30 commented 4 years ago

Since we have started using the 5.1.0 firehose-to-syslog release, we are now seeing invalid app id's for HttpStartStop events.

<6> 2019-10-29T10:24:00Z 0a42fb23-3298-4b4d-b240-e1149a6ec137 doppler[21517]: {"cf_app_id":"00000000-0000-0000-0000-000000000000","cf_origin":"firehose","content_length":0,"deployment":"","duration_ms":6,"event_type":"HttpStartStop" Could this be an issue with the loggregator v1 -> v2 change?
Scoobed commented 4 years ago

As we investigated this, we back tracked the code from Firehose_to_syslog to go-loggegrator

Conversion to V1 starts here https://github.com/cloudfoundry-community/firehose-to-syslog/blob/d7b57b3894bf9f67047741ccd8f7b04bf67539f6/firehoseclient/v2adapter.go#L63

Code which I think actually has an issue.. I don't have a good simple example to pull a message from loggegrator v2 api endpoint to see what is happening. On the other messages Source ID is string but on the timer conversation it has a different conversion method. Not sure if that is what is causing the issue, if something undocumented changed in loggegrator v2 in later versions.

https://github.com/cloudfoundry/go-loggregator/blob/3a9c7ed9ada2f3a687303e3bcccb06bcaf2cb457/conversion/tov1.go#L101

Should it use this instead https://github.com/cloudfoundry/loggregator/blob/3ce1eab57797e49c03fd9c262168c28b66d7a560/plumbing/conversion/tov1.go#L85

Which points to what the structure should be?
https://github.com/cloudfoundry/loggregator-api/blob/master/README.md#httpstartstop