cloudfoundry-community / firehose-to-syslog

Send firehose events from Cloud Foundry to syslog.
MIT License
44 stars 58 forks source link

F2S_DISABLE_LOGGING = true -- v5.0.0 -- does not disable logging #208

Closed Scoobed closed 5 years ago

Scoobed commented 5 years ago

F2S_DISABLE_LOGGING = true -- does not disable logging.

Does not set IgnoredApp to true in the boltdb.

Thus we this flag to stop flooding, but as it did not work. Only option was to stop the App, to stop the flooding.

shinji62 commented 5 years ago

Should work, the issue maybe could be the time of the apps info are reloaded in boltDB or memory.

geofffranks commented 5 years ago

The env var was set over weekend, and it was still not recognizing it Monday. Did something change in the 5.0.0 release to how the environment data is gathered from CF?

Scoobed commented 5 years ago

Ok when digging through the caching code...

It looks to never goes back to CF and recache it up again due to rv.TTL.After

https://github.com/cloudfoundry-community/firehose-to-syslog/blob/2c5a3f2a6016d31efdbbfd97dbb7ba95ccdabfc6/caching/caching_lazyfill.go#L96-L108

If we were to change the rv.TTL.Before then it would re-lookup it when the TTL has expired...

var rv entity
err = c.store.Get(makeCacheStorageKey(entityType, uuid), &rv)
switch err {
case nil:
    if rv.TTL.Before(time.Now()) {
        return &rv, nil
                     //continue
    }
    // else continue
case ErrKeyNotFound:
    // continue
default:
    return nil, err
}
shinji62 commented 5 years ago

It should go back, but randomly as cache will be invalidated https://github.com/cloudfoundry-community/firehose-to-syslog/blob/develop/caching/caching_lazyfill.go#L153

Scoobed commented 5 years ago

Is there a way to test that, because that is not what we are seeing? As we pulled the boltdb from the VM and looked at it

jmprice commented 5 years ago

The documentation says to create a UAA client with: --authorities doppler.firehose,cloud_controller.global_auditor

But apparently "cloud_controller.global_auditor" doesn't have sufficient authority to read the application environment vars so the IgnoredApp property in the BoltDB is never changed to true.

Changing your UAA Client to have "cloud_controller.admin_read_only" allows the app to be ignored again.

Scoobed commented 5 years ago

We will give it a try and see it fixes the issue

Scoobed commented 5 years ago

The above fixes the problem -- we recreated the UUA client with the correct permission and I used that setting today to stop an apps logging. -- I did have to cause a cf restage [app_name] to pick up the environmental setting