Bouncer lastPull strategy change (Allow bouncer to use stream mode while sharing same API key)

[buixor]

What would you like to be added?

Currently, bouncer sharing the same API key cannot reliably use the stream mode.

What happens

When a bouncer queries /decisions/stream with startup not set to true, we rely on bouncerInfo.LastPull timestamp to know when the bouncer last pulled and know which data should be returned. the bouncerInfo is fetched from the DB and only relies on the API key provided by the bouncer.

Problem

However, this is problematic when several bouncer instances (ie. nginx-ingress / kube) share the same API key and want to use stream mode: they get inconsistent data as they all share the same entry in the DB and thus LastPull.

Proposed change

The proposed change is the following:

• Use combo IP + Api Key as the unique identified for bouncers (instead of just API key)
• When a "new" bouncer (existing api key but new IP) comes in, after authentication, we duplicate the existing row (based on api key) and we update last pull, heartbeat etc.
• When a bouncer does a queries /decisions/stream with startup not set to true, we rely on the Api key + IP combo to select the relevant row, so that each bouncer instance get its own LastPull

Why is this needed?

Allow several bouncers to share the same API key and use stream mode.

[github-actions[bot]]

@buixor: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.

[github-actions[bot]]

@buixor: There are no 'kind' label on this issue. You need a 'kind' label to start the triage process.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind bug
• /kind packaging

Details

I am a bot created to help the [crowdsecurity](https://github.com/crowdsecurity) developers manage community feedback and contributions. You can check out my [manifest file](https://github.com/crowdsecurity/crowdsec/blob/master/.github/governance.yml) to understand my behavior and what I can do. If you want to use this for your project, you can check out the [BirthdayResearch/oss-governance-bot](https://github.com/BirthdayResearch/oss-governance-bot) repository.

[rohow]

HI~ Is there any related fix plan for this issue? I am currently using multi-agent mode in ingress-nginx and there will be problems when updating in stream mode.

I think it should be related to this. I haven't found a way to use helm deployment to allow each agent to have a different api key.

[LaurenceJJones]

HI~ Is there any related fix plan for this issue? I am currently using multi-agent mode in ingress-nginx and there will be problems when updating in stream mode.

I think it should be related to this. I haven't found a way to use helm deployment to allow each agent to have a different api key.

Hey 👋🏻 Yes we have floated some ideas internally around how we can move forward on this issue (we added them above in the proposed solutions) and yes the reason behind the issue was when we helping enterprise clients on a k8s environments without moving them over to mTLS authentication it currently does not allow stream mode with the API key.