search

apache / openwhisk-package-kafka

Apache OpenWhisk package for communicating with Kafka or Message Hub
https://openwhisk.apache.org/
Apache License 2.0
32 stars 43 forks source link

Verify trigger fire by sideeffect. #272

Closed cbickel closed 6 years ago

cbickel commented 6 years ago

Until now, we verify the kafkaprovider, by looking if the action in OpenWhisk has been executed with activation list. The problem with this call is, that the activation may not be in the list in time, because it only returns the result of an CouchDB view. If there is much load on the database, the view-computation may be behind and not return the activation. As side effect, we use a trigger creation. The name of the trigger will be read from the kafka message. All other things (like credentials, ...) are already present in the action, that is invoked anyway. To check if the trigger exists, we use the ID of the trigger. So no view is involved anymore.

jasonpet commented 6 years ago

@abaruni - can you please review this? I can merge once you approve it.

abaruni commented 6 years ago

@jasonpet LGTM. feel free to merge when you have time

csantanapr commented 6 years ago

@cbickel This is very cool solution. At one point I discussed with @jasonpet to have the action to write to another DB and check the entry there. But this way we re-use the whisk DB with an entity, it can be a trigger, a package, or action.

@mdeuser The current verification is nondeterministic and problematic because couchdb views are eventually consistent, and there could be large delay until the view is updated and the activation record to show up.