search

logzio / sawmill

Sawmill is a JSON transformation Java library
Apache License 2.0
116 stars 24 forks source link

DEV-32671 Grok processing should fail faster on timeout interruption #304

Closed DanMelman closed 2 years ago

DanMelman commented 2 years ago
  1. Add interruption checks before each grok match, to fail processing faster in case of interruption.
  2. Bump org.jruby.joni library version to gain some internal changes that will allow faster interruption of grok matching (https://github.com/jruby/joni/commit/35ddadc301453f7ac2119a44e22d13caba717eaa)
alexpalchuk commented 2 years ago

@DanMelman any chance it is reproducible so we can write proper test?

DanMelman commented 2 years ago

@DanMelman any chance it is reproducible so we can write proper test?

There is a test for the interruption itself -testMatchInterrupted. Is that what you mean @alexpalchuk ?

alexpalchuk commented 2 years ago

@DanMelman any chance it is reproducible so we can write proper test?

There is a test for the interruption itself -testMatchInterrupted. Is that what you mean @alexpalchuk ?

I mean to have test that fails without fix and passes with fix

DanMelman commented 2 years ago

Looking good :)

didn't we want to add Thread.interrupted check here as well?

@matvey-mtn I ended up adding it inside Grok::matches so it is checked on each iteration anyway

matvey-mtn commented 2 years ago

Looking good :) didn't we want to add Thread.interrupted check here as well?

@matvey-mtn I ended up adding it inside Grok::matches so it is checked on each iteration anyway

ah, ok. i missed it

DanMelman commented 2 years ago

@matvey-mtn @alexpalchuk I added the test, it fails on master - https://github.com/logzio/sawmill/pull/304/commits/224865120614483734e66584b5368c1f228dd77b