In reviewing the v1 PR, there were some issues with filtering which were difficult to map because of how the code was structured. This issue documents the changes made in order to resolve that issue, and fixup some of the issues in the filtering feature which can be found in this PR.
Commits referenced below will be squashed and rebased - but keeping the references here in order to refer to the specific changes in turn.
Additionaly, the following are things that I think we should improve, but shouldn't block v1 release for. Documenting them here in the name of completeness:
In reviewing the v1 PR, there were some issues with filtering which were difficult to map because of how the code was structured. This issue documents the changes made in order to resolve that issue, and fixup some of the issues in the filtering feature which can be found in this PR.
Commits referenced below will be squashed and rebased - but keeping the references here in order to refer to the specific changes in turn.
This also changes the API for unstruct filters here to make the checks on event name and version explicit on configuration.
Factor out compiling regexes so that this happens only once on startup: https://github.com/snowplow-devops/stream-replicator/pull/185/commits/b2c0d683605cbc7f3ab68f213dc679445d85c627
When no value is returned, the filter always evaluates to
false, and when a nil value is returned we don't match against nil, but against the string<nil>: https://github.com/snowplow-devops/stream-replicator/pull/185/commits/e643a526528fb9eda9e7fa4fa279bcfc7ba869b1Use
Compile()instead ofMustCompile()to avoid panics: https://github.com/snowplow-devops/stream-replicator/pull/185/commits/61dbba8befc650d5a9b2d87626234b180d35ac55Issue with the analytics sdk's API (https://github.com/snowplow/snowplow-golang-analytics-sdk/issues/36) leads to error when value doesn't exist, rather than simply matching nil. https://github.com/snowplow-devops/stream-replicator/pull/185/commits/db5841e82f76b5a0c55e7b80ec219483a12eec18
Incorrect tests: https://github.com/snowplow-devops/stream-replicator/pull/185/commits/5dcaaa4881f6d04787bc9501d2897f8bf39b0359 and https://github.com/snowplow-devops/stream-replicator/pull/185/commits/b77d24456cfdd8afb59cd1bc59b879780f424efa
Additionaly, the following are things that I think we should improve, but shouldn't block v1 release for. Documenting them here in the name of completeness: