search

philterd / phileas

The open source PII and PHI redaction engine
https://www.philterd.ai
Apache License 2.0
23 stars 5 forks source link

Have performance tests run during the build #141

Open jzonthemtn opened 2 months ago

jzonthemtn commented 2 months ago

Have performance tests run during the build and fail the build if a decrease in critical metrics.

RobDickinson commented 2 months ago

@jzonthemtn wonder if adding phileas-benchmark as a Phileas submodule would make sense?

In the last week or so I've cleaned up phileas-benchmark -- before it had some test cases for issues that we were trying to fix, and some domain-specific filtering logic that I was trying out. (phileas-benchmark is now pretty generic and not as specific to the "resurfaceio" usage of Phileas as it was early on)

Anyway, I was gonna ask you about perhaps moving phileas-benchmark over to the philterd org in GitHub -- but if you're looking to make benchmarking part of the regular build, might be better as a submodule?

ps. Because we have a few other integrations working now, we'll likely be focusing more on end-to-end benchmarking -- so I won't be bothered if phileas-benchmark morphs into something completely different that produces different results. I don't need backwards compatibility here so wouldn't want that to hold us back.

jzonthemtn commented 2 months ago

I think a submodule would be good. If it was Philter with an API I might say a separate repository to build as a docker image and run against the API but I think that's overly complicating it for what there is now. I'd love to include it and set up gates around it!