Closed winder closed 11 months ago
Merging #146 (0e01345) into master (442791a) will increase coverage by
4.05%
. Report is 55 commits behind head on master. The diff coverage is80.81%
.
@@ Coverage Diff @@
## master #146 +/- ##
==========================================
+ Coverage 67.66% 71.71% +4.05%
==========================================
Files 32 37 +5
Lines 1976 2747 +771
==========================================
+ Hits 1337 1970 +633
- Misses 570 678 +108
- Partials 69 99 +30
Files Changed | Coverage Δ | |
---|---|---|
conduit/data/block_export_data.go | 100.00% <ø> (+92.30%) |
:arrow_up: |
conduit/metrics/metrics.go | 100.00% <ø> (ø) |
|
conduit/pipeline/metadata.go | 69.11% <ø> (ø) |
|
conduit/plugins/config.go | 100.00% <ø> (ø) |
|
...duit/plugins/exporters/filewriter/file_exporter.go | 81.63% <ø> (-1.06%) |
:arrow_down: |
conduit/plugins/importers/algod/metrics.go | 100.00% <ø> (ø) |
|
...gins/processors/filterprocessor/fields/searcher.go | 77.50% <ø> (ø) |
|
...ins/processors/filterprocessor/filter_processor.go | 83.82% <ø> (+3.54%) |
:arrow_up: |
...plugins/processors/filterprocessor/gen/generate.go | 34.28% <ø> (ø) |
|
conduit/plugins/processors/noop/noop_processor.go | 64.70% <ø> (+6.81%) |
:arrow_up: |
... and 21 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
how do you tell from the output that there's fixed 8ms overhead for each filter?
BenchmarkProcess2/txn_1_equal_0_regex_0_numeric_1000 BenchmarkProcess2/txn_1_equal_0_regex_0_numeric_1000-8 6 197892161 ns/op
It seems that way because when running 1000 filters, it takes 8 seconds. EG:
BenchmarkProcess2/txn_0_equal_1000_regex_0_numeric_0-8 1 8057424687 ns/op
197892161
But numeric filters seems to be faster.
1 filter / 1 txn takes about 8ms 1 filter / 25000 txn takes about 8ms 1000 filters / 1 txn takes about 8 seconds 1000 filters / 25000 txn takes about 8 seconds
I think it follows that the number of txn seems to be nicely optimized, but there seems to be some per-filter overhead somewhere.
197892161
But numeric filters seems to be faster.
Good point, I was definitely looking at the first few results more than the other filter types.
Summary
Add new benchmarks Benchmarks for filters.
The results suggest that we're doing something extremely inefficient when setting up each additional filter.
Applying one filter to 1 or 25000 transactions has an almost negligible difference. On the other hand there seems to be a fixed 8ms overhead for each filter.
The numbers in each report are as follows: