armiol commented 2 years ago

Before this changeset, a launchTestProtoData task with the validation extensions took whooping 23 minutes to complete for the test Proto definitions in core-java/server (about 210 Proto files). This was unacceptable.

This PR improves the performance of the validation features.

So far, the improvements made reduced the execution time to about 3.5 minutes. Which is still too much, but significantly better than before.

Another, a minor one, part of the improvements is made via spine-server artifact. Its ColumnMapping performance optimization would allow to shave another 1.5 minutes off the execution time. See core-java#1457 for more details. The validation library is migrated to these core-java changes as well.

armiol commented 2 years ago

@dmdashenkov PTAL.

The builds will pass after core-java PR is merged.

codecov[bot] commented 2 years ago

Codecov Report

Merging #29 (942b269) into master (9d85858) will decrease coverage by 0.13%. The diff coverage is 0.00%.

@@             Coverage Diff             @@
##             master     #29      +/-   ##
===========================================
- Coverage      9.16%   9.02%   -0.14%     
  Complexity       46      46              
===========================================
  Files            61      62       +1     
  Lines          1441    1462      +21     
  Branches        127     130       +3     
===========================================
  Hits            132     132              
- Misses         1295    1316      +21     
  Partials         14      14

SpineEventEngine / validation

Optimize the performance #29

Codecov Report