search

intuit / fuzzy-matcher

A Java library to determine probability of objects being similar.
Apache License 2.0
226 stars 69 forks source link

#53 upgrade to java 11 #55

Closed aavaas closed 2 years ago

aavaas commented 2 years ago

53

old test `------------------------------------------------------- T E S T S

Running com.intuit.fuzzymatcher.component.ElementMatchTest SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.179 sec Running com.intuit.fuzzymatcher.component.MatchServiceTest Data: {[{'Stephen Wilkson'}]} Matched With: {[{'Wilson, Steven'}]} Score: 1.0 Data: {[{'Stephen Wilkson'}]} Matched With: {[{'Stephen Kilsman wilkson'}]} Score: 0.6666666666666666 Data: {[{'John Pierre'}]} Matched With: {[{'Pierre john'}]} Score: 1.0 Data: {[{'Wilson, Steven'}]} Matched With: {[{'Stephen Wilkson'}]} Score: 1.0 Data: {[{'Wilson, Steven'}]} Matched With: {[{'Stephen Kilsman wilkson'}]} Score: 0.6666666666666666 Data: {[{'Pierre john'}]} Matched With: {[{'John Pierre'}]} Score: 1.0 Data: {[{'Stephen Kilsman wilkson'}]} Matched With: {[{'Stephen Wilkson'}]} Score: 0.6666666666666666 Data: {[{'Stephen Kilsman wilkson'}]} Matched With: {[{'Wilson, Steven'}]} Score: 0.6666666666666666 Data: {[{'Steven Wilson'}, {'45th Avenue 5th st.'}]} Matched With: {[{'Stephen Wilkson'}, {'45th Ave 5th Street'}]} Score: 1.0000000000000002 Data: {[{'Stephen Wilkson'}, {'45th Ave 5th Street'}]} Matched With: {[{'Steven Wilson'}, {'45th Avenue 5th st.'}]} Score: 1.0000000000000002 Data: {[{'Steven Wilson'}, {'45th Avenue 5th st.'}]} Matched With: {[{'Stephen Wilkson'}, {'45th Ave 5th Street'}]} Score: 1.0000000000000002 Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.227 sec Running com.intuit.fuzzymatcher.component.MatchServicePerfTest Max memory used (gb): 2.129387672 Execution time (ms) for transactions : 15572 Execution time (ms) for + 2000 count : 59

Execution time (ms) for + 4000 count : 197

Execution time (ms) for + 6000 count : 422

Execution time (ms) for + 8000 count : 790

Execution time (ms) for + 10000 count : 1113

Execution time (ms) for + 12000 count : 1688

Execution time (ms) for + 14000 count : 2310

Execution time (ms) for + 16000 count : 2970

Execution time (ms) for + 18000 count : 3845

Execution time (ms) for + 20000 count : 4759

Execution time (ms) for + 22000 count : 5808

Execution time (ms) for + 24000 count : 6561

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.284 sec Running com.intuit.fuzzymatcher.component.TokenRepoTest Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec Running com.intuit.fuzzymatcher.function.TokenizerFunctionTest Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec Running com.intuit.fuzzymatcher.function.ScoringFunctionTest Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.154 sec Running com.intuit.fuzzymatcher.function.PreProcessFunctionTest Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec Running com.intuit.fuzzymatcher.domain.ElementTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec Running com.intuit.fuzzymatcher.domain.DocumentTest Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec

Results :

Tests run: 77, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 48.567 s [INFO] Finished at: 2021-10-08T09:57:43-07:00 [INFO] ------------------------------------------------------------------------`

New test result

`[INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running com.intuit.fuzzymatcher.component.ElementMatchTest SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.087 s - in com.intuit.fuzzymatcher.component.ElementMatchTest [INFO] Running com.intuit.fuzzymatcher.component.MatchServiceTest Data: {[{'Stephen Wilkson'}]} Matched With: {[{'Wilson, Steven'}]} Score: 1.0 Data: {[{'Stephen Wilkson'}]} Matched With: {[{'Stephen Kilsman wilkson'}]} Score: 0.6666666666666666 Data: {[{'John Pierre'}]} Matched With: {[{'Pierre john'}]} Score: 1.0 Data: {[{'Wilson, Steven'}]} Matched With: {[{'Stephen Wilkson'}]} Score: 1.0 Data: {[{'Wilson, Steven'}]} Matched With: {[{'Stephen Kilsman wilkson'}]} Score: 0.6666666666666666 Data: {[{'Pierre john'}]} Matched With: {[{'John Pierre'}]} Score: 1.0 Data: {[{'Stephen Kilsman wilkson'}]} Matched With: {[{'Stephen Wilkson'}]} Score: 0.6666666666666666 Data: {[{'Stephen Kilsman wilkson'}]} Matched With: {[{'Wilson, Steven'}]} Score: 0.6666666666666666 Data: {[{'Steven Wilson'}, {'45th Avenue 5th st.'}]} Matched With: {[{'Stephen Wilkson'}, {'45th Ave 5th Street'}]} Score: 1.0000000000000002 Data: {[{'Stephen Wilkson'}, {'45th Ave 5th Street'}]} Matched With: {[{'Steven Wilson'}, {'45th Avenue 5th st.'}]} Score: 1.0000000000000002 Data: {[{'Steven Wilson'}, {'45th Avenue 5th st.'}]} Matched With: {[{'Stephen Wilkson'}, {'45th Ave 5th Street'}]} Score: 1.0000000000000002 [INFO] Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.235 s - in com.intuit.fuzzymatcher.component.MatchServiceTest [INFO] Running com.intuit.fuzzymatcher.component.MatchServicePerfTest Max memory used (gb): 0.41831632 Execution time (ms) for transactions : 16246 Execution time (ms) for + 2000 count : 66

Execution time (ms) for + 4000 count : 237

Execution time (ms) for + 6000 count : 470

Execution time (ms) for + 8000 count : 843

Execution time (ms) for + 10000 count : 1270

Execution time (ms) for + 12000 count : 1829

Execution time (ms) for + 14000 count : 2433

Execution time (ms) for + 16000 count : 3460

Execution time (ms) for + 18000 count : 4086

Execution time (ms) for + 20000 count : 5059

Execution time (ms) for + 22000 count : 6188

Execution time (ms) for + 24000 count : 7461

[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.838 s - in com.intuit.fuzzymatcher.component.MatchServicePerfTest [INFO] Running com.intuit.fuzzymatcher.component.TokenRepoTest [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 s - in com.intuit.fuzzymatcher.component.TokenRepoTest [INFO] Running com.intuit.fuzzymatcher.function.TokenizerFunctionTest [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.intuit.fuzzymatcher.function.TokenizerFunctionTest [INFO] Running com.intuit.fuzzymatcher.function.ScoringFunctionTest [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.092 s - in com.intuit.fuzzymatcher.function.ScoringFunctionTest [INFO] Running com.intuit.fuzzymatcher.function.PreProcessFunctionTest [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in com.intuit.fuzzymatcher.function.PreProcessFunctionTest [INFO] Running com.intuit.fuzzymatcher.domain.ElementTest [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.intuit.fuzzymatcher.domain.ElementTest [INFO] Running com.intuit.fuzzymatcher.domain.DocumentTest [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.intuit.fuzzymatcher.domain.DocumentTest [INFO] [INFO] Results: [INFO] [INFO] Tests run: 77, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 52.559 s [INFO] Finished at: 2021-10-08T12:23:24-07:00 [INFO] ------------------------------------------------------------------------`

aavaas commented 2 years ago

@manishobhatia initial results from running the tests in java11, I see a significant decrease in max memory used, and while tests with updated java and libraries are taking slightly longer, it is not significant.

java 8: Max memory used (gb): 2.129387672 java 11: Max memory used (gb): 0.41831632

manishobhatia commented 2 years ago

@aavaas The reduction in memory footprint is a welcome news. thanks for calling it out