Closed osma closed 1 year ago
Kudos, SonarCloud Quality Gate passed!
0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells
No Coverage information
0.0% Duplication
Base: 99.56% // Head: 99.56% // Decreases project coverage by -0.01%
:warning:
Coverage data is based on head (
1c587bd
) compared to base (cc6dfcf
). Patch coverage: 100.00% of modified lines in pull request are covered.
:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
This PR implements
_suggest_batch
instead of single-document_suggest
in the SVC backend, allowing the use of more efficient vector operations.I tested this with the 20 Newsgroups data set example, as shown on the wiki page of the SVC backend. Evaluation on the test set is 20-30% faster with the results otherwise unchanged.
There is one special case that requires attention: what to do when the input to the SVC classifier is empty or near-empty. The old
_suggest
code handled this as a special case so it short-circuits the classifier and returns an empty result. There is also a unit test that tests this (using the special input text "j" which is an unknown token and thus equivalent to empty input). The SVC model has no problem returning some class (maybe the majority class?) in this case, but I reimplemented the special case check for empty input to retain the old behaviour that refuses to classify it and keeps the unit test happy.With 1 job
With 4 jobs
Fixes #667