Closed percevalw closed 6 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 97.00%. Comparing base (
6bfa4cc
) to head (a859540
).
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Changelog
v0.10.6
Added
batch_by
,split_into_batches_after
,sort_chunks
,chunk_size
,disable_implicit_parallelism
parameters to processing (simple
andmultiprocessing
) backends to improve performance and memory usage. Sorting chunks can improve yield up to twice the speed in some cases.max_tokens_per_device="auto"
parameter toeds.transformer
to estimate memory usage and automatically split the input into chunks that fit into the GPU.Changed
eds.text_cnn
pipe by running the CNN on a non-padded version of its input: expect a speedup up to 1.3x in real-world use cases.bool_attributes
parameter in favor of generaldefault_attributes
. This new mapping describes how to set attributes on spans for which no attribute value was found in the input format. This is especially useful for negation, or frequent attributes values (e.g. "negated" is often False, "temporal" is often "present"), that annotators may not want to annotate every time.eds.ner_crf
window is now set to 40 and stride set to 20, as it doesn't affect throughput (compared to before, window set to 20) and improves accuracy.overlap_policy='merge'
option and parameter renaming ineds.span_context_getter
(which replaceseds.span_sentence_getter
)Fixed
multiprocessing
backend (e.g., no more deadlock)eds.ner_crf
component are now disabled when windows are used (e.g., neitherwindow=1
equivalent to softmax andwindow=0
equivalent to default full sequence Viterbi decoding)eds
tokenizer nows inherits fromspacy.Tokenizer
to avoid typing errorsu[ne] cure de 10 jours
Pipeline.preprocess
methodeds.hyphothesis
Checklist