search

chengchingwen / Transformers.jl

Julia Implementation of Transformer models
MIT License
523 stars 74 forks source link

Handle post-processor from fast tokenizer and add some tokenizers #115

Closed chengchingwen closed 2 years ago

codecov[bot] commented 2 years ago

Codecov Report

Base: 19.23% // Head: 17.79% // Decreases project coverage by -1.44% :warning:

Coverage data is based on head (52672dc) compared to base (442c49c). Patch coverage: 4.86% of modified lines in pull request are covered.

:exclamation: Current head 52672dc differs from pull request most recent head 1420563. Consider uploading reports for the commit 1420563 to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #115 +/- ## ========================================== - Coverage 19.23% 17.79% -1.45% ========================================== Files 80 87 +7 Lines 4039 4176 +137 ========================================== - Hits 777 743 -34 - Misses 3262 3433 +171 ``` | [Impacted Files](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter) | Coverage Δ | | |---|---|---| | [src/gpt/textencoder.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2dwdC90ZXh0ZW5jb2Rlci5qbA==) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/bart/config.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2JhcnQvY29uZmlnLmps) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/bart/tokenizer.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2JhcnQvdG9rZW5pemVyLmps) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/clip/config.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2NsaXAvY29uZmlnLmps) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/clip/tokenizer.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2NsaXAvdG9rZW5pemVyLmps) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/gpt/config.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2dwdC9jb25maWcuamw=) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/gpt/tokenizer.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2dwdC90b2tlbml6ZXIuamw=) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/gptj/config.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2dwdGovY29uZmlnLmps) | `0.00% <0.00%> (ø)` | | | [src/huggingface/models/base.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL21vZGVscy9iYXNlLmps) | `0.00% <ø> (ø)` | | | [src/huggingface/tokenizer/fast\_tkr.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL3Rva2VuaXplci9mYXN0X3Rrci5qbA==) | `0.00% <0.00%> (ø)` | | | ... and [4 more](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/115/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter) | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.