Refactor hgf tokenizer code

[codecov[bot]]

Codecov Report

Base: 19.55% // Head: 19.11% // Decreases project coverage by -0.43% :warning:

Coverage data is based on head (d901f79) compared to base (ba8eded). Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #114 +/- ## ========================================== - Coverage 19.55% 19.11% -0.44% ========================================== Files 78 80 +2 Lines 3974 4064 +90 ========================================== Hits 777 777 - Misses 3197 3287 +90 ``` | [Impacted Files](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter) | Coverage Δ | | |---|---|---| | [src/huggingface/HuggingFace.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL0h1Z2dpbmdGYWNlLmps) | `0.00% <ø> (ø)` | | | [src/huggingface/download.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2Rvd25sb2FkLmps) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/bert/tokenizer.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2JlcnQvdG9rZW5pemVyLmps) | `0.00% <0.00%> (ø)` | | | [src/huggingface/implementation/gpt2/tokenizer.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL2ltcGxlbWVudGF0aW9uL2dwdDIvdG9rZW5pemVyLmps) | `0.00% <0.00%> (ø)` | | | [src/huggingface/tokenizer/fast\_tkr.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL3Rva2VuaXplci9mYXN0X3Rrci5qbA==) | `0.00% <0.00%> (ø)` | | | [src/huggingface/tokenizer/slow\_tkr.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL3Rva2VuaXplci9zbG93X3Rrci5qbA==) | `0.00% <0.00%> (ø)` | | | [src/huggingface/tokenizer/tokenizer.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL3Rva2VuaXplci90b2tlbml6ZXIuamw=) | `0.00% <0.00%> (ø)` | | | [src/huggingface/tokenizer/utils.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL3Rva2VuaXplci91dGlscy5qbA==) | `0.00% <0.00%> (ø)` | | | [src/huggingface/utils.jl](https://codecov.io/gh/chengchingwen/Transformers.jl/pull/114/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter#diff-c3JjL2h1Z2dpbmdmYWNlL3V0aWxzLmps) | `0.00% <0.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Peter)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.