search

chengchingwen / Transformers.jl

Julia Implementation of Transformer models
MIT License
523 stars 74 forks source link

Phi #169

Closed pevnak closed 4 months ago

pevnak commented 8 months ago

This is a follower of pull-request #168. It contains those changes adding phi models and I have also added a rudimentary support of loading safetensors. I can make a proper repository for SafeTensors if needed, but this is a quick fix. I have tested phi_1, phi_1_5 and phi_2 (for which I need safetensors) to the python version and outputs of all intermediate layers are equal

chengchingwen commented 8 months ago

I can make a proper repository for SafeTensors if needed

That would be better

pevnak commented 7 months ago

The package SafeTensors.jl will be registered in 3 days and then I will finish this.

pevnak commented 7 months ago

It should be fine now. SafeTensors library is registered and and I have uncommented test of the phi model.

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 59.07692% with 133 lines in your changes are missing coverage. Please review.

Project coverage is 47.82%. Comparing base (91a3fe0) to head (3bb4a58).

Files Patch % Lines
src/device.jl 0.00% 72 Missing :warning:
src/huggingface/weight.jl 58.53% 34 Missing :warning:
src/huggingface/implementation/phi/load.jl 91.24% 12 Missing :warning:
src/huggingface/implementation/llama/attention.jl 0.00% 5 Missing :warning:
src/huggingface/implementation/llama/load.jl 0.00% 3 Missing :warning:
src/huggingface/download.jl 0.00% 2 Missing :warning:
ext/TransformersAMDGPUExt/TransformersAMDGPUExt.jl 0.00% 1 Missing :warning:
ext/TransformersCUDAExt/TransformersCUDAExt.jl 0.00% 1 Missing :warning:
ext/TransformersMetalExt/TransformersMetalExt.jl 0.00% 1 Missing :warning:
src/huggingface/implementation/phi/config.jl 75.00% 1 Missing :warning:
... and 1 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #169 +/- ## ========================================== + Coverage 46.52% 47.82% +1.29% ========================================== Files 85 90 +5 Lines 4400 4659 +259 ========================================== + Hits 2047 2228 +181 - Misses 2353 2431 +78 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.