search

ml-explore / mlx-swift-examples

Examples using MLX Swift
MIT License
1.01k stars 107 forks source link

Is there a good way to load the Flux.1 T5 tokenizer? #136

Closed mzbac closed 1 month ago

mzbac commented 1 month ago

Maybe this is not directly related to the mlx swift example, but I am having difficulty porting Flux.1 to mlx swift due to It can't load the T5 tokenizer for Flux.1. I'm wondering if there are any alternatives or pointers that could help us work around this issue ?

davidkoski commented 1 month ago

I think this would be an issue for https://github.com/huggingface/swift-transformers -- this package provides all the tokenizer support (there is a little bit of mapping done in mlx-swift-examples but all the real work happens in swift-transformers)

davidkoski commented 1 month ago

Ah, indeed you saw: https://github.com/huggingface/swift-transformers/issues/127 :-)

mzbac commented 1 month ago

Thanks, @davidkoski. I was thinking maybe we could find an easy workaround. It seems like the swift -transformers team is quite busy at the moment and may not be able to help with that right now :)

davidkoski commented 1 month ago

OK, so swift-transformers does have a T5 tokenizer:

If it is simply a matter of naming you can supply an override:

though note that #126 has some adjustments in this area (for dynamic manipulation of the overrides). You can see if that does the job.

mzbac commented 1 month ago

OK, so swift-transformers does have a T5 tokenizer:

If it is simply a matter of naming you can supply an override:

though note that #126 has some adjustments in this area (for dynamic manipulation of the overrides). You can see if that does the job.

It seems the swift-transformers missed a Strip normalizer. I will try to get that added, and then it should be all good. Thanks for the heads up for the dynamic manipulation of the overrides :)

awni commented 1 month ago

Nice thanks @davidkoski ! I'm really looking forward to the FLUX port!!