search

huggingface / swift-transformers

Swift Package to implement a transformers-like API in Swift
Apache License 2.0
708 stars 77 forks source link

StableCode Fails During Inference (stabilityai/stablecode-completion-alpha-3b-4k) #13

Closed JustinMeans closed 1 year ago

JustinMeans commented 1 year ago

stabilityai/stablecode-completion-alpha-3b-4k fails at the assert here but converts successfully using: python -m exporters.coreml --model=stabilityai/stablecode-completion-alpha-3b-4k exported/

Screenshot 2023-08-09 at 3 44 43 AM

It's a gpt_neox model with GPTNeoXForCausalLM architecture.

When removing the assert and re-running, the SwiftChat app is able to load and run the model, but generates gibberish.

Screenshot 2023-08-09 at 3 46 51 AM

I have the converted model currently uploading to HF but it's going to take a bit to complete.

pcuenca commented 1 year ago

Thanks for the report, @JustinMeans!

The default task in exporters is feature-extraction: https://github.com/huggingface/exporters/blob/80d37d8004e31a9cbd00ce4bc5ef261c175f62ce/src/exporters/coreml/__main__.py#L83

Could you please try to convert adding --feature text-generation to your command line?

(We should probably use different defaults in exporters based on the model type).

pcuenca commented 1 year ago

Closing for now, feel free to reopen if appropriate!