search

huggingface / swift-chat

Mac app to demonstrate swift-transformers
Apache License 2.0
491 stars 37 forks source link

build error #23

Closed 1063910011 closed 3 months ago

1063910011 commented 3 months ago

image

pcuenca commented 3 months ago

Hello @1063910011. The preview branch, which you seem to be compiling, requires beta versions of macOS and Xcode. Could you please specify the versions you are using?

1063910011 commented 3 months ago

Yes, I use beta versions of MacOS 15.0 and Xcode Version 16.1, but build failed. image

Luckily, ios is build succeeded, but get another error when run the StatefulMistral7BInstructInt4.mlpackage .

Error Error Domain=NSCocoaErrorDomain Code=3840 "JSON text did not start with array or object and option to allow 
fragments not set. around line 1, column 0." UserInfo={NSDebugDescription=JSON text did not start with array or object and 
option to allow fragments not set. around line 1, column 0., NSJSONSerializationErrorIndex=0}

Ios version is iOS 18.1 (22B5023e)

image

pcuenca commented 3 months ago

Hi @1063910011. A couple of notes about the experimental preview branch:

Regarding the issue you reported, it could be the case (but I'm not sure) that you need to launch the app setting the environment variable HUGGING_FACE_HUB_TOKEN with your Hugging Face token. This is because the tokenizer is retrieved from the original model repo, which requires authenticated access.

1063910011 commented 3 months ago

Helpful, it works after I set the environment variable HUGGING_FACE_HUB_TOKEN

But it is very slow on the iPhone 15 Promax. It seems to be computed by CPU and GPU only by default. image

I am trying to modify computeUnits to .all to load model, then load failed, get error: no model could be loaded. I selected the same model StatefulMistral7BInstructInt4.mlpackage image

sekharchandra commented 3 months ago
Screenshot 2024-08-16 at 11 58 41 AM

Hi There, Even after adding the HUGGING_FACE_HUB_TOKEN, unable to generate the content. Anything am I missing?

pcuenca commented 3 months ago

Hi @sekharchandra, it should work with the envvar as @1063910011 reported. You need to add it here:

Screenshot 2024-08-16 at 09 44 32

Regarding iOS, the model has been optimized for GPU on Mac. Good performance on iPhone would require making it compatible with Neural Engine, we'll be running some tests soon!

pcuenca commented 3 months ago

The original issue (build error) is fixed, closing this now. Feel free to open additional issues for discussion as you need to!