search

jzhang38 / TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
Apache License 2.0
7.71k stars 453 forks source link

Is it supported to convert to Apple CoreML? I tried to use coremltools to convert this model to CoreML format, but encountered an error. #102

Open HoodL opened 10 months ago

HoodL commented 10 months ago

The error message is as follows: raise NotSupportedError(r, "Comprehension ifs are not supported yet") torch.jit.frontend.NotSupportedError: Comprehension ifs are not supported yet: File "/Users/Lucas/Library/Python/3.9/lib/python/site-packages/transformers/models/llama/modeling_llama.py", line 947 next_cache = next_decoder_cache if use_cache else None if not return_dict: return tuple(v for v in [hidden_states, next_cache, all_hidden_states, all_self_attns] if v is not None) return BaseModelOutputWithPast( last_hidden_state=hidden_states,

VatsaDev commented 10 months ago

This project was meant to be trained and inferenced on a GPU. It does extend to CPU via GGUF, but there is no real support for CoreML/Metal, beyond the standard levels from lit-gpt