search

FL33TW00D / deCoreML

Find out why your CoreML model isn't running on the Neural Engine!
21 stars 2 forks source link

Validation Message Not In `analytics.mil` #2

Open kinghchan opened 1 week ago

kinghchan commented 1 week ago

This is a very useful tool, thank you for sharing it!

It looks like the validation message is not present in the analytics.mil file for a profile I just ran

I profiled the AudioEncoder.mlmodelc from this repo: https://huggingface.co/argmaxinc/whisperkit-coreml/tree/main/openai_whisper-tiny.

For context, I have pasted the analytics.mil file here.

Do you have any idea why the validation message is not present? A lot of layers are running on the GPU instead of the ANE, and I wanted to know why that's the case.

I am using a MacBook Pro 15 M1 Max. My MacOS version 14.4.1. My Xcode is version 15.4. I set the minimum deployment target of the CoreML file (during coremltools conversion) to MacOS 14.

Thank you!

analytics.mil.txt

FL33TW00D commented 1 week ago

This is a very useful tool, thank you for sharing it!

It looks like the validation message is not present in the analytics.mil file for a profile I just ran

I profiled the AudioEncoder.mlmodelc from this repo: https://huggingface.co/argmaxinc/whisperkit-coreml/tree/main/openai_whisper-tiny.

For context, I have pasted the analytics.mil file here.

Do you have any idea why the validation message is not present? A lot of layers are running on the GPU instead of the ANE, and I wanted to know why that's the case.

I am using a MacBook Pro 15 M1 Max. My MacOS version 14.4.1. My Xcode is version 15.4. I set the minimum deployment target of the CoreML file (during coremltools conversion) to MacOS 14.

Thank you!

analytics.mil.txt

Hi @kinghchan, I think in this case there is no error, it runs on the GPU because that is optimal (look at the estimated runtimes). The fact it is whisper-tiny would also provide evidence for this.

Try it with a larger whisper version and see if the results are the same.

kinghchan commented 1 week ago

Thanks a lot @FL33TW00D for your help!

Yep gonna try the large one now, but that takes ages to run... we'll see!