App Fatal Error When Last Model is Missing

[Proryanator]

Love this app, gonna be really useful in my research. I had tried using a small model a few months back (tried to add it from my downloads) and the app froze, so I removed the model and tried to add a new one. However upon re-running the app, even after fresh clones of the repo, the app immediately crashes here when trying to load the model:

I've isolated this to when the app is trying to load the lastCompiledModel pulling data from Application Support/hf-compiled-transformers. If the shortcut/reference to the last loaded model is gone, the app doesn't handle this very well and crashes.

Could be a small improvement for this but, the fix is to wipe that hf-compiled-transformers folder altogether. But could add in some error checking to be able to handle if the model that you previously had loaded is missing.

[xavi0007]

I am getting this problem too.

[sgallese]

Fix: switching to the preview branch worked in my case

Issue:

I'm running issues in gettingStatefulMistral7BInstructInt4.mlpackage to work with Sequoia 15.0.1 on the Mac.

But when loading the model, I get Cannot obtain shape information within LanguageModel.init

This is the output of model at that point

(lldb) po model
modelDescription: 
functionName: (null)
inputs: (
    "causalMask : MultiArray (Float16, 1 \U00d7 1 \U00d7 1 \U00d7 1)",
    "inputIds : MultiArray (Int32, 1 \U00d7 1)"
)
outputs: (
    "logits : MultiArray (Float16, )"
)
states: (
    "valueCache : State (Float16, 32 \U00d7 1 \U00d7 8 \U00d7 2048 \U00d7 128)",
    "keyCache : State (Float16, 32 \U00d7 1 \U00d7 8 \U00d7 2048 \U00d7 128)"
)
predictedFeatureName: (null)
predictedProbabilitiesName: (null)
classLabels: (null)
updatable: NO
trainingInputs: (
)
parameters: (
)
metadata: {
    MLModelAuthorKey = "";
    MLModelCreatorDefinedKey =     {
        "co.huggingface.exporters.name" = "mistralai/Mistral-7B-Instruct-v0.3";
        "com.github.apple.coremltools.source" = "torch==2.3.1";
        "com.github.apple.coremltools.source_dialect" = TorchScript;
        "com.github.apple.coremltools.version" = "8.0b1";
    };
    MLModelDescriptionKey = "";
    MLModelLicenseKey = "";
    MLModelVersionStringKey = "";
}, 
configuration: 
 computeUnits: GPU and CPU,             
 useWatchSPIForScribble: NO,             
 allowLowPrecisionAccumulationOnGPU: NO,             
 allowBackgroundGPUComputeSetting: NO,             
 preferredMetalDevice: (null),             
 enableTestVectorMode: NO,             
 parameters: (null),             
 rootModelURL: file:///Users/sgallese/Library/Application%20Support/hf-compiled-transformers/StatefulMistral7BInstructInt4.mlmodelc/,             
 profilingOptions: 0,             
 usePreloadedKey: NO,             
 trainWithMLCompute: NO,             
 parentModelName: ,             
 modelName: StatefulMistral7BInstructInt4,             
 experimentalMLE5EngineUsage: Enable,             
 preparesLazily: NO,             
 predictionConcurrencyHint: 0,             
 usePrecompiledE5Bundle: NO,             
 experimentalMLE5BNNSGraphBackendUsage: Enable,             
 e5rtMutableMILWeightURLs: {
}             
 e5rtDynamicCallableFunctions: {
}             
 optimizationHints: MLOptimizationHints:
 reshapeFrequency Frequent
 hotHandDuration 1.00 seconds
 specializationStrategy Default
,             
 functionName: (null),             
 experimentalMLE5BNNSGraphBackendUsageMultiSegment: Enable,             
 e5rtComputeDeviceTypeMask: 0xffffffffffffffff,             
 e5rtCustomANECompilerOptions: (null),             
 serializesMILTextForDebugging: NO,    

swift-transformers has lots of fatalErrors instead of simply throwing a regular error: https://github.com/search?q=repo%3Ahuggingface%2Fswift-transformers%20fatalerror&type=code

We could propose to modify that library to throws on LanguageModel init, but it is a breaking change that impacts consumers of the library.

[Proryanator]

My thoughts as well on catching the missing model and handling it a bit better 👍