davidkoski
commented
1 month ago This is a tricky one -- you need to make sure there is no eval activity going on when you exit, including asyncEval. I think this means something like:
- hook into the app shutdown mechanism and stop evaluations
- a way to stop the LLM eval
- a way to wait for any existing LLM eval to be done, including asyncEval
@davidkoski menioned this in https://github.com/ml-explore/mlx-swift/issues/139, and I encountered it today: Quitting the app while generating text will result in a crash. You can easily reproduce this with LLMEval from mlx-swift-examples.