search

nomadkaraoke / python-audio-separator

Easy to use vocal separation from CLI or as a python package, using a variety of amazing pre-trained models (primarily from UVR)
MIT License
403 stars 65 forks source link

Utilising ANE (Apple Neural Engine) for inference #93

Open dj-nuo opened 1 month ago

dj-nuo commented 1 month ago

Hi! I know that PyTorch can use MPS to accelerate inference on Apple computers, but ONNX provides ability to use ANE (Apple Neural Engine) as well. Have you tried converting BS-Roformer or Demucs to ONNX or CoreML?

My previous attempts to do so were unsuccessful(

beveradb commented 1 month ago

Hey, thanks for the heads up! That's pretty cool - I haven't tried it, but if you could get the models which already use ONNX (eg. UVR_MDXNET_KARA_2.onnx, UVR-MDX-NET-Inst_HQ_3.onnx, UVR-MDX-NET-Inst_HQ_4.onnx) to work with ANE that would probably be easier than converting the model architecture of one of the others.

However, I'm not sure I really understand the goal - at least on my personal Macbook with an M3 Max, all of the models in audio-separator which I use frequently already use my GPU resources fully!

dj-nuo commented 1 month ago

The goal is to utilize ANE) GPU is powerful, but not as powerful for specific calculations as ANE)

beveradb commented 1 month ago

Gotcha; I'm afraid I don't know where to start with that (I'd imagine it needs work in PyTorch first?) - PRs very welcome though if you can figure it out! 🙏