search

ml-explore / mlx

MLX: An array framework for Apple silicon
https://ml-explore.github.io/mlx/
MIT License
17.48k stars 1.01k forks source link

Add orthogonal initialization #1586

Open noahfarr opened 1 week ago

noahfarr commented 1 week ago

Hi,

I would like to use orthogonal initialization as I would like to use it for a project of mine. Is it planned to add it in the near future?

awni commented 1 week ago

There is no plan that I know of to add it. If you are up for it you can send a PR. Otherwise we can add it soon.. I don't think it should be too difficult.

noahfarr commented 1 week ago

Its not trivial as you need e.g. qr decomposition and idk if thats implemented in mlx, so you either have to use some third party package or implement that as well

awni commented 1 week ago

We have a QR decomposition. The main limitation is it runs on the CPU.. so that part of the initialization needs to run on the CPU.

noahfarr commented 1 week ago

Ah very nice. Then I maybe will put together a pull request in a few days