Kan implementation

[Goekdeniz-Guelmez]

This adds the KAN (Kolmogorov-Arnold Network) example to mlx.

[awni]

I think this model is a little too niche still to add to our examples. I'd encourage you to put it in a standalone repo and we'd be happy to point people to it!

[Goekdeniz-Guelmez]

I think this model is a little too niche still to add to our examples. I'd encourage you to put it in a standalone repo and we'd be happy to point people to it!

Sure thing! Thank's for your feedback. I’ll create a standalone repo for the KAN model and keep you updated!

[awni]

Awesome!! Thanks!

[Goekdeniz-Guelmez]

hey awni, so I created a repo https://github.com/Goekdeniz-Guelmez/mlx-kan.git, I will later release a python package too, I'm stuck on the github action not creating the package, because its not installing the mlx package :D.

[awni]

What's the error message look like? You are using macOS in the workflow right?

[Goekdeniz-Guelmez]

You mean the runs-on setting in the python-publish.yml file? Yes its set on macos here is the full file:

# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries

name: Upload Python Package

on:
  release:
    types: [published]

permissions:
  contents: read

jobs:
  deploy:

    runs-on: macos-latest

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python
      uses: actions/setup-python@v3
      with:
        python-version: '3.10'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install build
    - name: Build package
      run: python -m build
    - name: Publish package
      uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
      with:
        user: __token__
        password: ${{ secrets.PYPI_API_TOKEN }}

Here is the error it gives me when building the package:

* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools >= 40.[8](https://github.com/Goekdeniz-Guelmez/mlx-kan/actions/runs/10123879697/job/27997615269#step:6:9).0
* Getting build dependencies for sdist...
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
    main()
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 308, in get_requires_for_build_sdist
    return hook(config_settings)
  File "/tmp/build-env-i3q3uvn[9](https://github.com/Goekdeniz-Guelmez/mlx-kan/actions/runs/10123879697/job/27997615269#step:6:10)/lib/python3.8/site-packages/setuptools/build_meta.py", line 330, in get_requires_for_build_sdist
    return self._get_build_requires(config_settings, requirements=[])
  File "/tmp/build-env-i3q3uvn9/lib/python3.8/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
    self.run_setup()
  File "/tmp/build-env-i3q3uvn9/lib/python3.8/site-packages/setuptools/build_meta.py", line 497, in run_setup
    super().run_setup(setup_script=setup_script)
  File "/tmp/build-env-i3q3uvn9/lib/python3.8/site-packages/setuptools/build_meta.py", line 3[13](https://github.com/Goekdeniz-Guelmez/mlx-kan/actions/runs/10123879697/job/27997615269#step:6:14), in run_setup
    exec(code, locals())
  File "<string>", line 18, in <module>
  File "/home/runner/work/mlx-kan/mlx-kan/kan/__init__.py", line 1, in <module>
    from .kan import KANLinear, KAN
  File "/home/runner/work/mlx-kan/mlx-kan/kan/kan.py", line 1, in <module>
    import mlx.core as mx
ModuleNotFoundError: No module named 'mlx'
ERROR Backend subprocess exited when trying to invoke get_requires_for_build_sdist
Error: Process completed with exit code 1.

[awni]

This is a python packaging issue which is easy to have.

This line is causing issues: https://github.com/Goekdeniz-Guelmez/mlx-kan/blob/main/setup.py#L18

When you do python -m build it creates a new empty environment to build your package and installs the dependencies. That line is trying to import your package (which requires mlx) before the dependencies have been installed. Check out MLX LM for an example of how you can work around that: https://github.com/ml-explore/mlx-examples/blob/main/llms/setup.py#L12-L13

[Goekdeniz-Guelmez]

Thank you!!! After some try and error, I figured it out but now I get Error: Container action is only supported on Linux.

[awni]

Hmm I haven't seen that one before. I haven't tried to use MLX with GitHub actions before. There could be some issues there depending on how the macOS virtualization works.

[Goekdeniz-Guelmez]

Update!

It's done! you can install it via pip install mlx-kan and try a quick training out:

python -m mlx-kan.quick_scripts.quick_train --dataset fashion_mnist --num-layers 3 --hidden-dim 128 --num-epochs 20 --batch-size 128 --learning-rate 0.0005 --seed 42 --save-path /Users/gokdenizgulmez/Desktop

Tranks for your help, I really appreciate that :D!!!!