pypi upload

[apoorvalal]

wheels are now pypi-able thanks to cibuildwheel; overhaul of build system to use micromamba for dependencies worked. @matthewwardrop would you mind deleting the pypi project you created (it isn't listed on pypi yet and it refuses to let me register it) so I can upload it through ci on this repo?

steps to verify wheel mobility:

• create a new venv
• run pip install https://github.com/apoorvalal/pyensmallen/releases/download/v0.0.161/pyensmallen-0.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (replace to python 3.11/3.12 depending on your system)
• run test script: test_ensmallen.py with contents

import numpy as np
import pyensmallen 

def rosenbrock(x, a=2):
    return np.sum((a - x[:-1]) ** 2 + 100.0 * (x[1:] - x[:-1] ** 2) ** 2)

def rosenbrock_gradient(x, a=2):
    grad = np.zeros_like(x)

    # Gradient for the first element
    grad[0] = -2 * (a - x[0]) - 400 * x[0] * (x[1] - x[0] ** 2)

    # Gradient for the middle elements
    grad[1:-1] = (
        -2 * (a - x[1:-1])
        + 200 * (x[1:-1] - x[:-2] ** 2)
        - 400 * x[1:-1] * (x[2:] - x[1:-1] ** 2)
    )

    # Gradient for the last element
    grad[-1] = 200 * (x[-1] - x[-2] ** 2)

    return grad

def objective_function(x, grad):
    grad[:] = rosenbrock_gradient(x)
    return rosenbrock(x)

# Initialize L-BFGS optimizer
lbfgs = pyensmallen.L_BFGS(numBasis=10, maxIterations=1000)

# Initial guess
initial_x = np.array([-1.2, 1.0])

# Optimize
result = lbfgs.optimize(objective_function, initial_x)

print("Optimized parameters:", result)
print("Objective value:", rosenbrock(result))

works on my ubuntu 22.04 machine

[apoorvalal]

the relevant section in the build file

[apoorvalal]

cc @JamesYang007 ; I copied your workflow for adelie for cibuildwheel and this seems to have worked fine for linux (the wheels are portable and pip install-able) but not macos (it fails to find ensmallen and armadillo). Any idea what might be going on? I was under the assumption that wheels would be built in the miniconda environment, but this behaviour seems to differ bw linux and macos (but adelie didn't seem to run into the same problem).

[Feel free to ignore this if you're busy with dissertation work]

[matthewwardrop]

Hi @apoorvalal! Sorry for dropping the ball on this one. I had most of this working for Linux too in a local branch, but never got it working for macOS and Windows (didn't even try). Glad you have it mostly sorted.

I've given you owner access to the pyensmallen project on pypi. Let me know if there's anything you want me to look at. My time these days is heavily constrained, but I'll do what I can to help :).

[JamesYang007]

@apoorvalal, I just took a look at the Github actions workflow file. Can you try adding generate-run-shell: true right after https://github.com/apoorvalal/pyensmallen/blob/ed9e4f0063ba77053146fbaa88ddb3beaba78578/.github/workflows/build.yml#L30-L34? I believe this flag makes sure that your shell will do a conda activate pyensmallen before running any command so that you're in the environment. The issue seems to be that when you invoke cibuildwheel, it can't detect a dependency in your conda env.

[apoorvalal]

:face_exhaling: that didn't work (the macos build still only looked in the homebrew path for PKG_CONFIG_PATH). Seems like env vars weren't persisting across steps somehow?

I rejigged the two steps (env creation + wheel building) to set CIBW_ENVIRONMENT in the wheel building step, which finally worked.

[apoorvalal]

Done! Thanks both for your help.

[matthewwardrop]

@apoorvalal Just in case you are interested, env is local to a job step. If you want things to persist across steps, you can write to GITHUB_ENV, which is a file that will be sourced into each subsequent step. e.g.

echo "MY_VAR=...." >> $GITHUB_ENV

I only discovered this recently while trying to get PyEnv working nicely in GitHub actions for ancient versions of Python.

More info here: https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/store-information-in-variables