search

mirrexagon / nixpkgs-esp-dev

Nix flake and overlay for ESP8266 and ESP32 development.
Creative Commons Zero v1.0 Universal
144 stars 72 forks source link

Python environment does not change to match when overriding ESP-IDF version #25

Open mirrexagon opened 1 year ago

mirrexagon commented 1 year ago

24 points out that since #23 changed from using mach-nix to dynamically read the requirements file to manually packaging and including the necessary Python packages, overriding the ESP-IDF version (see examples/shell-override-versions.nix) doesn't adapt the Python environment, making overriding the version less useful.

mirrexagon commented 1 year ago

I would like an automated way to check `requirements.core.txt, the latest constraints file (eg. https://dl.espressif.com/dl/esp-idf/espidf.constraints.v5.1.txt), and the latest versions of those packages on PyPi against the Python packages I have packaged here, to make it easy to see what is missing or could be updated.

mirrexagon commented 1 year ago

Note to self, could https://github.com/nix-community/nix-init help us?

jleightcap commented 4 months ago

I'm running into this issue when trying to compile a project depending on ESP IDF 5.0.1: the Python dependencies are declared incorrectly. Is there any advice on compatibility with a toolchain this old? Going through the commit history, it seems at one point a parsing on the requirements.txt was done, but has since been converted into a static list of dependencies?

mirrexagon commented 3 months ago

Yeah, I re-did the Python packages because we were using mach-nix before and it isn't supported anymore and started breaking anyway. I'm not sure what another good solution is right now to adapting the Python environment to the overridden version.

mirrexagon commented 1 month ago

See #66 for some thoughts on using the Docker images to get exact dependencies.

mirrexagon commented 1 month ago

I made an update to the Python packages in b81d09b229a9aef021d9a26584a291e43f9a0d48 by running pip install -r esp-idf/tools/requirements/requirements.core.txt --constraint=espidf.constraints.v5.3.txt --dry-run to get the resolved versions. That might lead to a way to at least generate a .nix file with the exact dependencies for a version of ESP-IDF. Then we might be able to at least run that separately manually and pass the result into the derivation when we want to specify an exact ESP-IDF version.

mksafavi commented 1 month ago

Did you find something more concrete?

From what I remember, there are requirements.txt files that only list the packages. They also have constraints files in the releases which don't lock the versions either. It has open ranges. For example the idf-component-manager was set to something like >1.0. so the docker release is using version 1.5 while nixpkgs-esp-dev has 2.0.

I think we can pip freeze in the docker container to get a locked requirements file and use that to update the python packages. What do you think?