symforce-org / symforce

Fast symbolic computation, code generation, and nonlinear optimization for robotics
https://symforce.org
Apache License 2.0
1.44k stars 147 forks source link

Fix lcmtypes install during editable installation #247

Closed bradley-solliday-skydio closed 2 years ago

bradley-solliday-skydio commented 2 years ago

Previously, lcmtypes package was installed by the InstallWithExtras (subclass of setuptools.install) command. However, this command is not run during an editable installation.

Because the lcmtypes package does not exist in the source directory, and is instead generated into the build directory by the CMakeBuild command (subclass of setuptools.build_ext), I've told setuptools to treat the lcmtypes package as an extension module.

This ensures setuptools does not attempt to install lcmtypes until the build_ext command has been run.

Note: if you treat it as an ordinary package, problems arise from the fact that the package isn't available until after the build step.

During a non-editable install, nothing particularly special needs to be done (other than being careful with the fact that we're dealing with a folder rather than a shared object file).

During an editable install, extension modules are placed in the source directory. Since the name of the package (lcmtypes) collides with the name of a directory in the top level of the source directory (the one containing the .lcm files that say how to generate the lcmtypes directory), I decided to make a point of placing the package in a different directory called lcmtypes_build. I don't think this is required for anything to work, but it seems less messy to me this way. In order to accomadate this, I told setuptools to look for the lcmtypes package in the lcmtypes_build directory by ammending the package_dir argument of setup().

To test, I added checks to the test_editable_pip_install github actions workflow.

Note: We currently don't test non-editable pip installations. This is just because we haven't gotten around to it yet.

Fixes #239