Closed mrstegeman closed 4 years ago
I'm not sure what rather than bootstrapping on target in the background for a (long) time before being able to actually start up
means.
You could even selectively include the property arduino-cli binary in your package, to make it smaller. I've done this for the Eufy add-on, as an example.
Would that mean I would have to create files for each architecture individually? And then upload them as releases on Github? That sounds like a lot of work.
Previously, you were using bootstrap.py
to download and build all of the Python dependencies on the Raspberry Pi, after initially installing the add-on. This can take a long time, and it's unclear to the user what's happening.
Recently, I added support for Python add-ons to our addon-builder, which is what we've always used to cross-compile our Node.js add-ons for different architectures. Since this add-on has native dependencies that are compiled for a specific target, the addon-builder will handle building the entire matrix of Python version and architecture pairs, e.g. linux-arm for Python 3.7, linux-arm64 for Python 3.5, etc.
The process is this:
package.sh
script across the matrix.I don't know anything about what the addon-builder is, how it works, why I would want to use it, etc. The addon builder github doesn't really help me understand. So for now I don't think this is a direction I want to take this addon in.
For a Python add-on with native dependencies, in order to run on everything we support, you have to cross-compile your add-on for the following targets:
It's a similar story for Node.js add-ons, although we only build for 3 versions of Node.
The addon-builder does this by using custom Docker-based toolchains (available here), which are run by this workflow, using this script and this script.
You may think this is unnecessary, if you only want to support the Raspberry Pi, but even with that, users could run different distributions, different versions of Python, etc.
If you are still resistant to using it, that's fine. However, you can still keep these changes in case you change your mind in the future. You just run the script like this now:
ADDON_ARCH=linux-arm ./package.sh
I add this add-on as a submodule for the builder.
Each time you want a new release built, you file a PR in the addon-list repo (as usual) and let me know.
I kick off the builder. It automatically pulls the latest code from your GitHub repo and runs the package.sh script across the matrix.
I update the URLs and checksums in your add-on list entry in your PR and we're done.
I must have overlooked this somehow. That sounds pretty good. And more secure too, since you can be sure that the Github code is what's in the addon.
How do I then do local tests first? Do I keep a separate 'old school' package-oldschool.sh
for that?
And can I still have files in the github that don't make it into the final 'tarball'? How do I indicate which files actually matter? Is that why the package.json file is still around?
That sounds pretty good. And more secure too, since you can be sure that the Github code is what's in the addon.
I agree. 😄
How do I then do local tests first? Do I keep a separate 'old school'
package-oldschool.sh
for that?
You can still use the same package.sh
script. Just run it like this:
ADDON_ARCH=linux-arm ./package.sh
And can I still have files in the github that don't make it into the final 'tarball'? How do I indicate which files actually matter? Is that why the package.json file is still around?
Your package.sh
script dictate what goes into the tarball. The addon-builder uses your package.sh
script. package.json
is still around to support gateway versions <= 0.9. If you don't care about those anymore, you can get rid of it.
I actually tried to remove package.json recently, and it lead to a failure. The github script that checks new addon uploads still requires this file to be present.
I noticed in the package.json
code you provided you are still adding files to the package.json? Is that just for legacy purposes? If so, can I remove that code and remove the package.json? (assuming the server block is lifted)
One more thing: the script currently deletes files in the base arduino-cli
folder. I'd like it to not do that :-) Could you modify it to only delete files in package
directory once it's created?
I actually tried to remove package.json recently, and it lead to a failure. The github script that checks new addon uploads still requires this file to be present.
You should know how to fix that now. 😃
I noticed in the
package.json
code you provided you are still adding files to the package.json? Is that just for legacy purposes? If so, can I remove that code and remove the package.json? (assuming the server block is lifted)
Yes, for 0.9.X and below, we actually checked that files
array at run time.
One more thing: the script currently deletes files in the base
arduino-cli
folder. I'd like it to not do that :-) Could you modify it to only delete files inpackage
directory once it's created?
No problem, PR updated.
thanks! done.
The addon-builder now has support for cross-compiling Python add-ons for various architectures and Python versions. It would be helpful to use it rather than bootstrapping on target in the background for a (long) time before being able to actually start up. You could even selectively include the property arduino-cli binary in your package, to make it smaller. I've done this for the Eufy add-on, as an example.
package.sh
change: https://github.com/mozilla-iot/eufy-adapter/commit/5cd3c3c394802b23e4ee402d3ea4befbb901d434