Publish to platformio?

[xoofx]

Would you be interested to publish monocypher to platformio as a library ? That would make monocypher easily installable/discoverable for IoT projects and other libraries could more easily depend/build on top of it. Let me know and I can make a PR with a library.json later this week...

[LoupVaillant]

Hmm, sounds interesting. You're the second person reaching out to us about this platform, it is time we address the issue.

First, there is one important thing to keep in mind, which may explain why I didn't follow up the first time: the files inside the git repository are not publishable. Only the files in the tarballs are. There are a couple crucial differences:

1. The repository lacks a critical header for tests (tests/vectors.h). It is generated by tools we don't expect all users will have, most notably Libsodium and Python. With that header, users only need a small part of the standard library to run the tests and print results.

2. Published files start with something like // Monocypher version 3.1.2. Versioned files start with something like // Monocypher version __git__ instead. We have a release script that automatically replaces __git__ with the right version number upon release. This helps users remember at a glance which version they were using, and decide whether they want to upgrade. (This is most important when a vulnerability is found — only happened once so far.)

Second, on the topic of installability, I believe Monocypher is already in an excellent place: by default, users are encouraged to copy & paste monocypher.h and monocypher.h, to their project, and voilà, they're done. Versioning is not a big problem: the files are clearly marked, and the API is extremely stable. It is extremely unlikely a user grabbing version 3.1.2 now will need to update in the foreseeable future (Monocypher is feature-complete, and has been audited).

I've also discussed with @fscoto, and our current stance at the moment is that we should not burden the main repository with minor conveniences like alternate build systems (CMake, Buck…). For instance when Buck support was proposed, it was pushed to an external repository, and maintained by someone who know Buck better than I do (I know nothing about Buck).

That leaves discoverability. That seems to be important. If PlatformIO can help users find Monocypher when they need it, that would be neat. If it could help users specify Monocypher as a seamless dependency, that would be nice too, though less critical, given how Monocypher is intended to be used in the first place (just copy & paste the files, you won't need to declare the dependency).

---

Now let's get practical.

Embedded is our most important user base that I know of, and PlatformIO probably warrants first class support. I just have no idea how it works. Is is an IDE or a collection of tools people install locally on their computer? Is there a central repository somewhere like Rust's Cargo? Is there a way to search through all libraries that support PlatformIO? How exactly is a project supposed to support PlatformIO? What is the expected user experience?

Then on our side: do we really need to host the manifest file ourselves? Isn't that file supposed to be published in some central repository? Does that file need to point to a Git repository, or can it fetch a tarball instead? Can we write that file and have it automatically point to, decompress, and extract source files from a given tarball?

Depending on the answers to those questions, we may need to spawn a dedicated repository just for PlatformIO to use. A sufficiently minor hassle that I'm willing to consider it. If however we could avoid that, and instead have the manifest file just be some external file published somewhere else, that would be nice. Regardless, it would be listed as an alternate distribution in the website's download page.

So, what is PlatformIO, and how can we make it work?

[xoofx]

Embedded is our most important user base that I know of, and PlatformIO probably warrants first class support. I just have no idea how it works. Is is an IDE or a collection of tools people install locally on their computer? Is there a central repository somewhere like Rust's Cargo? Is there a way to search through all libraries that support PlatformIO? How exactly is a project supposed to support PlatformIO? What is the expected user experience?

Actually, someone has already published Monocypher on PlatformIO, the repo is here (and it's an outdated version as you can see)

About PlatformIO, I'm relatively new to IoT development, but it seems the de-facto way to work these days. You develop from VS Code and so far I have enjoyed their dev experience (support for lots of microcontrollers, libs installable directly into your projects, all compiler tool chains downloaded automatically...etc.). You can be up and running within a few minutes with PlatformIO to almost whatever microcontroller/boards on the market.

Then on our side: do we really need to host the manifest file ourselves? Isn't that file supposed to be published in some central repository? Does that file need to point to a Git repository, or can it fetch a tarball instead? Can we write that file and have it automatically point to, decompress, and extract source files from a given tarball?

As you can see above, folks can already do it on their own outside of your repo. It's really up to you. Do you want other folks to distribute your work and library on PlatformIO? I don't know if tarball is supported by PlatformIO, but I doubt. Like rust, the development experience these days is having a repo URL, a git version, and PlatformIO will figure out the rest. I don't think they host a copy of the repo files and it's probably only a registry (I might be wrong)

My advice would be to not complicate your work, just add a library.json to your repo, publish it to PlatformIO and you are done. You control the version that are published in sync with your repo/publish workflow. You can create a separate repo for sure if you want, but it seems to complicate more the publish workflow as you would have to update 2 repositories when you push a new version.

[LoupVaillant]

My advice would be to not complicate your work, just add a library.json to your repo, publish it to PlatformIO and you are done.

Here's the thing: that wouldn't work. Users would get the latest files from the master branch, which isn't the stable version. We could point to another branch, perhaps even a tag, but then users wouldn't get the real files, they'd get the __git__ ones.

That's the problem with the outdated PlatformIO reference you pointed to: it points to our master branch, so if I introduce a temporary critical vulnerability, only to fix it 2 commits later, users might get in trouble. (Not that it's likely to happen now, since Monocypher is basically "finished".)

More generally, it seems that several tools and ecosystems work with git repository URLs. For those, maintaining a dedicated repository with the content of the tarball would solve many problems. Yes, it would complicate my release process. But right now I should be able to release new versions only once a year, for typos and documentation. And I can always automate it with a little script.

I'll think some more about it.

[xoofx]

That's the problem with the outdated PlatformIO reference you pointed to: it points to our master branch, so if I introduce a temporary critical vulnerability, only to fix it 2 commits later, users might get in trouble. (Not that it's likely to happen now, since Monocypher is basically "finished".)

Not really. The PlayformIO library.json version will point to the git tag of the repo, not to master. So unless you remove the tags on this repo, that should work out of the box. The version I linked above is really pointing to the tag 2.0,6 version, not to some random latest master.

I will let you think about it and closing for now. I can ask the repo above to update the version or roll my own package on platformIO as well, so I'm not blocked.

[LoupVaillant]

_The PlayformIO library.json version will point to the git tag of the repo, not to master.

Oh I didn't realise. Well, that solves a big worry.

I can ask the repo above to update the version

That can work as well. There's still the __git__ issue, but since there's PlatformIO to keep track of the version number, that's not too bad. I'll open an issue there, and make sure I list it as a third party distribution.