Future of yotta

[plauche]

I noticed while reading the release notes for mbed OS 5.1 that a new tool, mbed CLI, has been introduced in place of yotta. Looking back over the yotta commit history...it appears to have been fairly active until May of this year. These two things indicate to me that yotta is possibly being phased out by the ARM mbed community.

Would it be possible to know -

• Is yotta going away? If so, why? Were there pitfalls or limitations without solutions?
• Are there any plans for future development and maintenance of yotta?

I currently work on a project built around the yotta system/tooling so this is of great interest to me :).

[janjongboom]

Hi @rplauche,

For mbed OS development yotta will not be used anymore. Some background in here, but it boils down to:

The key point for dropping yotta / the library system for mbed OS 3 is that it's incompatible with the module system we use in mbed 2. This was hurting both the ecosystem and the community, with new features coming to mbed OS 3, but the community still building most apps and libraries on mbed 2.

Given that the community is what has made mbed to what it is today, we decided to port all new fancy features of mbed OS 3 (nanostack, uVisor, Client support, etc.) to the mbed 2 library system, and write a new tool to work offline (because we very much like that of yotta).

[ilg-ul]

need the community to take ownership of the tool

this might be possible, but the question is how can the community also take ownership of the yotta registry? what is the point of further maintaining the tool if at a certain moment the registry is shut down?

I expressed my concerns related to yotta design (#604), and plan to develop an alternate solution, more GUI friendly, and less CMake centric, but I currently have no solution for a central modules registry, which is a very nice thing to have.

[janjongboom]

@ilg-ul We can either move yotta into a separate GitHub organisation, or add contributors to the project. I'll be happy to do so if people want to take over.

@autopulated probably has some opinions on this too though, so let's wait and see what he thinks :-)

[ilg-ul]

@janjongboom, my initial message was confusing, I was referring to the yotta public registry, not the github repository.

[janjongboom]

@ilg-ul Ah, good question. No plans to shut it down, we have plenty of projects building on top of yotta ourselves (f.e. all micro:bit projects). But we should make some formal commitments there. I'll talk to some people when I'm back in the ARM offices.

[ilg-ul]

@janjongboom, the issue is even trickier. If a formal commitment to keep the registry on is made, it would be helpful to also clarify the terms of use.

the current terms of use, as publicly mentioned at http://yottadocs.mbed.com/reference/registry.html#terms, can be relaxed, to be sure modules created and maintained with other tools than yotta (in my case by some Eclipse plug-ins, plus that the modules would include more elaborated metadata), will not be kicked out of the registry.

this issue was extensively discussed in #590, and the conclusion was:

yes you can use yotta for any project: embedded/desktop/ARM or non-ARM, and that's what the language on the website refers to.

but it would still be good to clarify the terms and conditions to allow the use of the yotta repository with future build tools, not only the (almost defunct) yotta tools.

[janjongboom]

@ilg-ul, thanks, good point. I'll discuss internally.

[autopulated]

On Thu, Sep 1, 2016 at 9:31 AM, Jan Jongboom wrote: Then on to the future of yotta, I still think that yotta is a great tool for building modular C++ applications, and I'm actually using it in some projects myself. However, given our shifted interest and given that @autopulated (the maintainer of yotta) left ARM this summer, we would need the community to take ownership of the tool, probably to be used in a much wider spectrum than just mbed / embedded development. I hope that that'll happen at a point in the near future.

For sure ARM should either actively maintain yotta, or set it free.

If ARM wants to maintain yotta only for a narrow set of projects, then one option might be to just open source the registry, so other people could set up another instance of yotta (under another name & different domains) for different uses. The registry is actually very nicely architected, and it would be fairly straightforward to host your own instance. Since modules are stored in blob storage (Azure blob storage at the moment, but using Amazon S3 would also be possible), it's fairly cheap to host even with a large number of modules being stored. I think this would be better than mixing modules for different build tools in the single registry.

(We had originally intended to open-source the registry when open-sourcing yotta, but it was kept closed as how/whether it would integrate with the (then) future ARM IoT web services product offering was not clear.)

FWIW a https://github.com/yottabuild github org exists already, it's just empty ;)

For me the best outcome would be some organisation taking over yotta which has an interest in maintaining a broad appeal to all sorts of projects, and which is actively using yotta itself. Ideally it'd also take over the running of the registry, name, and domain names.

[ilg-ul]

open source the registry ... it would be fairly straightforward to host your own instance. ... this would be better than mixing modules for different build tools in the single registry.

yes, I evaluated this solution, at first it looked attractive, and initially all my projects had various servers in the livius.net domain, hosted on a server I manage.

however, I realised that this setup has a major flow: as we are not immortals, what will happen with all those servers if I suddenly die? I do not have an organisation behind me, to take over and further maintain them, and I doubt broadband internet availability in hell/heaven is good enough to consider remote management a solution. :-)

that's the reason why I'm searching for a public registry where to host not only my modules, but also modules created by others using my tools.

the plan was to make my modules yotta compatible, meaning to have a compliant module.json so that yotta will identify them, and to add a custom CMake file in order to pass a simple build, with the advanced features, defined by the additional metadata, available only with my tools & plug-ins.

as fallback, my tools will work with git/hg repos only, but it would still be very nice to have a public registry of all available modules.

[plauche]

Thanks for the info about where yotta is going. This is obviously going to spark some discussions within my team about the future of our product and yotta. We may have some interest in helping maintain the tool in the future if it is spun off. Or we might take the ideas we like and integrate them into our own internal tooling :).

Re: the registry, it is actually less of a concern to us. We store all of our modules in GitHub. I don't expect it will be going away anytime soon.

It would be great to be kept in the loop if possible about future decisions ARM makes regarding the yotta tools and ecosystem.

[ilg-ul]

any progress?

[autopulated]

It would be awesome to have a public commitment from ARM on what the future of yotta is.

[ilg-ul]

it would be awesome, but here we are talking about a long term commitment, which also has some financial consequences (related to covering the registry cloud fees and maintenance costs), and this raises the question of why would ARM do this. as a comparison term, npm, which has a very successful registry, is managed by 'npm, Inc.', a company which seems committed to support the registry on long term.

unless ARM comes with a very convincing message, I would not count on the future of the yotta public registry.

personally, I already scrapped the plans to use the yotta public registry and I'm redesigning the XCDL/xPacks mechanism to identify packages using distributed repository index files.

[ilg-ul]

@janjongboom, any official statement from ARM?

I'm about to remove all references to yotta from my projects, but if things are not yet decided, I might reconsider.

[0Grit]

@janjongboom was this ever discussed internally?

To me, Yotta and everything it stands for is the next logical step for embedded development.

Can you give it one last spark to see if something can be done with it? Whether it be open sourcing the registry or making an announcement to the community that Yotta is now in their hands.

Off topic, but I can't express strongly enough how much I wish the course had been stayed with mbed os 3.0.

[ilg-ul]

Yotta and everything it stands for is the next logical step for embedded development.

I also strongly believe that a component based build system is the future of embedded development (and perhaps more generally for C/C++ development).

however I would try to make the best use of the experience gained with yotta, and do not expect miracles from ARM.

for those interested, I'm continuing work on the project xPacks/XCDL, that somehow addresses these issues. for the moment I received the ok to use the NPM repository to store xPack packages, and a tool to publish packages there and install packages from there will be available (hopefully in a few weeks time).

the tools will be written in node.js, and generally will favour JSON for configurations and JavaScript for scripting. GUI support will be available as part of future versions of GNU ARM Eclipse.

if anyone is interested in contributing to the design and implementation of this project, please let me know.

[kyleparrott]

@janjongboom Was there a conclusion from your discussions on the official status of yotta?

With the number of PR's coming from mbed team members the past week or so it would seem that this project is still alive. It would be nice to have official word as to the status of the project.

[thegecko]

The official status of yotta is as follows:

The yotta tool is an open source product, originally created by ARM for mbed OS 3, but developed in a generic fashion to allow package management and composition of other C/C++ libraries and applications.

ARM have moved away from using yotta as the composition/build tool for mbed OS since version 5.

Yotta is still used and is being actively maintained, albeit at a slower pace. We plan to continue hosting the yotta registry in its current state.

[kyleparrott]

👍 Perfect! Thank you for the info @thegecko.