the great gateware and firmware refactoring

[sbourdeauducq]

• Migen -> nMigen2 (There are too many improvements to list here, see the various nMigen discussions. For example, on a very practical level, this should take care of issues like https://github.com/m-labs/artiq/issues/1384)
• MiSoC -> HeavyX, nmigen-soc
• mor1kx -> Minerva. So we can use upstream Rust and LLVM, it's written in pure nMigen (needs to be ported to 2), and it has been formally verified with riscv-formal.
• Generally, introduce more formal verification in various cores used by ARTIQ, including (D)RTIO. nMigen supports it directly, unlike Migen.

Since we started writing the ARTIQ firmware, the Rust ecosystem has matured and we should now use:

• embedded-hal instead of ad-hoc drivers and CSR accesses.
• RTFM or the Rust coroutines instead of the current scheduler in the ARTIQ runtime.

Many of those components are not currently very developed or mature, and this is quite a large and long-term undertaking, but this is something we should start thinking about.

[jordens]

Everybody should understand that continuous maintenance and improvements that keep ARTIQ up to date and aligned with the other project that we rely on and develop for is a must. It's unfortunately also frequently forgotten when using ARTIQ and Sinara that investments like these are required and that they pay off and that the load should be shared among the users.

With an estimated 200 ARTIQ/Sinara systems worldwide now, it should be relatively easy to stem the cost of that continued maintenance and sustainable development. But since the work items are pretty big individually, we will need to be creative to sell these. There is the classical model of having a few users pay all of it which we have used for most of the ARTIQ developments so far. It's obviously problematic. Another could be publicly listed "sponsoring levels" per user where the community strongly encourages that these correlate with the number of ARTIQ systems in use. Yet another would be to informally add a "tax" onto the hardware manufacturing.

[hartytp]

Absolutely.

I suspect the "classic model" will work best for big shiny new features that users want for their experiments and expect to see an immediate pay off for (DRTIO was a good example of this).

Sponsoring is an interesting route and worth exploring. But I suspect that the "tax" route could ultimately be the main source of funding. c.f. the turn-over of NI and how crappy their hardware/software is compared with ARTIQ/Sinara...

[dtcallcock]

But I suspect that the "tax" route could ultimately be the main source of funding.

I see a few obvious problems with this if it's applied to hardware purchases:

• How do we get contributions from groups who have already purchased the ~200 systems?
• Related to the above, any groups with modest hardware requirements will probably buy a couple crates and then be happy for a long time. This means that to generate even a flat revenue stream ARTIQ must continually expand into new groups (and given the size of ion-trapping, inevitably into new sub-fields). Do M-Labs want to do that kind of marketing legwork? They have expressed reluctance in the past.
• How to deal with the fact you can buy untaxed boards directly off Technosystem and Creotech (where @gkasprow has a grant to commercialize Sinara-hw right?) and that some Sinara-hw contains no or little ARTIQ gateware/firmware.
• If Sinara-hw is to explicitly become the revenue-generating arm of M-Labs (the NI model alluded to above) then I think it will change how people approach it. Right now I see Sinara-hw as an academic collaboration between physicists, engineers (ie. WUT) and commercial entities (M-Labs, QUARTIQ, Creotech, Technosystem). This is something that I can get credit for with funders and my university (especially if published), and I think it's something we can leverage in the future (especially if there were a clearer statement of what Sinara-hw actually is). If new board development just becomes 'paying NRE on an NI-like product' some of those opportunities would be lost - it just becomes an equipment line-item on a grant. There are also COI issues with having students work on commercial R&D.

I think that adding the tax to the M-Labs maintained bitstreams could be a good way forward. Am I right in thinking everyone apart from Oxford and NIST uses this service? It has the nice feature that it correlates with the number and complexity of crates in actual use and is recurring.

Otherwise the sponsoring levels sound like it could work.

If we all just pay into a big pot, how would be decide how to allocate/prioritize effort though?

[jordens]

Using the hardware to pay for the software is definitely not optimal and comes with its own problems. I'm also undecided as to the best of the schemes proposed. Maybe it needs to be a mixture.

But many of the concerns you raise about a hardware tax to fund ARTIQ are not high on my list of things to worry about. Other will also apply to having a ARTIQ bitstream tax.

How do we get contributions from groups who have already purchased the ~200 systems?

The tax would not be applied retroactively. The assumption is that these groups will buy more to extend/improve/maintain existing systems and to add new ones. This is a rational assumption since in the past almost all setups have been "new" ones and the rate of "radical replacements" of control hardware in existing experiments is close to zero in practice. There is no saturation arising from that.

In the same logic NIST won't get any money back from any of those 200 users reaping the benefits of the first ARTIQ contract we set up many years back. It's a different scheme.

We need to generate revenue in any case. This is our strategy and our business. This scheme or any other scheme doesn't change that. And the market doesn't really change with the funding scheme. The point is that with a hardware mark-up to fund ARTIQ, we are doing exactly the same as most other companies are doing to generate revenue and fund development. Expanding into other fields has always been the plan but it's pretty much orthogonal to this discussion.

How to deal with the fact you can buy untaxed boards directly off Technosystem and Creotech

As you have noticed, these individual boards target a limited user base that is able and willing to understand, integrate, configure and maintain their own configurations and don't need the added value or support that we provide. These users are also typically (but not always) contributing already. As it is open hardware, this is completely fine. Same goes for software.

If Sinara-hw is to explicitly become the revenue-generating arm of M-Labs

Sinara has been a revenue stream for years, for M-Labs, QUARTIQ, and other companies as well. It's meant to be. No surprise here.

If new board development just becomes 'paying NRE on an NI-like product' some of those opportunities would be lost

The worry about being labeled "paying NRE on an NI-like product" is misleading: There is NRE. It needs to be paid. It has been paid in the past. And this is an NI-like product in some aspects. All undisputed. It holds for Sinara as much as it holds for NI or the NIST Servo Box. The trick is to distinguish it from NI-like products on the technical level and on the level of the development model. Don't say that these new boards are purely academic will not be turned into products. That's a trap.

it just becomes an equipment line-item on a grant. There are also COI issues with having students work on commercial R&D.

The designs are open hardware. The licensing is completely non-discriminatory. Since this doesn't favor any commercial entity it's hard to see a COI. Just make sure that the involvement of the academic funding stops at the design/testing/characterization and does not reach into the "technology transfer" or the commercialization of these designs and that you are not doing product development. The alternative scheme we have suggested, is to directly involve the companies as partners in the funding proposals. Then it's our job to ensure that there are no COI and that the funding agencies are happy, and it's one that we have been doing before. Wrapping company involvement in a subcontract of an academic funding proposal on the other hand always comes with challenges to argue why this is not product development.

Am I right in thinking everyone apart from Oxford and NIST uses this service?

No. There are more: LUH and U Birmingham both to some extent, and probably (many) others that we don't know or hear about. This is also fine.

If we all just pay into a big pot, how would be decide how to allocate/prioritize effort though?

"You" don't, at least not unilaterally. It's strictly contradictory to the scheme. Same as with any other company where you don't decide about how they invest. And same as with paying bitstream tax or paying for sponsoring levels. This is certainly one of the challenging things to get right. But it does and must align with the market and marketing strategy.

[dhslichter]

I don't see COI issues with funding NRE on hardware -- this already happens any time you ask for a custom laser system, or cryo system, or optical system. You get your custom system, having paid for the NRE, and then the vendor usually turns around and offers it (or components of it) as a standard product. This is how many new products get started. As the researcher, you usually contribute important information, design ideas, and/or measurements to the process, but you're not the one doing the actual commercialization.

I also think that most users want ARTIQ and sinara to be a turn-key solution for running experiments. Whether this takes the form of paying a premium for pre-configured hardware assemblies (vs. build-it-yourself), or for having easy upgrade capability to the software (pre-compiled bitstreams etc), most people will pay some amount because it is more cost-effective for them. Purchasing departments usually understand the notion that scientific software may be associated with ongoing maintenance costs, potentially on a per-seat (or per-core-device) basis. The challenge is to set the price such that people will pay for the convenience, rather than opting to build hardware/compile software for themselves to save the money, while at the same time generating enough revenue to pay for R&D work. This model would also require a bigger customer service/reliability focus than has been the case previously (people will expect more when they are paying for something), which is a challenge and eats into revenue. FWIW NIST is currently on a maintenance contract with M-Labs, which ensures that M-Labs continues to maintain and supply NIST-specific bitstreams with its CI service, adding in new features as they appear in the releases.

If we all just pay into a big pot, how would be decide how to allocate/prioritize effort though?

"You" don't, at least not unilaterally. It's strictly contradictory to the scheme. Same as with any other company where you don't decide about how they invest.

Well, I would argue that if you are paying for a product, you don't get to decide what the company does with your money, but if it's more of a "goodwill donation" for R&D then one could see how the donator might want some say in what work is done. In that instance, the proper thing would be to write out a contract specifying some work to be done in exchange for payment, and then nobody gets grouchy, rather than having some vague "donation pot".

One possibility would be to set up a "steering committee" for ARTIQ/sinara, where membership requires the contribution of funding for longer-haul or less-shiny developments (or where there might be a market failure - like poor old #652) that are then prioritized by mutual agreement within the steering committee. M-Labs/QUARTIQ would be an essential part of the steering committee and would provide suggested paths forward, but there would need to be general consensus reached on the final direction before money is allocated to the work. Probably this would take the form of individual contracts between steering committee members and M-Labs for sub-portions of the work, but with the idea that the contracts are coordinated as necessary to enable multiple participants to contribute to funding a single larger development. De facto, NIST, Oxford, and others such as ARL have been playing this role to varying degrees already. Meanwhile, M-Labs and others would of course be free to invest their own money in ARTIQ/sinara development in parallel, based on their own perceived priorities, if different. If we reach an impasse where two different fundamentally incompatible design philosophies are in conflict, then one would have to branch into different flavors, but this seems relatively unlikely in the near term.

[dtcallcock]

Just make sure that the involvement of the academic funding stops at the design/testing/characterization and does not reach into the "technology transfer" or the commercialization of these designs and that you are not doing product development.

Yes, that's all I wanted to say. NRE is obviously not a COI issue.

The alternative scheme we have suggested, is to directly involve the companies as partners in the funding proposals.

I'm happy to push for something like this. It's hard to see a path forward (eg. SBIR/STTR) without a US company within the ARTIQ/Sinara ecosystem to partner with. If we were to go out and try to find someone, what niche would we want them to fill?

[dtcallcock]

Another could be publicly listed "sponsoring levels" per user

This is apparently built into github now: https://github.blog/2019-05-23-announcing-github-sponsors-a-new-way-to-contribute-to-open-source/

[sbourdeauducq]

It's hard to see a path forward (eg. SBIR/STTR) without a US company within the ARTIQ/Sinara ecosystem to partner with.

We have one.

[dtcallcock]

Exciting. Can you reveal them?

[sbourdeauducq]

* Rust [coroutines](https://doc.rust-lang.org/std/ops/trait.Generator.html) instead of the current scheduler in the ARTIQ runtime.

New firmware on Zynq https://git.m-labs.hk/m-labs/artiq-zynq is pioneering this, as well as newer rustc versions. Way forward could be to port this new firmware to RISC-V and MiSoC/ARTIQ SoC gateware with Vexriscv (already in MiSoC) and then Minerva.

[sbourdeauducq]

Switching to RISC-V should be done with the new compiler (NAC3), as we don't want to use old/patched LLVM there, which is likely more painful than using RISC-V.

In 2014, we decided to use OpenRISC and not RISC-V, because the RISC-V LLVM backend was worse than the OpenRISC one, and there was no good gateware implementations of RISC-V. There are now several RISC-V implementations that are better than mor1kx (the OpenRISC implementation we have), and RISC-V support is now high-quality and upstream LLVM whereas OpenRISC is essentially unmaintained.

The VexRiscv core already works with MiSoC and has good compatibility with the rest of the current gateware code.

[sbourdeauducq]

RISC-V support funded by Duke, thanks @lriesebos @b-bondurant

Since this week we can run simple kernels compiled with LLVM 11 (using the old compiler, NAC3 probably following soon) and a VexRiscv core device, great work @occheung !

[sbourdeauducq]

The full test suite is passing now. The new system with VexRiscv and LLVM 11 seems in general slightly faster than mor1kx. https://nixbld.m-labs.hk/build/34155/log

[jordens]

Cool!

[jordens]

I remember a while ago mor1kx and specifically its multiplier was limiting the cpu fabric clock. Can we increase it again with vexrisc or are the worst timing paths not in the cpu anymore?

[sbourdeauducq]

Doesn't seem so, I just tried a Kasli 1 build (wipm variant) at 125MHz and it failed timing.