Closed japaric closed 6 years ago
I like the idea of moving components into rust-embedded (and increasing the bus factor with that org 😂).
I'd like to see a code of conduct (something line the contributor covenant and the contacts and policy to support it, as well as contribution guidelines (ie. how to pr, policy on breaking changes and versioning, adoption of projects by rust-embedded etc.) at the org level.
Also willing to pick up some of rust2svd/svd-rs(/embedded-hal) development/support/maintenance, and to update japaric/svd#37 to get it merged as a first step in the refactor if that's useful.
@japaric
It's expected that svd2rust will also grow AVR support -- Atmel has its own description file format: atdf -- using the atdf2svd approach. And there are XSVD (JSON version of SVD files) files for RISCV -- I don't think they are official though? -- so svd2rust could also grow support for RISCV via a xsvd2svd tool.
I have already written a library to parse ATDF files into data structures, which should be fairly easy to plug in.
It is [almost being] used in the ruduino library to implement a trait-based AVR-specific HAL. It is being used in another library that defines all registers/IOs for every AVR device.
I like the idea of moving components into rust-embedded (and increasing the bus factor with that org 😂).
Agree strongly. I have mixed feelings on whether this should be done for AVR before it actually gets merged into upstream Rust though.
I've tried to be quite careful to manage expectations with the AVR fork - there's been quite a bit of interest, but the experience is still quite buggy. I think that moving some of the AVR stuff to the rust-embedded
org might give off the impression that the AVR experience is well supported, where in fact things such as having optimisations disabled on a nontrivial project make it quite likely to have an LLVM assertion error.
I think there's a lot of value in bringing the embedded projects into one organisation. It'd definitely help the embedded experience become a lot more seamless, especially with strong collaboration on things such as HALs.
@japaric I'm happy to help out here and there with the cortex-m crates, but I don't have the bandwidth to drive any major initiatives.
Fixing this requires writing or putting together an SVD file that contains only core peripherals, running
svd2rust
over it and committing the code to thecortex-m
repo.
I did some digging. I already knew about these incomplete SVD files, so at least there must have been some interest by ARM at some point.
I found this issue, specifically this comment, which states that core SVD files exist now, referring to some Keil pack. I found this download page, and I believe the pack that the comment refers to is "ARM V2M-MPS2 Board Support PACK for Cortex-M System Design Kit" under the "Keil" heading.
I downloaded the file (despite the name it's just a zip file; add ".zip" to the name and you should be able to open it), and yes, it contains SVD files for all the Cortex-Ms. I don't know if they're any good, and I don't know what the EULA says about using them, but I guess at the very least it makes sense to check them out.
Regarding organization I think it may make sense to move these core components under the rust-embedded org and then create, say, a cortex-m team in charge of maintaining the cortex-m crates.
:+1:
@hannobraun It looks like the SVDs in that pack are for some kind of abstract Cortex-M device. They include peripherals (timer, GPIO, serial engine, etc) that aren't common to all Cortex-M designs. Maybe they're just copied directly from the Cortex-M System Design Kit?
@japaric If a SVD needs to be written from scratch, I can try to put one together directly from the Technical Reference Manuals so that there's no problem with licensing. Unless someone is already doing that.
@docbrown Ah, too bad. And I'm guessing we're not allowed to modify it (though I haven't read the EULA).
So moved all riscv crates to a rust-riscv org. Should help find riscv crates, contributing etc. If someone wants to collaborate I can add you to the org. There is also a riscv-crates repo that creates a cargo workspace to make hacking on riscv crates easier - rust and llvm forks are updated. Some of these crates could be moved to rust-embedded once they're more mature.
@hannobraun Yeah, it's kind of a minefield. There's the EULA (which appears to only reference example code and drivers, but IANAL), a 3-clause BSD(?) license from ARM on the SVDs themselves, and then another license from ARM applied to the code generated from the SVD (the licenseText element). And if the SVDs were just copied from the reference design in the Cortex-M System Design Kit, then there's likely a EULA for that, too (which we can assume Keil is in compliance with, but the kit doesn't appear to be publicly available to verify that). Writing an SVD from scratch seems like the safest option, IMO.
@docbrown
Writing an SVD from scratch seems like the safest option, IMO.
I'd try to contact ARM first, to try and clarify the license. That might not come to much, but it probably won't hurt either. But if you're willing to write an SVD from scratch, I'm not trying to stop you! Just offering suggestions.
I'd be happy to help maintaining svd2rust. I'm also glad that were taking this direction because the pressure and responsibility has been something I've noticed to be far to big for a one-man team.
@Emilgardis @ryankurte
Thanks for volunteering!
I have just sent you collaborator invites for both japaric/svd and japaric/svd2rust. Those two are in a worse state than the cortex-m repos and I don't want to block development on those until we figure out an org structure.
@ryankurte
I'd like to see a code of conduct (..) as well as contribution guidelines (..) at the org level.
Great idea! Any suggestion on how to make it visible at the org level? I don't think you can have pinned repos at the org level. Though it seems you can have some description text at https://github.com/$ORG
@dylanmckay
I have already written a library to parse ATDF files into data structures
Awesome! I know that @kunerd and ecstaticmorse_ (IRC nickname) are interested in svd2rust support for AVR and/or a atdf2svd tool. You should all synchronize efforts.
@dylanmckay @japaric re: ATDF
atdf2rust
really would be nice to have, even for micros in the ATSAM families (Cortex-M). Atmel's SVD files are non-compliant and don't play nicely with svd2rust
. I'd love to be able to generate code for those without have to hack svd2rust
to ignore problems in the SVDs.
The other significant pain point I had working with svd2rust
was lack of register group support, but I see that there's progress on pulling in svd v0.6.0, so I won't harp too much on that.
@dylanmckay @japaric: ATDF
@ecstatic-morse created this atdf
parser repo, which also can parse .atdf
files into svd2rust
structs, if I remember correctly. From my point of view it would make sense to separate the code generation part from the description file parsing part, so that for example svd2rust
and atdf2rust
tools can be created by using the same code generation base, either as a lib
or by adding different executables to svd2rust
. I think I can offer some time to support on this.
@dylanmckay: btw - thanks for the great work on the avr-rust
fork
@japaric I would be interested in also helping to maintain at least svd2rust
. I have now spend 1/2 a week with it (due to the cluster issue), and feel more comfortable with it (but I am still far from being an expert).
@dylanmckay @dvc94ch Since you have org for core AVR / RISCV crates I would suggest you add a link to your orgs to the README of this repo. Also if you tag issues with the help-wanted
label then when someone visits the org page they'll see how many help-wanted
issues each repo under the org has.
@jamesmunns Awesome! Thanks. Sent you an invite.
@kunerd if the ATDF parser produces a svd2rust::Device struct then I think it would make sense to have an atdf2rust tool in the svd2rust repo. Could you open an issue in the svd2rust repo to discuss this?
@japaric org level code of conduct / contributor guidelines / how to contact are probably good candidates for rust-embedded.org? Then the org description can just be a link there and the main page already has a link back.
I don't think you can have pinned repos at the org level.
You can have pinned repositories for orgs! Might require some specific permission though, I don't know. See screenshot:
Sorry this has taken so long.
This my initial proposal for re-organizing the existing embedded repos under the rust-embedded org. Feedback is of course welcome!
Collaborators are split in two groups:
Repository specific collaborators or just "collaborators". They have access to a specific repo, and they help triaging issues, and reviewing and merging PRs.
Team members. Teams oversee the coherent development of the crates / tools under their purview. They share the same set of tasks with collaborators but have a few more executive / administrative tasks: they set the overall development direction by approving RFCs, and they are also in charge of managing crates published on crates.io.
This is the tentative distribution of teams and repos. The initial membership reflects more or less things as they are right now.
The names after the tilde (~) indicate collaborators to that repo.
embedded-linux team
gpio-utils
i2cdev
linux-embedded-hal
~ @japaricrust-sysfs-gpio
rust-sysfs-pwm
spidev
tools team
cargo-binutils
?itm
?svd2rust
~ @ryankurte @Emilgardis @jamesmunnssvd
~ @ryankurte @Emilgardiscortex-m team
cortex-m-quickstart
cortex-m-rt
cortex-m-semihosting
cortex-m
resources team
awesome-embedded-rust
~ @RandomInsano @berkus @therealprof @rahul-thakoordiscovery
?hal / common dependencies team
bare-metal
embedded-hal
~ @hannobraun @ilya-epifanov @thejpster @therealprofnb
All participants in rust-embedded projects will adhere to the Contributor Covenant Code of Conduct v1.4.
The code of conduct (CoC) will be linked from the rust-embedded org landing page as a pinned repository. A copy of the CoC will be included in each repository under the rust-embedded org (CODE-CONDUCT.md), and a note about the CoC will be included in the repository README (see failure for an example).
Teams and collaborators will enforce the code of conduct in the repositories they oversee and on the #rust-embedded IRC channel.
Thoughts?
Also, we need more people on the teams so this is your chance to volunteer to a specific team!
Finally, we need an e-mail address, or some other private channel, to receive reports related to violations of the CoC. @posborne @nastevens would be possible to set a mods@rust-embedded.org e-mail alias?
Hello! I would like to help more. However, I am new to the Rust programming language and would definitely love to learn and contribute more, if I get assistance from this wonderful community! I am very interested in contributing to embedded-linux ecosystem.
Some thoughts:
Are there plans to implement the serial interface for linux-embedded-hal
? Could be interesting to interact with modules such as ESP8266 WiFi SoC or WISOL Sigfox module via UART.
As of Linux 4.8 the new gpiochip
char devices provides a new API and sysfs gpio is supposedly deprecated. There is already a C library, libgpiod, for interacting with the linux GPIO
character device with C++ and Python bindings in the works. It would be great to have a Rust implementation.
I can also help with French and Chinese(Simplified)() translation of awesome-embedded-rust
or other resources if needed.
Regarding maintaining the several projects, perhaps derek can help?
Thanks
@japaric I'd like to get involved in the tools team and would like to add two important topics to the list which we should investigate: flashing and debugging, potentially also non-ITM logging (i.e. serial monitor support). I'd be definitely interested in lending a hand with cargo-binutils.
Potentially I'd also love to help with cortex-m
though I currently have no idea what to contribute there.
Hello together. I'm also new to Rust but in software engineer business since 20 years now. Currently I'm not so happy with the organization of the different SVD files for the STM family of MCUs. @adamgreig is working on a promising approach to keep the SVDs from STM in one place and collect the needed patches there. Another point I'll like to help out are the currently spread implementations of the HAL driver implementations all across over different crates. Best, Ingo
Hello, I can also give a hand and I've proposed to help in the refactoring of SVD. I develop with Rust since end 2012 and I have a few experience in embedded ARM Cortex (I work on a STM32f7 card for audio applications). I can help with french translation too.
As of Linux 4.8 the new gpiochip char devices provides a new API and sysfs gpio is supposedly deprecated. There is already a C library, libgpiod, for interacting with the linux GPIO character device with C++ and Python bindings in the works. It would be great to have a Rust implementation.
Hi @rahul-thakoor. I'm the author of the sysfs gpio crate. I have most of the core bits of a new crate based around the GPIO character device here: https://github.com/posborne/rust-gpio-cdev/tree/first-pass. I need to make some final decisions on APIs and document but an initial version should be on crates.io in a few weeks at the latest.
Once stabilized, this crate will likely move over to the rust-embedded org (which I created awhile back). I considered wrapping libgpiod but decided it made more sense to just interface with the kernel directly -- the API is very similar where it makes sense, but I have also strayed some in order to provide better safety/ergonomics using Rust's ownership system.
Both will need to be supported for some period of time, but I think it makes sense to start pushing people toward the cdev implementation if they know they will only be targeting 4.4+.
I would be happy to help on the resources team. Most probably I can contribute by reviewing material.
Regarding resources, I am also in the process of porting tutorials for bare metal programming on RPi3 (aarch64, cortex-a) to Rust.
Currently, the code is more or less literally translated from C, with some minor abstractions here and there. But it is definitely lacking a layered approach like there is with cortex-m
, cortex-m-rt
, cortex-m-quickstart
, with all its safety abstractions etc.
Therefore, I wonder if there are any plans for cortex-a
in the not too distant future, or is that not on the radar?
Of these projects I'm probably able to help with embedded-hal
somewhat, so will be happy to do it when i can, but alas won't take maintainership.
Hello, @japaric! Can you please give me access to https://github.com/japaric/msp430-quickstart so I can merge https://github.com/japaric/msp430-quickstart/pull/3
I'd be happy to help manage PR's etc. with the https://github.com/japaric/stm32f103xx-hal crate.
Hi @japaric, I have been working for a long time with the core Cortex-M parts for the C++ version of RTFM, where I am now (~slowly) transitioning to Rust.
If you want help with the core cortex-m*
crates, give me a ping.
I finally (!) wrote an RFC for this: #136. We'll discuss this RFC in the next embedded WG meeting (#130) which is in ~12 hours. If you can't make it to the IRC meeting but can comment on the RFC before the meeting please do! Even if everyone who attends the meeting is on board with the RFC we'll leave the RFC open for at least a week before we officially approve it. So leave comments over there if you have suggestions or concerns.
A.K.A. @japaric is a bottleneck for development :-)
There are critical components of the embedded / Cortex-M ecosystem that are currently being maintained by me. Unfortunately I don't have the bandwidth to keep up with everything so issues and PRs have been piling up in some repos.
This issue is a call for maintainers (if you want to help leave a comment!) and a place to discuss how to organize maintainership duties.
First, these are the crates that should be maintained by more people:
embedded-hal
Abstractions for things like Serial, SPI, I2C, etc. to build generic drivers that work with AVR, ARM Cortex-M, MSP430, RISCV, etc. microcontrollers.
There's a bunch to do here as some abstractions are missing and async support is currently nonexistent. Most of the work to do here is ensuring that the established process for adding new traits is followed and that discussions move towards consensus.
Quite a few people have volunteered to help and are now collaborators so I think this one is well covered for now.
svd2rust
Tool used to transform CMSIS SVD files (XML files that describe the hardware registers of a device) into code for low level manipulation of hardware registers. Although the SVD spec was created by ARM
svd2rust
is being used for both ARM Cortex-M and MSP430 devices. In the later case TI provides DSLite files (their version of SVD files) and those are converted to SVD files using the dslite2svd tool.It's expected that
svd2rust
will also grow AVR support -- Atmel has its own description file format: atdf -- using the atdf2svd approach. And there are XSVD (JSON version of SVD files) files for RISCV -- I don't think they are official though? -- sosvd2rust
could also grow support for RISCV via a xsvd2svd tool.Unfortunately
svd2rust
has grown into an unmaintainable mess. The test suite takes a lot of time to run (it takes longer than the CI limit) and it only checks if the generated code still compiles -- it provides no feedback about whether the generated code exposes a certain interface or not. The underlying parser lacks error handling and simply panics on errors. Also,svd2rust
doesn't give you helpful error messages when your SVD file has some problem -- this is an important feature to have when manually editing SVD files.So both
svd2rust
and the underlyingsvd
parser need to be refactor (read: rewritten) to make them more modular and testable. I think the approach to take here is a refactor like the one proposed in japaric/svd#37. Basically we should be able to test code generation on parts of a SVD file. For example that<enumeratedValues>..</enumeratedValues>
produces thisenum Foo { .. }
and this APIimpl Foo { fn is_variant(&self) -> bool { .. } }
, etc. This will let us write unit tests that can replace the huge existing tests that only check for compilation errors. And while you are at it you can add error handling to the parser and nice error messages tosvd2rust
.This is a huge task but it will make it easier to continue the development of
svd2rust
. There a bunch of open issues related to SVD files not being supported; the refactor should make it easier to fix those issues.If you want to tackle this let me know in the comments!
Then we have core
cortex-m
crates.cortex-m
This repo provides access to Cortex-M specific features like access to registers, safe wrappers around assembly instructions and access to core peripherals.
One old issue in this repo is providing a
svd2rust
generated API to access the core peripherals. The current API has been manually written and is not as type safe as asvd2rust
one would be. Fixing this requires writing or putting together an SVD file that contains only core peripherals, runningsvd2rust
over it and committing the code to thecortex-m
repo.cortex-m-rt
This crate takes care of communicating the memory layout of a program to the linker and providing implementations of language items that are required to build a
no_std
program.There are a few missing features in this crate: support for placing functions / interrupt handlers / the whole program in RAM, relocation of the vector table, etc. that need a design. It may be useful to coordinate with the representatives of the other embedded targets to see if we can provide a similar user interface for these features.
cortex-m-quickstart
This crate is used as a Cargo project template for writing Cortex-M applications.
Every now and then a user figures a new way to misfollow the instructions and encounters a new error message. These errors should be documented in the troubleshooting guide.
Regarding organization I think it may make sense to move these core components under the rust-embedded org and then create, say, a cortex-m team in charge of maintaining the cortex-m crates. As the AVR, MSP430 and RISCV communities grow they may also want to move core components into the rust-embedded org and create their own maintainer teams. Thoughts on this?
cc @ryankurte @Emilgardis might be interested in picking up svd2rust development cc @dylanmckay (AVR) @pftbest (MSP430) @dvc94ch (RISCV) see last paragraph, also
svd2rust
support