Beaglebone AI Support

[jallwine]

I've been working on getting MachineKit working on the Beaglebone AI. Things seem to work for the most part, but key HAL components that aren't working are hal_bb_gpio and hal_pru_generic. I've made good progress on hal_pru_generic, and I wanted to start a conversation here to move toward a decent pull request. I've been working on this branch.

Some of the key things that needed changing are:

1) The PRUs on the BBAI seem to implement SR2.0 whereas the BBB implements SR1.1 (see section 30.1.11 vs section 30.2.11 in the AM572X Technical Reference Manual). I'm not exactly sure what this means, but I found that the IEP timer, which hal_pru_generic uses, is a 64-bit timer on the BBAI and requires slightly different steps to initialize and request its status than on the BBB.

2) The BBAI uses remoteproc in order to load/start/stop PRU code whereas hal_pru_generic uses the uio_pruss kernel module to perform those steps. The BBAI has a uio_pruss_shmem module that allows writing to the memory segment of the PRUs, but not any of the control registers, so we can't check the version of the PRU automatically and we can't load/start/stop code. I decided to check /proc/device-tree/model to see if the we're on an AI and use remoteproc if we are. Using remoteproc also requires the PRU binary to be in ELF format, so some extra conversion steps with objcopy and ld are required.

3) The BBAI has 8 GPIO ports vs the BBB's 4 GPIO ports all with different memory addresses. Allowing all 8 GPIOs seemed like more of an undertaking than I wanted to make so I focused on the 4 GPIO ports with the most pins mapped to the P8 and P9 headers. This limits some of the flexibility of hal_pru_generic on a single PRU, but the BBAI also has more PRUs with many direct inputs and outputs so I decided making it an instantiable component would provide added flexibility and the ability to use multiple PRUs.

4) In order to be able to instantiate multiple instances of hal_pru_generic a second timer is needed in order to leverage the second core of each PRU. Currently, the IEP timer is the sole timer used and there is only one IEP timer per PRU chip (which is shared between each PRU core on that chip). The solution I implemented is to use a PWM (eCAP) timer for the second core.

I still need to take a closer look at hal_bb_gpio, but in the meantime I've been using sysfs_gpio that I implemented as a stop gap measure (sysfs gpio is deprecated, so this isn't a good long term solution).

I've gotten a Pocket NC V2 machine running with my changes on the Beaglebone AI. I still need to clean things up a bit and test that everything still works on the BBB.

Anyway, does anyone want to take a look at what I've done and suggest other improvements while this is in progress?

[jallwine]

@cerna @cdsteinkuehler Following the build steps documented here, I'm able to cross compile my fork of machinekit-hal, but it appears that I'm not properly handling the cross compilation of the PRU code. The compilation step actually works fine, but to run on the Beaglebone AI it needs to be in ELF format so the PRUs can be started via remoteproc. These are the steps I take to convert the .bin file that works on the BBB, to an .elf file that the AI needs. When I try to run the .elf file that is generated during the docker build it doesn't work, but if I manually run the commands from the Submakefile on a Beaglebone AI after installing the .deb file it works. Any thoughts on how I should adapt those lines to work with the docker build process?

[jallwine]

Looks like it was just a matter of replacing objcopy with $OBJCOPY and ld with $LD variables. Sorry for the spam!

[jallwine]

Looks like there are issues with using PRU0 on the BBB when using the changes from pull request #320. Still tracking down what they are, but PRU1 seems to work.

[jallwine]

I developed a new HAL driver for reading/writing to the pins of the P8 and P9 headers on the Beaglebone Black and AI. The component is also generic enough that instead of referencing P8 and P9 header pins, it can reference specific GPIO chip and line numbers, which could potentially be leveraged on systems other than Beaglebone boards (maybe I need to come up with a more generic name than bb_gpio for that reason, but it does still use memory mapped from /dev/mem so it's still BBB and BBAI specific without some extra changes).

I developed a new driver instead of modifying hal_bb_gpio because I wanted to change the interface, as well. Instead of being a single real time component that is instantiated once with all pins defined at that time, it's now an instantiable component that is instantiated once at the top of the HAL file so functions can be registered to a thread and then once for each pin that is used (which can be done anywhere in your HAL file or later at runtime). This allows for better encapsulation of HAL files in that pins can be defined and grouped with the specific net that they tie into rather than needing to be defined all at once at the top of your main HAL file. This allows for functionality to be added in encapsulated chunks rather than making small changes to a number of different places in a large HAL file.

I still need to add it into the MachineKit-HAL repository and more testing would be nice, but it seems to work for me and currently can be installed manually into a working MachineKit-HAL setup using instcomp.

[cerna]

@jallwine,

if I understand it correctly - and I am not sure about it - you would like to create a generic HAL driver for all devices using memory access under some Linux device file? So all drivers which can now be found in the Machinekit-HAL source-tree could be rewritten to use it with just some vtable specifying what at what place means?

This sounds interesting.

This allows for better encapsulation of HAL files in that pins can be defined and grouped with the specific net that they tie into rather than needing to be defined all at once at the top of your main HAL file. This allows for functionality to be added in encapsulated chunks rather than making small changes to a number of different places in a large HAL file.

This is a great approach!

[jallwine]

if I understand it correctly - and I am not sure about it - you would like to create a generic HAL driver for all devices using memory access under some Linux device file? So all drivers which can now be found in the Machinekit-HAL source-tree could be rewritten to use it with just some vtable specifying what at what place means?

I'm not sure I quite understand what you're asking, but I think I'm solving a simpler problem than that. hal_bb_gpio allowed for easy access to specific pins on the P8 and P9 headers of the Beaglebone Black. This allowed for wiring up buttons or LEDs that could be controlled with HAL pins such as bb_gpio.p8.in-09 (pin 9 on the P8 header). All the pins you wanted to use, though, had to be defined in one place when hal_bb_gpio was created. That is the first problem I've attempted to solve with my implementation.

Furthermore, the GPIO pins on the Beaglebone Black and AI can be controlled via registers in global memory that can be mapped using /dev/mem (hal_bb_gpio does this, and so does bb_gpio). This is very platform specific, but is also the fastest way to manipulate those pins that I'm aware of. There's a libgpiod library that goes about manipulating GPIO pins in a more official and cross platform way via the kernel by specifying a GPIO chip and line number. I currently use that library to let the kernel know that I've claimed specific GPIO pins. I map the P8 and P9 numbers to the corresponding GPIO chip and line numbers so a user doesn't have to know them, but taking it one step further, I also allow the user to directly specify a GPIO chip and line number if they wish. Allowing for this is a bit more cross platform, except I still leverage the global memory registers to directly control the GPIO pins, which isn't. I left a few comments in the code, though, to show where it could all be done using libgpiod, which would avoid using the global memory registers at all. The overhead of going through the kernel, though is significant relative to the speed at which it can be done through the global memory registers, so I didn't commit to doing it that way. In reality, though, the overhead is likely negligible for most real world applications, so I could see only using libgpiod to ensure a more cross platform GPIO driver at the expense of some speed. At that point, I would likely rename the driver from bb_gpio to simply hal_gpio or something like that.

[cerna]

@jallwine,

I'm not sure I quite understand what you're asking, but I think I'm solving a simpler problem than that (...)

Yes, I understand now. I wasn't sure if you don't want to go further in your and create a general driver for accessing memory mapped peripherals.

Ergo:

Furthermore, the GPIO pins on the Beaglebone Black and AI can be controlled via registers in global memory that can be mapped using /dev/mem (hal_bb_gpio does this, and so does bb_gpio)(...)

As far as I know, a process can have memory mapped (mmap) the segment controlling the device and not use system calls in this case (the mentioned problem with going through libgpiod library), but bad things are going to happen if more than one process is trying to access this space at the same time and obviously similar goes for threads of single process.

So I was originally thinking if you are not trying to solve this issue in Machinekit-HAL - the issue when you have multiple threads each with different CPU affinity, and you need to serialize access to that one shared resource. Of course, you would hit the rate monotonic scheduling which is for now the only way to schedule in Machinekit-HAL.

But I realize now that I was reading too much into it.