Add memoryio module to allow raw memory access

[ryang14]

The main goal of memoryio is to allow for pure python drivers for custom memory mapped peripherals when running CircuitPython on a FPGA SoC.

It should also be usable for reading raw memory on microcontrollers to assist debugging and allow access to memory regions not otherwise used by CircuitPython.

The API concept is currently memory = memoryio.bytearray(<start address>, <length>). memory can then be accessed as a byte array.

[ryang14]

Memoryio is now functional and I was able to read and write to SAMD51 backup RAM. The next step is to figure out how to restrict memory access. My initial idea is an array of memoryio_region_type with start address and length fields for regions that can be written to. The board specific code would then check the set region against the allowed regions to determine if writing is allowed.

[tannewt]

I think that's a good approach. Maybe have it as an "allowlist" so that only the regions in the list can be written.

[kevinjwalters]

I didn't know about it til asking https://forum.micropython.org/viewtopic.php?f=17&t=8553 but MicroPython can access memory via machine library. The docs say it's for reading but the forum reply implies write works too. Is there a reason not to use that as a basis for this functionality?

That has three different sizes mem8, mem16 and mem32. That might be to allow single/atomic reads/writes?

[tannewt]

@kevinjwalters I think that the API is similar. I was assuming we'd do atomic writes if the data given is the correct length and alignment. Acting like a bytearray means it only works with ints 0 - 255 but also avoids the issue of needing longints for 32 bit values.

[dhalbert]

Issue #1268 also brought this up, taking inspiration from MicroPython machine.mem (which may only exist on the micro:bit port, I'm not sure).

[t-ikegami]

The uctypes module may provide the similar functionality of memoryio. I just enabled uctypes in CircuitPython7.0, to manipulate the Configurable Custom Logic (CCL) peripheral of SAMD51 like:

import uctypes as ct
ct.bytearray_at(0x40000800 + 0x1C + 1, 1)[0] |= 0x40
ct.bytearray_at(0x42003800, 1)[0] |= 0x01

This enables clock supply to CCL, and issues software reset on CCL. By using struct facility of uctypes, the scary code above can be replaced by:

from SAMD51p19 import MCLK, CCL
MCLK.CCL = 1
CCL.CTRL.SWRST = 1

[tannewt]

@t-ikegami that'd struct facility would be awesome! Could we autogenerate them from SVD files?

I'd be happy to see ctypes enabled after we do a pass to ensure what we have matches CPython.

[t-ikegami]

I didn't know about SVD files, and I wrote my SAMD51p19.py with my eyes squared on the data sheet. Yes, I think it's possible to write a tool to generate peripheral definitions from SVD files.

As for the naming, "memoryio" may be more adequate than "ctypes", as uctypes has almost nothing in common with ctypes of CPython.

[tannewt]

We could probably have the class load the SVD itself rather than needing to generate an intermediary. This repo is a great collection of svd files: https://github.com/posborne/cmsis-svd

[t-ikegami]

Parsing SVD files may be a bit harder for MCU, I think. (The size of ATSAMD51P19A.svd is 1.7 MB.) I have just finished a crude version of the SVD converter: https://github.com/t-ikegami/WioTerminal-CircuitPython/tree/main/ATSAMD51P19A Many thanks for the information about SVD files.

I first tried the python tool on the link you have pointed out, but it didn't work for ATSAMD51P19A. It seems unable to handle arrays in a cluster correctly, namely the PORT peripheral. The tool above uses the python XML parser directly, and only tested on ATSAMD51P19A.svd.

[tannewt]

Ah, very true! They are giant. Splitting the peripherals could help.