search

monome / norns

norns is many sound instruments.
http://monome.org
GNU General Public License v3.0
614 stars 144 forks source link

low-level screen improvements #1600

Closed tlubke closed 1 year ago

tlubke commented 1 year ago

Gamma

Related PRs:

Cairo Surface

Related PRs

tlubke commented 1 year ago

made a lot of progress on the cairo A4 format item this week. the norns side is pretty much implemented, just have to go back to my libcairo branch now and find what I missed.

tlubke commented 1 year ago

Found there was nothing wrong with my cairo branch in the end, it just needs to be configured with XCB surfaces.

sudo apt-get install libxcb1-dev libxcb-render0-dev cd cairo; sudo ./autogen.sh --enable-xcb

However, I found with A8 and A4 drawing surfaces, some screen text appears to be double width. I think it may have something to do with anti-aliasing. I'm looking into it now.

tlubke commented 1 year ago

However, I found with A8 and A4 drawing surfaces, some screen text appears to be double width. I think it may have something to do with anti-aliasing. I'm looking into it now.

This also appears to be a compilation issue, using the standard package with the same norns code doesn't have this result. Essentially when the text extents of screen.text_center() aren't integer numbers, the text bleeds into other pixels.

I'll see if I can find what options the apt package is compiled with.

tlubke commented 1 year ago

Successfully tested 60 FPS this morning. Only change to the driver was actually clocking down the refresh rate a bit ~111Hz to ~61 Hz. A change to the SPI overlay is what traverses through FBTFT. No noticeable increase in battery usage.

https://user-images.githubusercontent.com/38451140/194125712-556e5172-b4a9-4b1d-9738-ff7aaf3219da.MOV

Will put together a fresh image with gamma controls, Cairo changes, and the FPS all rolled in. After that gonna work on compiling the driver out of tree, and see if I can make it installable with a Debian package. If not, I may try building my own SPI driver.

This is captured with my phone at 120 fps. Just a simple script that increments a number every frame at 60 fps.

tlubke commented 1 year ago

https://github.com/tlubke/norns/releases/tag/220802

tehn commented 1 year ago

very cool, and surprised the driver didn't need a more major change

i can test out the image here. the video above is a bit confusing, but perhaps it's a web codec issue on my end? it counts up to 60 twice over the course of 10 seconds?

tlubke commented 1 year ago

... perhaps it's a web codec issue on my end? it counts up to 60 twice over the course of 10 seconds?

@tehn Sorry, should've left more of an explanation. This is just proof via slow-motion capture that every frame counting up to 60 is able to be shown, I just let it loop twice. Running the same script without the image shows only every 3rd or 4th number.

Script:

c = 0
fps = 60

function init()
  clock.run(function()
    while true do
      clock.sleep(1/fps)
      redraw()
      c = c+1
    end
  end)
end

function redraw()
  screen.clear()
  screen.move(32,32)
  screen.text_center(c % fps)
  screen.update()
end
tehn commented 1 year ago

apologies for the delay. just checking the build info for the image you posted: is it for a shield?

copied it to a standard norns and getting this error when trying to run matron (from the command line, since the system service failed):

build/matron/matron: error while loading shared libraries: /lib/arm-linux-gnueabihf/libpixman-1.so.0: invalid ELF header
tlubke commented 1 year ago

@tehn

apologies for the delay. just checking the build info for the image you posted: is it for a shield?

No, the image should be for the standard norns. I will double check the image later today, you can reinstalling libpixman-1-dev with apt to see if that fixes the error.

I've also thought of another way of accomplishing this that perhaps I could get your thoughts on, since it wouldn't require changes to the kernel.

We can change the device overlay (it's not built into the kernel) to make the SPI behave as just a generic SPI device, and interact with the screen in the userspace via matron. This not only opens up a lot more possibilities for future improvements, but also removes the need for a frame buffer abstraction in the first place since we could copy the cairo surface directly from screen.update(). Additionally the gamma and brightness settings would become much more straightforward, and one would be able to implement things like adaptive refresh rates or frame limiting.

I was working on this a bit last week. I can share more when I have a proof of concept.

tehn commented 1 year ago

@tlubke thanks for the tip, i'll attempt to re-install the lib

honestly i'd very much like to avoid a kernel change if possible--- we just recently put the entire userbase through re-imaging and it's perceived (rightly so) as a hassle. but i think 60fps is worth it, personally :)

so yes, i'm very interested in your proposal. i don't have any issues philosophically with userspace SPI. in general i'd like to think of the norns more as an embedded system rather than a computer (if that makes sense).

tlubke commented 1 year ago

@tehn I know norns hardware isn't open source, but is there a schematic I could reference? Is it the same pin layout as the shield?

I've got SPI devices set up in /dev/, and I'm controlling the GPIOs for DC and Reset. Not sure, but I think I might need to know chip select pins.

tehn commented 1 year ago

here's the pinout: https://github.com/monome/norns-image/blob/main/readme-hardware.md#pinout-1

let me know if you need more, happy to provide

On Tue, Oct 18, 2022 at 6:27 PM tlubke @.***> wrote:

@tehn https://github.com/tehn I know norns hardware isn't open source, but is there a schematic I could reference? Is it the same pin layout as the shield?

— Reply to this email directly, view it on GitHub https://github.com/monome/norns/issues/1600#issuecomment-1283079258, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB4I4EB2HQ5XEJBTL3SLW3WD4P4JANCNFSM6AAAAAAQKYXZLM . You are receiving this because you were mentioned.Message ID: @.***>

tehn commented 1 year ago

wow. drawing some 60 fps circles with slowly pulsing radii. it's really, really impressive!

it'd be great to have some broader testing with this big change--- i'm curious to see how certain heavy scripts react to the new capability. ie, if people can easily convert their scripts to 60fps with a few character edits, this might have ripple effects on the performance of their script. so we may need to do some more docs on optimization.

(ps, ended up my first download of the image was corrupted, hence my troubles)

any further insights into userland spi?

tlubke commented 1 year ago

@tehn

it'd be great to have some broader testing with this big change--- i'm curious to see how certain heavy scripts react to the new capability. ie, if people can easily convert their scripts to 60fps with a few character edits, this might have ripple effects on the performance of their script. so we may need to do some more docs on optimization.

Agreed. I think optimization docs should come first, but having SPI in the userspace might allow matron to smartly throttle things and give users a warning. The few scripts I've seen that come closing to maxing out the CPU don't have very intense screen operations. I assume there is no current priority to one system over another (i.e. screen over clock over super collider)?

any further insights into userland spi?

I believe I'm close, but I may need some help or some more eyes to see what I'm missing. I've tried both using linux IOCTL, and using the BCM2835 library. I just uploaded branches. As a proof of concept, I am trying to turn on "ALL ON" mode, and get the whole screen to light up, the branches only set normal mode like in the driver init.

For both of the branches, I experimented with setting different SPI modes, chip selects, and write speeds.

IOCTL

Default SPI enabled via raspi-config and editing /boot/config.txt

...
# Display
# Connected using SPI
dtparam=spi=on
#dtoverlay=ssd1322-spi

https://github.com/tlubke/norns/tree/tlubke/userspace-spi-IOCTL

I am able to see that the right amount of bytes are being written to spidev0.0, and I don't see any error messages on the GPIO writes. Nothing appears to happen on the screen though.

BCM2835

Since I saw we use the driver in the overlay, figured I would try using the library. Use of the library's SPI function requires that SPI be disabled in raspi-config, and matron has to be run as sudo. I didn't have access to a device file system to check if bytes were being written, so it was a bit of a shot in the dark.

https://github.com/tlubke/norns/tree/tlubke/userspace-spi-bcm2835

Side note: I see there is a discussions page in the norns repo now, should I move this issue there (if that is easy/possible)?

tehn commented 1 year ago

this would be a great use of the discussions page!

On Tue, Oct 25, 2022 at 2:56 PM tlubke @.***> wrote:

@tehn https://github.com/tehn

it'd be great to have some broader testing with this big change--- i'm curious to see how certain heavy scripts react to the new capability. ie, if people can easily convert their scripts to 60fps with a few character edits, this might have ripple effects on the performance of their script. so we may need to do some more docs on optimization.

Agreed. I think optimization docs should come first, but having SPI in the userspace might allow matron to smartly throttle things and give users a warning. The few scripts I've seen that come closing to maxing out the CPU don't have very intense screen operations. I assume there is no current priority to one system over another (i.e. screen over clock over super collider)?

any further insights into userland spi?

I believe I'm close, but I may need some help or some more eyes to see what I'm missing. I've tried both using linux IOCTL, and using the BCM2835 library. I just uploaded branches. As a proof of concept, I am trying to turn on "ALL ON" mode, and get the whole screen to light up, the branches only set normal mode like in the driver init.

For both of the branches, I experimented with setting different SPI modes, chip selects, and write speeds. IOCTL

Default SPI enabled via raspi-config and editing /boot/config.txt

...

Display

Connected using SPI

dtparam=spi=on

dtoverlay=ssd1322-spi

https://github.com/tlubke/norns/tree/tlubke/userspace-spi-IOCTL

I am able to see that the right amount of bytes are being written to spidev0.0, and I don't see any error messages on the GPIO writes. Nothing appears to happen on the screen though. BCM2835

Since I saw we use the driver in the overlay, figured I would try using the library. Use of the library's SPI function requires that SPI be disabled in raspi-config, and matron has to be run as sudo. I didn't have access to a device file system to check if bytes were being written, so it was a bit of a shot in the dark.

https://github.com/tlubke/norns/tree/tlubke/userspace-spi-bcm2835

Side note: I see there is a discussions page in the norns repo now, should I move this issue there (if that is easy/possible)?

— Reply to this email directly, view it on GitHub https://github.com/monome/norns/issues/1600#issuecomment-1291004490, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB4I4CJ4Q2ONVXN6BHAOVTWFAUMNANCNFSM6AAAAAAQKYXZLM . You are receiving this because you were mentioned.Message ID: @.***>

tlubke commented 1 year ago

@tehn @dndrks

I don't think I have the repo permissions to do this. Could one of you convert the issue when you find the time?

https://docs.github.com/en/discussions/managing-discussions-for-your-community/moderating-discussions#converting-an-issue-to-a-discussion

tlubke commented 1 year ago

I wonder if it's an endian issue, isn't rpi little endian? Setting SPI_IOC_WR_LSB_FIRST isn't working for me, however.

https://www.kernel.org/doc/Documentation/spi/spidev