Could the framebuffer driver be updated to provide the ability to create custom screen modes whose pixels have an arbitrary aspect ratio?

[Grasshopper2]

I've been experimenting with the Advance MAME emulator on my Raspberry Pi 2.

Advmame has the ability to program the framebuffer driver to create a custom display mode that has exactly the same resolution as the game being played. And for performance reasons, I'd prefer to do this instead of having the emulator do the scaling itself.

On a regular PC running Linux, the custom display generation works extremely well. However, on my RPi the results have been a bit unpredictable.

After a lot of experimentation, it finally dawned on me that the RPi framebuffer driver always scales the image by the same factor in both the x and y planes, regardless of the resolution you have selected.

If the resolution you've selected won't fit exactly on the screen, the RPi driver compensates by adding black bars to the left and right, or the top and the bottom of the usable display area. This is very different to how a traditional framebuffer on a PC version of Linux works. On a traditional framebuffer, the usable display area is typically stretched to fit the entire screen, regardless of the resolution chosen. This means that sometimes the custom display mode will end up having pixels that don't have a square aspect ratio.

Most of the time the behaviour of the RPi's framebuffer is appropriate. However, when emulating old hardware, there are times when it's necessary to create a custom display mode that has pixels that are not square.

I would therefore like to put in a request for the RPI's framebuffer to be updated to allow the creation of screen modes whose pixels have an arbitrary aspect ratio.

Thanks in advance.

[popcornmix]

You can do this by using the dispmanx API rather than the frame buffer - is this an option for you? See: https://github.com/raspberrypi/firmware/tree/master/opt/vc/src/hello_pi/hello_dispmanx

[Grasshopper2]

Unfortunately, I don't know much about the dispmanx API. I've done quite a lot of googling, and the imformation out there seems patchy, to put it mildly.

However, from what little I've learned, I'd say the answer is no, or at least not without doing a huge amount of work. The Linux version of AdvanceMAME uses the SVGALIB library and FrameBuffer kernel service to directly program the video board. As far as I can tell, neither of these options utilise the dispmanx API.

[popcornmix]

Can you provide a concrete example?

Describe what the framebuffer resolution and aspect ratio is. Describe what the HDMI mode's resolution and aspect ratio is.

Describe how you would like the framebuffer to fill the screen (e.g. should it use every hdmi display pixel, or should it have black bars at top/bottom or left/right.

Should the image be displayed without distortion? (e.g. should circles be circular or ovals?) Should the framebuffer be scaled when sent to HDMI output?

Note: in the HDMI world, there only exist a small set of aspect ratios that can be communicated to display (1=4:3, 2=14:9, 3=16:9, 4=5:4, 5=16:10, 6=15:9). You can't tell a display use a completely arbitrary aspect ratio.

[Grasshopper2]

Hi,

It's quite hard to explain, However, I provided an example at the end of this thread on the RPi forums:

https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=136899

With regards to your final question, I don't believe the aspect ratio that is reported as part of the HDMI signal necessarily has to match the real-life aspect ratio that you see on the screen.

[popcornmix]

With regards to your final question, I don't believe the aspect ratio that is reported as part of the HDMI signal necessarily has to match the real-life aspect ratio that you see on the screen.

No, but if aspect ratio do not match then resizing by the Pi is required which adds some blurriness, especially for text.

[popcornmix]

A useful command is:

pi@domnfs:~ $ vcgencmd dispmanx_list
display:2 format:XRGB8888 transform:0 layer:-127 src:0,0,1824,984 dst:48,48,1824,984 cost:1103 lbm:0

Note the "src" and "dst" rectangles. First two number are x,y offset and final two numbers are width,height. Note that if the width,height of src and dst match then no resizing is done and you get a sharper image.

Perhaps you can grab that for an example when mame is running and explain what numbers you would like it to be using.

[Grasshopper2]

Unfortunately, when I run "vcgencmd dispmanx_list" I get a "Command not Registered" error.

I'll update my firmware (assuming that's the problem) and get back to you.

[Grasshopper2]

OK. In order to get the dispmanx_list option working, I've installed the test firmware that you provided in the other thread. Unfortunately, my initial experiments suggest that it's not working as expected. However, one problem at a time.

My display is currently set to 1920x1080 in config.txt. As an experiment, I tried setting two different resolutions with fbset.

The first resolution was 512x224. I chose this resolution because it produces black bars at the top and bottom of the usable display area. To carry out the test, I ran the following commands:

fbset -g 512 224 512 224 8 vcgencmd dispmanx_list

The output from vcgencmd was:

display:2 format:8BPP transform:0 layer:-127 src:0,0,512,224 dst:0,120,1920,840 cost:1186 lbm:9216

I then chose a resolution of 640x480 because it produces black bars at the left and right of the usable display area. This time I ran the following commands:

fbset -g 640 480 640 480 8 vcgencmd dispmanx_list

The output from vcgencmd was:

display:2 format:8BPP transform:0 layer:-127 src:0,0,640,480 dst:240,0,1440,1080 cost:919 lbm:11264

It appears that, in the case of the 512x224 mode, the dst y values (both offset and size) are being adjusted to produce the black borders. However, the dst x values are left alone.

For the 640x480 mode, the opposite happens. The dst x values (both offset and size) are being adjusted to produce the black borders. However, this time it is the dst y values that are left alone.

Therefore, I would assume that, in order to create a display mode where the usable area completely fills the screen, the dst values should always be set as follows:

dst:0,0,1920,1080

[popcornmix]

But do you really want it to fill the screen? In the thread you described the game as being played on a 4:3 monitor (presumably filling that screen). If you play it on a 16:9 monitor then everything will appear too wide. Wouldn't you want it to just use the central 4:3 part of monitor (with black bars on sides)? E.g. dst:240,0,1440,1080 ?

[Grasshopper2]

Wow. That was a quick response.

To answer your question, yes I do want it to fill the screen. I've already manually calculated the adjustment needed for a widescreen monitor.

The games's resolution is 384x224 but I've chosen a resolution of 512x224. The extra 128 pixels in the x plane are to allow for space on the left and right of the game's display area to compensate for the fact that I'm using a widescreen monitor.

I worked this out manually (see link I provided for details). However, AdvMAME will actually do this calculation for you automatically.

[popcornmix]

If you add sdtv_aspect=1 to config.txt, I think you will never get the black bars. (Technically sdtv_aspect should only apply to composite output, but there is a bug where it also affects hdmi output).

[Grasshopper2]

I've just tried adding sdtv_aspect=1 to my config.txt file and it works!

Thanks for looking into this. That's really helpful. However, I'm a little concerned about relying on a 'bug' that could potentially be fixed by a future firmware update. Could this option be put on a more permanent footing?

Also, it is possible to change the sdtv_aspect setting 'on the fly' (i.e. without having to carry out a reboot)?

[popcornmix]

There is a bit of magic that doesn't work for all settings (some are cached or only evaluated at boot) but it should work for this: vcgencmd get_config sdtv_aspect set 1

[popcornmix]

I agree you should solve this without the 'bug'.

I think you shouldn't be creating the 512x224 framebuffer. You really want to create a framebuffer that is 384x224 with a pixel aspect ratio of 7:9.

You then should find the normal aspect ratio preserving scaling will insert the correct amount of black bars. I believe you want:

pi@domnfs:~ $ vcgencmd dispmanx_list
display:2 format:XRGB8888 transform:0 layer:-127 src:0,0,384,224 dst:240,0,1440,1080 cost:919 lbm:6144

i.e. on a 1920x1080 display, 240 pixels of black bars on left and right. 1440x1080 pixels are used, which is a 4:3 image.

Does that sound right?

[popcornmix]

There doesn't seem to be way of specifying the source pixel aspect ratio through fbset, so I've added a config.txt setting until a better solution is found. This is actually a 16.16 fixed point number. To set it to 7:9 you would want framebuffer_aspect=0x00070009

You can set this at runtime:

pi@domnfs:~ $ vcgencmd get_config framebuffer_aspect set 0x70009
pi@domnfs:~ $ fbset -xres 384 -yres 224
pi@domnfs:~ $ vcgencmd dispmanx_list
display:2 format:XRGB8888 transform:0 layer:-127 src:0,0,384,224 dst:240,0,1440,1080 cost:919 lbm:6144

Does this work for you? This feature is in latest rpi-update firmware.

[Grasshopper2]

I think you shouldn't be creating the 512x224 framebuffer. You really want to create a framebuffer that is 384x224 with a pixel aspect ratio of 7:9.

I agree that, under most circumstances, it would make more sense to create a framebuffer that is 384x224 with a pixel aspect ratio of 7:9. It is, after all, wasteful to assign memory to a part of the screen that is never going to be written to.

Unfortunately, in the case of Advance MAME, and possibly other programs of a similar age (2006 and earlier), that approach doesn't really work.

The algorithm that Advance MAME uses for generating an exact video mode is based on the assumption that the usable display area will always completely fill the screen, even if that means that the pixels are not square. The program was written in an era when most people were still using CRT monitors, and that is how displays worked at that time.

You can switch off Advance MAME's automatic video mode generating facility, and define the modes manually. But I think that would be a shame, as it works pretty well.

[popcornmix]

Okay, what I said should still work with the 512x224 framebuffer. You still want a pixel aspect ratio of 7:9. There will be a little unnecessary memory and bandwidth wasted to transfer the black pixels but it's unlikely to be noticeable.

[Grasshopper2]

Okay, what I said should still work with the 512x224 framebuffer. You still want a pixel aspect ratio of 7:9. There will be a little unnecessary memory and bandwidth wasted to transfer the black pixels but it's unlikely to be noticeable.

Yes, that would work. However, you'd still need to manually calculate and set the framebuffer_aspect parameter on a per-game basis, so you wouldn't have completely automatic video mode generation.

I don't have an objection to the framebuffer_aspect parameter per se. However, I wouldn't like it to be seen as a replacement for sdtv_aspect=1.

[Grasshopper2]

OK. I've been pondering the best way to solve this problem.

I think there are, broadly speaking, two approaches you can take (which are not necessarily mutually exclusive).

The first is to allow the user to read and write the raw parameters that control GPU scaling, and then leave it up to the user (or program written by the user) to perform the necessary calculations to create the required video mode. It would be nice, for instance, if all the parameters displayed by running 'vcgencmd dispmanx_list' were under direct user control.

The second approach would be to allow the user to specify the characteristics of the display mode that he/she is trying to create, and then have the firmware drivers perform the necessary calculations to achieve that aim.

If one chooses the second approach, then I think, from the user's perspective, it might be more intuitive to allow him/her to specify the required aspect ratio of the usable display area, and then set a flag that's the equivalent of sdtv_aspect=1 (i.e. always completely fill the usable display area regardless of the resolution chosen). An advantage of this approach is that it mimics the behaviour of PC Linux based framebuffers, and therefore allows programs such as Advance MAME to work correctly out of the box.

With your approach, you're requiring the user to manually calculate the pixel's aspect ratio from the usable display area's aspect ratio, and the game's actual resolution. You then work backwards from this to recalculate the usable display area's aspect ratio. This seems to me, to be an unnecessarily circular way to go about things. And if you're going to require the user to calculate something, then you may as well require the user to calculate everything.

[popcornmix]

I could allow framebuffer_aspect=0xffffffff (same as framebuffer_aspect=-1) to be interpreted as "unknown aspect ratio - just full the screen", so same effect as sdtv_aspect=1, but a documented behaviour.

[Grasshopper2]

_I could allow framebuffer_aspect=0xffffffff (same as framebuffer_aspect=-1) to be interpreted as "unknown aspect ratio - just full the screen", so same effect as sdtvaspect=1, but a documented behaviour.

Yes, that might work.

Incidentally, why are you using hex numbers when other parameters in config.txt are expressed in decimal?

[Grasshopper2]

I haven't had a chance to test the framebuffer_aspect parameter yet (I'm already juggling two sets of firmware...). However, as a quick experiment, I thought I'd see if I could achieve the same result by manipulating the overscan parameters.

I put the following parameters into config.txt

sdtv_aspect=1
overscan_left=240
overscan_right=240
overscan_top=0
overscan_bottom=0
disable_overscan=1

I then rebooted the RPi, and ran the following commands:

fbset -g 384 224 384 224 16
vcgencmd dispmanx_list

The output was a follows:

display:2 format:RGB565 transform:0 layer:-127 src:0,0,384,224 dst:240,0,1440,1080 cost:919 lbm:6144

So it does appear to work. Maybe this is another option to consider.

Incidentally, I initially tried it without the disable_overscan=1 parameter, and to my surprise 48 pixels were added to all four of the other overscan parameters. Is this expected behaviour? I'd always assumed that disable_overscan=1 was just a shorthand for setting all the other overscan parameters to zero.

I'm also slightly confused about how the sdtv_aspect parameter works. From what I've been reading, it has a default value of 1. So why does setting it to 1 make any difference?

[Grasshopper2]

_I could allow framebuffer_aspect=0xffffffff (same as framebuffer_aspect=-1) to be interpreted as "unknown aspect ratio - just full the screen", so same effect as sdtvaspect=1, but a documented behaviour.

On the other hand, now I think about it, would there ever be a requirement for a negative pixel aspect ratio (which would presumably create a mirror image of what's on the screen)? Can the GPU even do that?

How about a second parameter called something like 'preserve_pixel_aspect_ratio' that is set to 0 or 1, with 1 being the default?

Also, could 'framebuffer_aspect' be changed to something like 'framebuffer_pixel_aspect'. I think 'framebuffer_aspect' on its own could be interpreted in different ways.

[Grasshopper2]

Alternatively, you could dodge the issue of whether a negative aspect ratio is valid, by saying that framebuffer_aspect=0 is to be interpreted as "unknown aspect ratio - just full the screen".

[popcornmix]

framebuffer_aspect=0 is to be interpreted as "unknown aspect ratio

In general config.txt parameter default to zero, so setting to zero or not setting them is equivalent (there are one or two exceptions but that requires some ugly code to handle). So I'd prefer to use -1 rather than 0 for a non-default setting.

[Grasshopper2]

In general config.txt parameter default to zero, so setting to zero or not setting them is equivalent (there are one or two exceptions but that requires some ugly code to handle). So I'd prefer to use -1 rather than 0 for a non-default setting.

But surely the default should be: framebuffer_aspect=0x00010001 ?

[popcornmix]

The default is framebuffer_aspect=0 which is interpreted as no custom aspect ratio (so same as previous behaviour, i.e. 1:1). framebuffer_aspect=0x00010001 would have the same behaviour.

[Grasshopper2]

OK. Thanks for clarifying that. So zero represents a kind of null or undefined value. That makes more sense now. However, I think it's unfortunate that you've placed that limitation upon yourself because there will inevitably be situations when zero represents a valid value.

But anyway, if that's the way things have to be, then it does increase the case for simply having a separate parameter that defines the scaling behaviour. That approach would also allow for the possibility of creating other scaling options later on, without breaking backwards compatibility.

Incidentally, can you tell me why you're using hex numbers when other parameters in config.txt are expressed in decimal?

Thanks

[popcornmix]

Incidentally, can you tell me why you're using hex numbers when other parameters in config.txt are expressed in decimal?

All paramaters can be specified in hex or decimal - they are just 32-bit integers.

framebuffer_width=0x320
framebuffer_height=0x1e0

would be perfectly valid.

[pelwell]

For packed values I find 0x00010001 more readable than 65537 (easy), and 0x000B000A preferable to 720906 (harder).

[alessioscand]

Hi, I'd like to thank you for all your work: allowing AdvanceMAME (and other emulators) to easily generate custom video modes on RPi is a great feature. So can I ask if it's confirmed that, in the latest firmware (via rpi-update)

sdtv_aspect=1

has been replaced (also for HDMI) by

framebuffer_aspect=-1

...is that actually correct? Thank you again.

[popcornmix]

...is that actually correct?

Not currently, but I'll add it in next firmware update

[alessioscand]

Thank you very much for your kind feedback!

[Grasshopper2]

Hi, it's great news that this facility is going to be added to the main firmware branch. However, could we have solution that allows the three parameters to be set separately, instead of having them all packed into a single 32bit number?

[popcornmix]

@Grasshopper2 no plans to change the current packed number. It's an advanced feature that is not expected to be useful to many, so I think the small amount of effort in packing the numbers is justified.

[popcornmix]

@Antiriad the framebuffer_aspect=-1 setting is supported in current rpi-update firmware.

[alessioscand]

@popcornmix thank you very much!

So

framebuffer_aspect=-1

is actually the same as

sdtv_aspect=1

...right?

[popcornmix]

The effect is currently the same, but sdtv_aspect should only really affect composite output and not hdmi. Having an effect on hdmi is a bug. I'd suggest you use framebuffer_aspect=-1 if you want the framebuffer stretched to fit screen whilst ignoring aspect. You may find using sdtv_aspect stops working on hdmi at some point in the future.

[JamesH65]

@popcornmix and other. Can this be closed? Seems to have fixed the original request.

[JamesH65]

Closing this issue as questions answered/issue resolved. Closing due to lack of activity. Please request to be reopened if you feel this issue is still relevant.

[alessioscand]

@popcornmix pardon me if I write here but I suppose it is relevant: there is a way, using framebuffer_aspect, to make the composite out to output a 768 horizontal resolution (instead of 720) of non-square pixels? Thank you!

[popcornmix]

I don't have a convenient composite display, and the framebuffer_aspect was added almost 4 years ago so can't say for sure. Why not test it? You'll probably want framebuffer_width=768 (although without a description of exactly what you are trying to do, it's hard to advise).

[alessioscand]

@popcornmix thank you very much for your kind feedback. What I'm trying to achieve is related to the https://accentual.com/bmc64/ project, a bare metal Commodore 64 emulator. The horizontal resolution of the emulator is 384px, while the framebuffer size of RPi seems to be limited, in composite mode, to 720px. Since a horizontal resolution of 768px is not allowed, integer scaling is not possible. So I wonder if using non-square pixels could fix this... Thank you.

[popcornmix]

So what happens when you set

sdtv_aspect=1
framebuffer_width=768
framebuffer_height=480 (or whatever you wanted)

Do you get an image? Does it fill screen? What difference are you hoping for?

[alessioscand]

Actually I get an image but unfortunately I don't see any change (as if the options have no effect). In the emulator I am always bound to a maximum horizontal resolution of 720px, now I wonder if the options are not working anymore, or if it depends on the software emulator (RPi3 B+, always on composite-out).

Thanks

[randyrossi]

BMC64 doesn't use the default frame buffer any more. That's why those options have no effect. It creates its own layers using dispmanx API and lets users adjust the frame buffer width/height as well as how the frame buffer is scaled/positioned from within the app. For composite out, I suggest keeping scaling interpolation on and not worry about integer scaling in the horizontal dimension. You want 2x multiple on the frame buffer height and adjust the vertical position and scaled frame buffer height to match (more or less) the visible lines on your CRT. The interpolation takes care of smoothing out the irregularities from non integer scaling on the horizontal dimension. You can also get the aspect ratio to match the real thing this way too. Even if you had a larger H resolution and you managed integer scaling, the aspect ratio would be wrong.

[tdaniel22]

Hasn't the framebuffer_aspect setting been removed entirely? Using the RPI kernel 4ed0f31 it was working, but after updating to 804d8f8 it doesn't work anymore (and I think this goes back to summer of last year already).

But now I need to update my kernel so I'm stuck again on the 7" touchscreen with the default framebuffer size and a stretched image. It also seems to have broken the touchscreen input rotation but that's probably a separate issue.

Edit: well nevermind, I haven't touched the firmware version so that's probably just a kernel issue.

[popcornmix]

If you can narrow down when the behaviour changed it would help. The two kernel commits you mentioned are a year apart. framebuffer_aspect is handled by the firmware and it's not been removed. It's behaviour may be influenced by the kernel version but without a description of exactly what the change in behaviour is or a specific commit in the kernel tree that changes the behaviour I can't really guess.

[tdaniel22]

Basically, to compensate for the rectangular pixels of the official 7" touchscreen, I was using these options:

framebuffer_width=860
framebuffer_height=480
framebuffer_aspect=-1

And with commit 4ed0f31 the result was as expected: the system was rendering to a 860x480 framebuffer, which was then squeezed to fit the screen, and I ended up with roughly square pixels (with the caveat that the image wasn't pixel perfect).

Somewhere around july last year I tried updating my kernel (staying on the 4.19 branch) but it broke the custom framebuffer setting. The framebuffer was back to the native display resolution, as if all the framebuffer_ options where ignored. I didn't really need the latest kernel at the time so didn't bother searching for a solution, but know I really need it. And the current 4.19 kernel still shows the same behaviour. I simply ended up giving up on the framebuffer_ options and am now doing the ratio compensation in my OpenGL app. (And actually, I am getting better frame times and no screen tearing. Not sure if this is due to other optimisations in the kernel or if the framebuffer resizing thing was inducing those in the first place.)

For context, my system is a buildroot setup on a RPI 3B+. I am using the vc4-fkms-v3d overlay, and my program is an SDL2/GLES2 app running in a cage compositor (wayland, not X).

Because the framebuffer_aspect property disappeared from the config.txt docs, I assumed it had been removed. Well, I think it was documented ..

Now, if I'm not mistaken, the RPI firmware is independant from the kernel in Buildroot. So I should have been running the latest firmware (or at least the last one packaged by upstream Buildroot). If that's correct, then I should be able to bisect the exact kernel commit (which is going to be fun).

[popcornmix]

framebuffer_aspect is still present in latest firmware. If you are able to switch between an older (working) kernel and recent (not working) kernel then it would be useful to confirm that both cases are happy with latest firmware. If they are, then bisecting kernels should provide useful information.