N900: screen rotation segfaults in sgx_exa_update_pixmap

[MerlijnWajer]

Rotating the screen, e.g. with

xrandr -o right

Crashes the X in sgx_exa_update_pixmap in https://github.com/maemo-leste/xf86-video-pvrsgx/

(gdb) bt
#0  0xb6716e96 in sgx_exa_update_pixmap () from /usr/lib/xorg/modules/drivers/pvrsgx_drv.so
#1  0xb67187b8 in update_flip_pixmaps () from /usr/lib/xorg/modules/drivers/pvrsgx_drv.so
#2  0xb6718814 in PVR2D_PostFBReset () from /usr/lib/xorg/modules/drivers/pvrsgx_drv.so
#3  0xb670981a in post_fb_reset () from /usr/lib/xorg/modules/drivers/pvrsgx_drv.so
#4  0xb670ae48 in fbdev_crtc_config_resize () from /usr/lib/xorg/modules/drivers/pvrsgx_drv.so
#5  0x004fd378 in xf86RandR12ScreenSetSize (pScreen=0x61d318, width=<optimized out>, height=<optimized out>, mmWidth=211, mmHeight=127)
    at ../../../../../../hw/xfree86/modes/xf86RandR12.c:698
#6  0x0053255e in ProcRRSetScreenConfig (client=<optimized out>) at ../../../../randr/rrscreen.c:1088
#7  0x004ab3e6 in Dispatch () at ../../../../dix/dispatch.c:478
#8  0x004adfbe in dix_main (argc=11, argv=0xbe9aec94, envp=<optimized out>) at ../../../../dix/main.c:276
#9  0xb6a0b524 in __libc_start_main (main=0x49ef9d <main>, argc=11, argv=0xbe9aec94, init=<optimized out>, fini=0x59e3e1 <__libc_csu_fini>,
    rtld_fini=0xb6f5b6c5 <_dl_fini>, stack_end=0xbe9aec94) at libc-start.c:308
#10 0x0049efd4 in _start ()

Oddly enough there is no debug symbols available at all inside the sgx code:

(gdb) info locals
No symbol table info available.

[MerlijnWajer]

I rebuilt the package with -ggdb and now the debug symbols are usable, the problem is that the page_flip->bufs[i].pixmap is NULL in update_flip_pixmaps.

I believe this is the case because PVR2D_PreFBReset calls pvr2d_page_flip_destroy which just memset()s the page flip to 0, thereby removing the reference to the pixmap (which only gets created in PVR2DCreateScreenResources using sgx_exa_create_pixmap).

PVR2D_PostFBReset attempts to restore the page flip using pvr2d_set_frame_buffer which calls pvr2d_page_flip_create, but this function critically does not (re) create any pixmaps.

So I think a potential solution could be to call PVR2DCreateScreenResources(scrn_info->pScreen) in PVR2D_PostFBReset before calling update_flip_pixmaps.

[MerlijnWajer]

So the above fixes the segfault, but then xrandr reports:

# xrandr -o right
Failed to change the screen configuration!

And the screen turns black.

I suppose one of the problems is that the memset I discussed above actually overwrites pixmap pointers that it should not override (or if it does, at least it should free them)

[MerlijnWajer]

It looks like something fails:

[   162.628] (WW) PVRSGX(0): omap/sysfs: can't write to '/sys/devices/platform/omapdss/display0/enabled': 22:Invalid argument
[   162.631] (EE) PVRSGX(0): Unable to enable output

-22 is invalid argument, so perhaps it's trying to write an invalid value, but that might just be a red herring.

I think that perhaps after post_fb_reset in fbdev.c the GetScreenPixmap call fails because of the pixmap overriding as I mentioned before. Requires more stepping with gdb.

[MerlijnWajer]

In addition to fixing this problem, I should also test it with tv output.

Looks like this is the next cause of failure:

[   267.926] omap_fb_alloc: ioctl(OMAPFB_SETUP_MEM) failed 12:Cannot allocate memory
[   267.930] omap_fb_alloc: failed omapfb_mem_info
   size 976158720
   type 0

That size seems kind of large.

[MerlijnWajer]

Also, unrelated, but we probably want to fix this for the N900 resolution:

/* SGX has a 2048x2048 maximum texture size limit */
/* FIXME should be limited further based on the amount of vram */
xf86CrtcSetSizeRange(pScrn, 1, 1, 2048, 2048);

[MerlijnWajer]

Upon re-run the ioctl memory error did not occur, so perhaps that's related to something else, still, the code seems a little off.

[MerlijnWajer]

It looks like the enabled write fails because of this (the write returns EINVAL even though the value is '1' which is correct, but it doesn't seem to like to underlying state):

[  852.551239] omapdss OVERLAY error: overlay 0 vertically not inside the display area (0 + 800 >= 480)
[  852.551269] omapdss APPLY error: failed to enable manager 0: check_settings failed
[  852.551391] acx565akm_panel_power_on sdi enable failed

[MerlijnWajer]

This is now fixed by @freemangordon with the new omap ddx + kernel

[MerlijnWajer]

As a clarification, rotation doesn't -work- yet, but it doesn't segfault.