opal-prd: mmap(range:ibm,hbrt-code-image,...) fails with EPERM

[gyakovlev]

Hi, on Raptor Talos II I used to run opal-prd just fine, but recently re-installed the system and now can't launch opal-prd.

opal-prd compiled from 6.6.3 tag.

error happens in map_hbrt_physmem

https://github.com/open-power/skiboot/blob/f901fcafae14d38e29f1cc11440086ee678785d0/external/opal-prd/opal-prd.c#L984-L991

Any ideas why mmap returns EPERM here?

full debug output

 opal-prd --debug --stdio
CTRL: Starting PRD daemon

I2C: Found Chip: 00000000 engine 2 port 0
I2C: Found Chip: 00000000 engine 1 port 0
I2C: Found Chip: 00000008 engine 3 port 0
I2C: Found Chip: 00000008 engine 1 port 0
I2C: Found Chip: 00000000 engine 3 port 0
I2C: Found Chip: 00000000 engine 1 port 2
I2C: Found Chip: 00000008 engine 3 port 1
I2C: Found Chip: 00000008 engine 1 port 2
I2C: Found Chip: 00000000 engine 3 port 1
CTRL: Listening on control socket /run/opal-prd-control
FW: 17 PRD ranges, instances assigned by firmware
FW:  0000203ffcbf0000-0000203ffcceffff HCODE [0]
FW:  0000203ffcfa0000-0000203ffd09ffff OCC [0]
FW:  0000203ffcbb0000-0000203ffcbdffff RINGOVD [0]
FW:  0000203ffcbe0000-0000203ffcbeffff VERSION [0]
FW:  0000203ffccf0000-0000203ffcf9ffff WOFDATA [0]
FW:  0000203ffd700000-0000203ffd7fffff ibm,arch-reg-data [0]
FW:  00000000f0000000-00000000ffffffff ibm,hb-rsv-mem [0]
FW:  0000203ffd0f0000-0000203ffd54ffff ibm,hbrt-code-image [0]
FW:  0000203ffd550000-0000203ffd6fffff ibm,hbrt-data [0]
FW:  0000203ffd800000-0000203ffdbfffff ibm,homer-image [0]
FW:  0000203ffdc00000-0000203ffdffffff ibm,homer-image [8]
FW:  0000203fff800000-0000203fffffffff ibm,occ-common-area [0]
FW:  0000203ffd0c0000-0000203ffd0cffff ibm,sbe-comm [8]
FW:  0000203ffd0e0000-0000203ffd0effff ibm,sbe-comm [0]
FW:  0000203ffd0b0000-0000203ffd0bffff ibm,sbe-ffdc [8]
FW:  0000203ffd0d0000-0000203ffd0dffff ibm,sbe-ffdc [0]
FW:  0000203ffd0a0000-0000203ffd0affff ibm,secure-crypt-algo-code [0]
FW: Chip init
FW: Chip 0x8
FW: Chip 0x0
IMAGE: can't find code region hbrt-code-image
IMAGE: mmap(range:ibm,hbrt-code-image, phys:0x0000203ffd0f0000, size:0x0000000000460000) failed: Operation not permitted
IMAGE: Can't access hbrt physical memory
CTRL: stopping PRD daemon

here's the reg hexdump, numbers seem to match.

od -x --endian=big /sys/firmware/devicetree/base/reserved-memory/ibm\,hbrt-code-image@203ffd0f0000/reg
0000000 0000 203f fd0f 0000 0000 0000 0046 0000

and full strace, just in case. opal-prd-stace.txt

Thanks in advance.

[klauskiwi]

This is a known-issue caused by modern distros using a newer systemd vresion that mounts devtmpfs as noexec in /dev..

There was some discussions in fixing this by copying the hbrt image to a standard memory location, and then executing it from there, instead of trying to execute directly from it's location on the reserved memory range, but no one came around to implement it yet (patches welcome!)

For now, the workaround is to remount the /dev filesystem without noexec, and re-start the opal-prd daemon.

-Klaus

[gyakovlev]

I tried mounting /sys without noexec, but it was ~/proc~ /dev, makes sense now, thanks for explanation! I can try making an attempt at writing a patch as time permits, but my C is rather weak, so no promises.

edited for clarity

[klauskiwi]

That's odd - it should have been devtmpfs mounted at /dev with noexec preventing opal-prd to run.

[gyakovlev]

you are right, my brain meant /dev but fingers typed /proc for some reason.

this line in fstab devtmpfs /dev devtmpfs nosuid,mode=755 0 0 works around the problem for now, but I'll still make an attempt at a patch as I have time later this week.

[oohal]

I think @hegdevasant was looking at fixing this, but you're also welcome to try. We figured allocating some memory with an anonymous mmap() that's initially mapped R+W, copying the HBRT image into there then using mprotect() to remap it as R+X would do the trick.

[gyakovlev]

I've sent patch to skiboot list for review, it was easier than I imagined, thanks for guide how to do it. works now on my system with /dev mounted with noexec.