support R_386_SEGRELATIVE relocations

[stsp]

I queried the nasm list about R_386_COPY or R_386_SEGRELATIVE relocations: https://forum.nasm.us/index.php?topic=2747.0 but there was no reply.

Whether or not does this mean that we'll have to implement this in nasm on our own, is unclear but the relocations are the whole point of fdpp. Looking into yasm might be interesting too.

@tkchia what was the deal with R_386_SEGRELATIVE relocations? They are mentioned in your doc as a part of Anvin's segelf, but are not in nasm. Are they anywhere?

[tkchia]

Hello @stsp,

R_386_SEGRELATIVE is part of Anvin's segelf proposal (which is available). For some reason, it seems that Anvin has yet to fully implement the proposed relocations in nasm --- though he did create a separate Git branch to work on this a while back.

In the meantime, yes, my binutils-ia16 fork does have support for R_386_SEGRELATIVE and the other proposed segelf relocations. Among other things, ia16-elf-as knows how to output R_386_SEG16 relocations --- even if nasm does not yet do so --- and the gold linker ia16-elf-ld.gold knows how to transform them into R_386_SEGRELATIVE relocations.

Thank you!

[tkchia]

Hello @stsp,

Anyway, one difficulty in implementing the segelf relocation scheme --- whether in nasm or yasm --- is that the assembler needs to be able to automagically emit a bunch of additional symbols and segments (foo!, .data!, etc.), for the whole scheme to really work.

Thank you!

[stsp]

Thanks for info! That's really helpful. I built the experimental nasm but I don't know how to emit the relative relocation. readelf -r kernel.o doesn't show either one is emitted. So how to do that?

whole scheme to really work.

But I don't need the whole scheme because I have converted the linkage to tiny model (and re-link it back at run-time). All I need is to "unhardcode" those TGROUPs, turning them into either a copy or segrelative relocs. Can I do that?

[tkchia]

Hello @stsp,

I built the experimental nasm but I don't know how to emit the relative relocation. readelf -r kernel.o doesn't show either one is emitted. So how to do that?

I think Anvin's implementation is not yet complete. In any case, I think you will most probably have to check with him on how to proceed...

But I don't need the whole scheme because I have converted to linkage to tiny model (and re-link it back at run-time). All I need is to "unhardcode" those TGROUPs, turning them into either a copy ot segrelative relocs. Can I do that?

What exactly are the kind of ELF files you want to build? How will they figure in the whole dosemu2 + fdpp system? Will you need to build them from nasm assembly sources?

Thank you!

[stsp]

What exactly are the kind of ELF files you want to build?

I already do: I switched fdpp to elf recently. Its just a normal ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked.

How will they figure in the whole dosemu2 + fdpp system?

By the custom elf loader: https://github.com/dosemu2/fdpp/blob/master/fdpp/elf.c

Will you need to build them from nasm assembly sources?

Yes, the asm sources are not supposed to be modified (except maybe the mods needed to emit the needed reloc).

[tkchia]

Hello @stsp,

I think a stop-gap solution --- until nasm really gets segelf support --- might be to just leave TGROUP as a "normal" kind of undefined symbol.

You can see that, when building the stock FreeDOS kernel with ia16-elf-gcc, the symbols TGROUP, DGROUP, etc. are simply normal external symbols (extern TGROUP etc.) which are defined in kernel.ld. If you do not define (!) these symbols, and if you do a partial link (-Wl,-r), then I expect that the references to the undefined TGROUP etc. will appear in the output .elf as good old R_386_16 relocations.

(And R_386_16 is well-supported by mainline nasm and binutils.)

This is probably not ideal, but it should be workable.

Thank you!

[stsp]

I tried partial linking: it produces hundreds of relocations besides TGROUP. Not sure why that is the case. Also it seems to change the ELF type from executable to relocatable.

[stsp]

Basically I think partial linking doesn't resolve any relocations at all. Which is then not what we need.

[stsp]

until nasm really gets segelf support

What are the pre-requisites for segrelative? I remember that HPA wanted something from ia16-binutils for a full thing, but just for segrelative I don't suppose anything special is needed? Or is it exactly that nasm can only emit R_386_SEG16 and it needs binutils to "translate" that to R_386_SEGRELATIVE? Why it can't just emit R_386_SEGRELATIVE?

[tkchia]

Hello @stsp,

I tried partial linking: it produces hundreds of relocations besides TGROUP. Not sure why that is the case.

Oops. It seems -r is much more conservative than I thought.

I remember that HPA wanted something from ia16-binutils for a full thing, but just for segrelative I don't suppose anything special is needed?

I guess you will need to clarify with Mr. Anvin about that --- as he is the maintainer of nasm.

Thank you!

[stsp]

But the idea is not too bad. It seems I can arrange the link script for solib, which does essentially what you proposed.

[tkchia]

Hello @stsp,

It seems I can arrange the link script for solib, which does essentially what you proposed.

Thank you --- this is interesting. Does it actually produce relocations in the output file? What option(s) did you use? I think I also tried -shared instead of -r, but for some reason I do not see any relocations for TGROUP etc.

Thank you!

[stsp]

No relocations - just an undefined symbol. I am not sure why it is so but I think it doesn't matter for my needs. Perhaps undefined symbol and the COPY relocation is the same thing? I really don't know the difference.

[tkchia]

Hello @stsp,

It is probably best to check that the output fdppkrnl.elf file does actually have the needed relocations --- readelf -r or readelf -d should give something.

At least on my setup, it seems that the linker is just silently dropping the undefined symbols (and effectively giving them a value of 0), i.e. it is not working.

Also, I think it is best that you clarify with Mr. Anvin on what feature(s) you would like to see in nasm, since, again, he is the maintainer after all. I cannot speak for him.

Thank you!

[tkchia]

Hello @stsp,

Also, R_386_COPY is described here. It really means something very different from a good old R_386_16.

Thank you!

[stsp]

It is probably best to check that the output fdppkrnl.elf file does actually have the needed relocations --- readelf -r or readelf -d should give something.

That unfortunately doesn't give anything, I already checked.

At least on my setup, it seems that the linker is just silently dropping the undefined symbols (and effectively giving them a value of 0), i.e. it is not working.

That's because (in case you tried on fdpp) the dynamic sections are dropped by the script. I now have the linker script suitable for solib, but its not yet pushed. I will remove DISCARD completely, but the special care needs to be taken for some sections to go before PSP (which is what linker does by default, so some adjustments were needed).

Also, I think it is best that you clarify with Mr. Anvin on what feature(s) you would like to see in nasm

He didn't reply to the forum. Of course it would be good to get segrelative working, but I think I can get the very similar results with solib.

[stsp]

OK, I pushed what I currently have, into a separate branch. You can see that solib is created properly, but writing a dynamic loader is a big task. :(

[stsp]

Does anyone have a simple dlopen() examples to steal the code from? :) I am not going to write a dynamic linker...

[stsp]

My current theory of what's going on, is this:

• No matter whether we link exec or solib, we will not see the relocations there. That's simply because R_386_16 (among others) is a link-time reloc, not a run-time one. So the linker will resolve them all, no matter what. In case of solib, it just produces the list of undefined symbols, and in case of an exec that would result into an error.
• If nasm emits some run-time reloc like R_386_SEGRELATIVE, then we will see it in exec elf. But not in solib: for solib we will simply get the "text relocation found, please recompile with -fPIC" error because solib would probably only support the GOT relocs.

So if we now go for solib and write a dynamic linker (who will?), then switching to segrelative will be difficult when it is there, as we will have to abandon solib and dynamic linker all together.

[stsp]

I committed the linker script arrangements alone so that you can play around solib now and it will work: you can boot dosemu from it, or you can make the DGROUP symbols undefined. But not both...

[stsp]

And also I think the solib approach is actually broken. IIRC normally text relocations to undefined symbols are bound to local bss. Later you have a COPY reloc to move the global symbol from solib to bss, but the text itself is never "patched" when undefined symbols are resolved. So I suspect there is a bug in GNU ld: it silently resolves the text relocs to undefined symbols, writing zero there. But lld says this:

ld.lld: error: can't create dynamic relocation R_386_16 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output

And if you pass -z notext then it resolves no relocations at all, like ld -r does.

So... it seems the ability to create solib that way is just a GNU ld bug.

[stsp]

I think its a bug, because of 2 things:

• It doesn't even warn about an undefined global symbols in text segment.
• It doesn't seem to propagate the symbol size to symtab: I always see 0 for undefined symbols. Maybe it can't because we need not the size of the symbol but rather the size of the reference to symbol (which is 16 bits when the reloc was R_386_16). So maybe the size is stored somewhere else, I just need to find it in another section...

The correct lld error is this:

ld.lld: error: can't create dynamic relocation R_386_16 against symbol: IGROUP in readonly segment;
recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output

GNU ld doesn't emit one, but if it stores the original reloc size somewhere, then at least this can work.

[stsp]

And if I link the x86_64 solib, then I (correctly) get this for all relocs smaller than the native word size:

ld: kernel.o: relocation R_X86_64_16 against `INIT_TEXT' can not be used when making a shared object;
recompile with -fPIC

So I really think it can't store the reloc size anywhere. And what it does on i386 arch is unclear.

[stsp]

https://reviews.llvm.org/D63121 This seems to confirm my theory that you can't leave the "small" relocations to run-time in a form of undefined symbols. At least not on x86_64. Why GNU ld allows that for i386, is something to find out later.

[stsp]

fdpp: we'll find work-around everywhere. :) Instead of implementing the dynamic linker (that would depend on GNU ld misbehaviour and all that), I did a simple trick: I link the kernel twice with different segments, then compare the resulting binaries and create a "relocation table" of some sort. A trivial hack saves the world again. :)

[stsp]

Hi @hpax how difficult would it be to support R_386_SEGRELATIVE in nasm? Currently I have to create my own relocation table by linking the object twice and diffing the resulting binaries. Which is not the best hack in the world.

[stsp]

Hi @hpax So I have downloaded your segelf binutils fork and your WIP nasm elf16 branch. It all works! I have ported my code to your elf16 branch without a single issue! The patches are referenced in this ticket. So... what's the deal? Why not to upstream the code, if it all already works?

Note that, contrary to what is advertised, the ia16-elf-ld just segfaults on these new relocs. While your code works perfectly, even if you never said it should. :)

[andrewbird]

Maybe let @tkchia know?

[stsp]

It doesn't matter since I am not going to use ia16-elf-ld, and also nasm is the most important piece of the puzzle anyway.

If nothing else, I can make nasm and binutils forks a submodules of fdpp... Though it will take quite a while to build it then. :) And overall that would be a maintenance nightmare.

[stsp]

I am not going to use ia16-elf-ld

... nor ld. We need this in llvm's lld, sigh. Another 10 years to wait. :(

[stsp]

I opened this binutils ticket for that: https://sourceware.org/bugzilla/show_bug.cgi?id=30970 This is all I can do. @hpax you can comment there if you want. Would be very bad if such a perfectly working code is lost.

[hpax]

If the binutils guys accept the code I'll definitely put it in NASM; I don't want to push it out into a release version unilaterally, since if the relocation type numbers start conflicting it will be a disaster.

[stsp]

But never announcing your projects is not the best strategy getting them upstream. :) If I knew you have a fully working code, I'd do something about it 5 years ago.

[stsp]

Hi @hpax I can't get the -static -no-pie linking to work with your code. If I do that, I get the elf binary w/o any relocations, and it of course doesn't work. But if I try this example:

;
; assemble and link with:
; nasm -f elf printf-test.asm && gcc -m32 -static -no-pie -o printf-test printf-test.o
;
section .text
global main
extern printf
;global _start
;_start:
main:

  mov eax, 0xDEADBEEF
  push eax
  push message
  call printf
  add esp, 8
  ret

message db "Register = %08X", 10, 0

%ifidn __OUTPUT_FORMAT__,elf
section .note.GNU-stack noalloc noexec nowrite progbits
%endif

Then, even with -static -no-pie it still contains these relocs:

Relocation section '.rel.plt' at offset 0x178 contains 13 entries:
 Offset     Info    Type            Sym.Value  Sym. Name
080e7030  0000002a R_386_IRELATIVE  
080e702c  0000002a R_386_IRELATIVE  
080e7028  0000002a R_386_IRELATIVE  
080e7024  0000002a R_386_IRELATIVE  
080e7020  0000002a R_386_IRELATIVE  
080e701c  0000002a R_386_IRELATIVE  
080e7018  0000002a R_386_IRELATIVE  
080e7014  0000002a R_386_IRELATIVE  
080e7010  0000002a R_386_IRELATIVE  
080e700c  0000002a R_386_IRELATIVE  
080e7008  0000002a R_386_IRELATIVE  
080e7004  0000002a R_386_IRELATIVE  
080e7000  0000002a R_386_IRELATIVE

I'd like to avoid the use of -pie because lld doesn't accept the 16bit relocs in pie mode, and llvm devs say its intentional: https://github.com/llvm/llvm-project/issues/68874 So I am a bit lost. I think you need to add the R_386_SEGRELATIVE relocs regardless of a -pie mode, but I am not sure. Can you please help?

[stsp]

Maybe assembler should better put R_386_SEGRELATIVE relocs, and linker doesn't have to do the guess-work? For example llvm devs seem to find it unjustified that R_386_32 does transform to R_386_RELATIVE by -pie, but R_386_16 - not. And I can't find it ever properly specced. Perhaps to avoid the same confusion in segelf, it would be better to allow assembler to deal with SEGRELATIVE relocs explicitly?

[stsp]

OK it seems like unbinding R_386_SEGRELATIVE from -pie, requires a bit of work. I'll try to come up with some solution. Hope I am not dragging myself into writing yet another ia16-elf-ld...

[stsp]

I set up this binutils fork: https://github.com/stsp/binutils-gdb/tree/seg16 Most of HPA's work is rebased to master there. Though I had to skip the gas commits as they conflict too much, and I had to change some ld commits to the point that even the fdpp patches had to be adjusted (due to disallowed relocations against absolute symbols in newer binutils). But now it works again.

@hpax It would really help if you help rebasing your gas patches to master. I myself am not interested in a gas support, but I realize its needed for upstreaming.

[stsp]

Affected by this bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30974

[stsp]

OK, I've now ported the gas, too, heavily modifying the original patches. Was able to build the example, not sure what it does though. Also I made it possible to link w/o -pie, which is a huge relief.

@hpax maybe you can at last check if my gas port of your work is still good? Its all here: https://github.com/stsp/binutils-gdb/tree/seg16

[hpax]

On October 16, 2023 10:38:35 AM PDT, Stas Sergeev @.***> wrote:

OK, I've now ported the gas, too, heavily modifying the original patches. Was able to build the example, not sure what it does though. Also I made it possible to link w/o -pie, which is a huge relief.

@hpax maybe you can at last check if my gas port of your work is still good? Its all here: https://github.com/stsp/binutils-gdb/tree/seg16

It looks reasonable to me. The comment about R_386_16 is valid but has an answer: 16-bit code is (normally) not relocated on an intrasegment basis, so R_386_RELATIVE16 is superfluous. A PIC/PIE executable in 16-bit mode is relocated via the segment bases, hence R_386_SEGRELATIVE.

So although it would seem inconsistent on the surface for someone who is not familiar with x86 real-mode programming, it does, in fact, make sense. You may wish to give that feedback to the lld maintainers.

[hpax]

On October 15, 2023 3:41:28 AM PDT, Stas Sergeev @.***> wrote:

Maybe assembler should better put R_386_SEGRELATIVE relocs, and linker doesn't have to do the guess-work? For example llvm devs seem to find it unjustified that R_386_32 does transform to R_386_RELATIVE by -pie, but R_386_16 - not. And I can't find it ever properly specced. Perhaps to avoid the same confusion in segelf, it would be better to allow assembler to deal with SEGRELATIVE relocs explicitly?

The linker should probably not generate RELATIVE relations for relocations that have a (non-absolute) SUB relocation attached to it. That would be the Right Thing™, and given that, an R_386_RELATIVE16 would, in fact, fall out naturally (R_386_16 without an attached R_386_SUB16), although I do not know of anyone who has any immediate use for it... unless someone is actually considering tiny-model dynamic linking...

[stsp]

It looks reasonable to me.

But someone needs to verify my gas porting. It is modeled after your code, but is very different in details.

So although it would seem inconsistent on the surface for someone who is not familiar with x86 real-mode programming, it does, in fact, make sense. You may wish to give that feedback to the lld maintainers.

No, I like their approach. Also arguing with them to allow R_386_16 in -pie mode is counter-productive. Which is why I created this patch: https://github.com/stsp/binutils-gdb/commit/373af8514d6c06f6e043077b3f00fe064138b54d The problem is solved, you can now control the creation of R_386_RELATIVE from the linker script, rather than with -pie flag! Here's how I do that: https://github.com/dosemu2/fdpp/commit/599fcc129a257af7cbbc883301677c3dcf7579cb#diff-493ac0f2fd01cf0858316ec1ff7eedac0cf71909ff7ea504b42fc0737abae5cfR9 _DGROUP = DEFINED(loadaddr) ? loadaddr + ABSOLUTE(.) : .; So if someone defined loadaddr with --defsym, then I create absolute symbol. Otherwise I create R_386_SEGRELATIVE. We don't need -pie for this, that's the main point.

although I do not know of anyone who has any immediate use for it... unless someone is actually considering tiny-model dynamic linking...

Why not? That exactly is the point of llvm guys, and its a valid point. So I just got rid of -pie. -pie has nothing to do with segmentation, really. I can relocate tiny-pie within the 64K segment, or I can put non-pie into the beginning of the 64K segment, and both cases are valid. But we don't need tiny-pie within 64K, so we just disallow -pie rather than to add RELATIVE16. I'd like you to evaluate this idea, because I think its a good one.

The linker should probably not generate RELATIVE relations for relocations that have a (non-absolute) SUB relocation attached to it.

Good point! I think we need another relocation then. How about R_386_SEG16WRT which is similar to R_386_SEG16 but doesn't add SEGRELATIVE? Do you really need the R_386_SUB32 that you specced? I don't have a use for that one, so maybe repurpose it into R_386_SEG16WRT?

[hpax]

On October 16, 2023 10:33:09 PM PDT, Stas Sergeev @.***> wrote:

It looks reasonable to me.

But someone needs to verify my gas porting. It is modeled after your code, but is very different in details.

So although it would seem inconsistent on the surface for someone who is not familiar with x86 real-mode programming, it does, in fact, make sense. You may wish to give that feedback to the lld maintainers.

No, I like their approach. Also arguing with them to allow R_386_16 in -pie mode is counter-productive. Which is why I created this patch: https://github.com/stsp/binutils-gdb/commit/373af8514d6c06f6e043077b3f00fe064138b54d The problem is solved, you can now control the creation of R_386_RELATIVE from the linker script, rather that with -pie flag! Here's how I do that: https://github.com/dosemu2/fdpp/commit/599fcc129a257af7cbbc883301677c3dcf7579cb#diff-493ac0f2fd01cf0858316ec1ff7eedac0cf71909ff7ea504b42fc0737abae5cfR9 _DGROUP = DEFINED(loadaddr) ? loadaddr + ABSOLUTE(.) : .; So if someone defined loadaddr with --defsym, then I create absolute symbol. Otherwise I create R_386_SEGRELATIVE. We don't need -pie for this, that's the main point.

although I do not know of anyone who has any immediate use for it... unless someone is actually considering tiny-model dynamic linking...

Why not? That exactly is the point of llvm guys, and its a valid point. So I just got rid of -pie. -pie has nothing to do with segmentation, really. I can relocate tiny-pie within the 64K segment, or I can put non-pie into the beginning of the 64K segment, and both cases are valid. But we don't need tiny-pie within 64K, so we just disallow -pie rather than to add RELATIVE16. I'd like you to evaluate this idea, because I think its a good one.

The linker should probably not generate RELATIVE relations for relocations that have a (non-absolute) SUB relocation attached to it.

Good point! I think we need another relocation then. How about R_386_SEG16WRT which is similar to R_386_SEG16 but doesn't add SEGRELATIVE? Do you really need the R_386_SUB32 that you specced? I don't have a use for that one, so maybe repurpose it into R_386_SEG16WRT?

Your SEG16WRT sounds exactly like SEGRELATIVE to me, unless you have specific other semantics in mind.

As far as SUB32: people can, and do, use 32-bit addressing in 16-bit mode. Otherwise it would be a nightmare for the tools to deal with stuff like:

 mov ax,[edx+foo] ; 32-bit displacement

... especially since the tool has no way of knowing if you are in real mode, big real mode, or 16-bit protected mode with extended segments.

So that is in my opinion an absolute requirement.

So I feel that the best option is:

1. Add RELATIVE16 for completeness;
2. Key generating RELATIVE16 and RELATIVE(32) on whether or not there is a SUB relative relocation attached (a subtraction between symbols is by definition not relative; this is probably handled elsewhere in ld as a lot of other platforms have SUB relocations, too.)

For mixed-mode programming PIC/PIE could be quite useful; e.g. the Linux kernel low memory stub ends up (re-) doing exactly that work using a postprocessor. I feel like "not my use case" is the wrong answer.

[stsp]

Your SEG16WRT sounds exactly like SEGRELATIVE to me, unless you have specific other semantics in mind.

No, I meant SEG16WRT does never add SEGRELATIVE, while SEG16 can do that. So if the assembler sees the wrt reference, it can put SEG16WRT to make sure SEGRELATIVE won't be generated for it. Otherwise you'll need to do some messy stuff on checking if SUB reloc is attached to it or not. I'd like to avoid that checking.

1. Add RELATIVE16 for completeness;

Maybe in the future. For now I added the possibility to disable -pie (or you can still use pie with RELATIVE(32)), and that solves all the practical needs I can think of. Later RELATIVE16 can be added, but its an entirely separate task IMHO. It even looks strange: you were OK without RELATIVE16 for all these years, and when I've found the perfectly working solution (to just disable -pie), you suddenly need RELATIVE16 right now. :) Lets do things step-by-step.

2. Key generating RELATIVE16 and RELATIVE(32) on whether or not there is a SUB relative relocation attached

You probably mean SEGRELATIVE? Otherwise I don't understand.

For mixed-mode programming PIC/PIE could be quite useful;

You mean tiny-pie with RELATIVE16 relocs? I don't know why do we need it right now. And RELATIVE(32) relocs haven't "disappeared" in my patch, no. You can still use -pie if you want, I just don't recommend doing so.

[stsp]

It won't hurt if you add RELATIVE16 to your spec. But I want to concentrate on a practical problem, which is SUB vs RELATIVE confusion. So my proposal is:

• Add R_386_SEG16WRT, which should always be accompanied by the SUB reloc.
• Make it so that R_386_SEG16 is never accompanied with SUB reloc, but may be translated to SEGRELATIVE.

What do you think?

[stsp]

Essentially R_386_SEG16 and R_386_SEG16WRT are similar in their action. But the usage is different: R_386_SEG16 is a direct seg reference, while R_386_SEG16WRT is for wrt references that need SUB.

[hpax]

On October 17, 2023 3:09:22 PM PDT, Stas Sergeev @.***> wrote:

It won't hurt if you add RELATIVE16 to your spec. But I want to concentrate on a practical problem, which is SUB vs RELATIVE confusion. So my proposal is:

• Add R_386_SEG16WRT, which should always be accompanied by the SUB reloc.
• Make it so that R_386_SEG16 is never accompanied with SUB reloc, but may be translated to SEGRELATIVE. What do you think?

SEG16 should never be paired with SUB; SUB is for R_386_16/32. The RELATIVE relocations of all flavors are markers for locations that need to be updated at load time as opposed to link time.

[stsp]

Ah, good to know! I thought the following syntax is possible: mov bx, seg symbol wrt some_base Obviously I am not using wrt myself to not know such syntax is impossible. :)

[stsp]

But in this case I think something isn't working as expected. When I do mov ax, seg _DGROUP, then I get mov ax, _DGROUP>>4, but, according to your spec, it should be mov ax, _DGROUP!>>4? But I never defined _DGROUP!.

[stsp]

Can it be that your current nasm impl doesn't add ! when applying seg, and I took it as a feature and got the entire picture wrongly?