Linux from scratch patches

[Arch91]

A patch for building ncurses-5.9

[frno7]

Hmm. I believe it’s probably best to host user-space programs and libraries in a separate repository. @Arch91, if you create a repo for these things we could link to it from the wiki.

This repo is mainly for the Linux kernel, and we should probably move the INITRAMFS things to a separate repo too.

[Arch91]

Understood. Answered em@ily, need some assistance...

[Arch91]

Done. Please, deny those pull requests... somehow...

[frno7]

Thanks, @Arch91! You should be able to close the pull requests yourself. Don’t you have a button that says “Close with comment” or suchlike? Also, what about starting up a wiki on your new repo PS2Linux_v5.X_assistance? We can cross-link between them, but the main part of building programs for PlayStation 2 Linux seems to be better off in your new repo, at least when building from sources. I’m thinking maybe we’ll add a page for building programs with Gentoo too, and perhaps other distributions.

Finally, I’m considering starting-up the discussions section. I haven’t studied it yet but it seems promising, because having longer discussions in the issues section is sometimes a bit awkward, off-topic, etc.

[Arch91]

@frno7, if you ask me, I would leave your wiki as it is, I don't see why it is necessary to restructure it/remove some part from it just because the files noted in them are moved from one repository to another. That's still your project) And a big one I'll say. The links in building initramfs, glibc part of cross-compiler from sources and in that "building programs for PS2Linux" from sources already edited to link to the repository of mine. What about those pull requests - yes, I closed them (except this one yet), but according to github info, to remove them completely, it's up to you as a repository keeper to go along them and remove them (as they are closed now) one by one. Yes, discussions section is a nice idea. Now I can flood there :D To let the people find it better, maybe note about it's existion in your wiki. About the section of building programs with Gentoo... Well, to describe it, if you are hinting that I should handle that, I'll have to go through the steps of installing that "emerged" cross-compiler... I don't like it) It's not mips>>r5900<<el, so everything of it was compiled not for a mipsr5900el target. That will be very important when (if I'll live to that moment) the hard floating point will be finally supported and supported well so the kernel will not crashes. And the base of it is mostly a busybox. That means that "mount/umount -a" will not read the /etc/fstab file, that ethernet abilities are some way cut off, and else... Am I right?

[frno7]

Many thanks for your contributions, @Arch91! In my opinion it’s practical to have (somewhat) separate repos and wikis for kernel space and user space, especially if user space grows significantly larger than a minimal INITRAMFS having only Busybox. Likewise the iopmod repo is for the IOP, and the qemu repo is for the emulation of user space, and so on. But let’s not rush things. We can think about it for a while.

You don’t need to do anything about Gentoo yourself. I’m using it, though, which is why I thought I might add some pages about it, eventually, as an alternative to source builds. So people may choose tools and user space by their preference. Hmm, I don’t follow exactly what you mean with “mips>>r5900<<el”, but it’s indeed quite possible to compile Gentoo specifically for the R5900. Can you elaborate? Again, you don’t have to do anything about Gentoo if you don’t want to. :smile:

[Arch91]

mipsr5900el. I mean when we are building the cross compiler from the sources, one of the configure flags for building binutils and gcc are --target=mipsr5900el-unknown-linux-gnu . mips+r5900+el, every character here will trigger something in the result build binutils/gcc - it will set which ISA will be as an output - mipsr5900el means MIPS-III; it will set which ABI will be the output - mipsr5900el means O32 as a standard mipsel target (mips64r5900el means N32 while standard mips64el may mean O64); it will set which opcodes (assembler... terminology) are supported or are not; and also it will or will not set the default FP - hard, double, soft. (never single) Suppose Gentoo's compiler/cross-compiler was built for mipsel-unknown-linux-gnu target. The output binaries and libs of the Gentoo'es compiler/cross-compiler means the entire operating system of Gentoo can be of the next properties - MIPS-I O32 hard float. Will the kernel handle the system of hard-float? Will the kernel handle the MIPS-I and MIPS-II ISAs? Because I beleive that the only combo which the current 5.X PS2Linux kernel can handle is MIPS-III (excluding unsupported for the r5900 opcodes) O32 soft-float. When the kernel meets the unsupported instruction - it panic. When the kernel meets the system of hard-float... I think the FPU emulator activates... and everything becomes sooo sloooow...

[frno7]

@Arch91, the three most relevant wiki articles covering this topic are

• the R5900 processor specific ABI (psABI),
• the R5900 short loop erratum,
• and the R5900 floating point unit (FPU).

As noted in the psABI wiki article, PlayStation 2 Linux currently accepts the o32 ABI for MIPS I and II. The n32 ABI for MIPS III could, eventually, be supported as well. All instructions not supported properly by R5900 hardware are emulated by the Linux kernel. Floating-point instructions are notable, but also for example the RDHWR instruction, which by accident happens to overlap with the R5900-specific SQ instruction, triggering an address exception which the kernel traps to do RDHWR; review commit MIPS: R5900: Trap the RDHWR instruction as an SQ address exception for details.

The R5900, like many other processors, has several hardware bugs. The most grave one, explained in the mentioned short loop wiki article, causes undefined behaviour, bugs. The GAS and GCC option -mfix-r5900 must be given to fix the instruction scheduling, which remains compatible with MIPS I, II and III. Other processors have similar flags for various hazards, as they call them, so this type of problem is not unique for the R5900. In short, with careful compilation, one can make a generic Linux distribution that runs on all MIPS I, II and III hardware, including the PlayStation 2. Wouldn’t that be nice? :smile:

To keep everyone happy, it’s very important that PlayStation 2 Linux doesn’t misbehave in any way. This is why, for example, floating-point instructions are emulated by the kernel: the R5900 hardware simply doesn’t work as expected of MIPS I, II and III. There would be subtle and not-so-subtle bugs otherwise. Review issue #3 for some ideas on how the floating-point hardware could be enabled as a properly supported and official ELF extension.

Will the kernel handle the system of hard-float?

Yes.

Will the kernel handle the MIPS-I and MIPS-II ISAs?

Yes, with the GAS and GCC option -mfix-r5900 to fix-up instruction scheduling for the R5900; other MIPS processor won’t care about it and they will work too.

Because I beleive that the only combo which the current 5.X PS2Linux kernel can handle is MIPS-III (excluding unsupported for the r5900 opcodes) O32 soft-float. When the kernel meets the unsupported instruction - it panic.

Linux 5.x doesn’t accept the n32 ABI for MIPS III yet. It can be done, with some effort.

When the kernel meets the system of hard-float... I think the FPU emulator activates... and everything becomes sooo sloooow...

Yes, but what’s the alternative? Enabling R5900 floating-point hardware fast and loose causes bugs and undefined behaviour, as noted in the mentioned wiki article. People will get mad about it. Again, have a look at issue #3 for a starting-point for further discussions.

[Arch91]

@frno7, many, many thanks for that deep describition of the situation with FP. After reading all of this, the thought appeared in my head... that FP will never be supported...

-mfix-r5900

is this flag set by default for the target mipsr5900el-unknown-linux- ? I could see it in gcc's specs though... According to the short-loop erratum article, that flag is set by default for mipsr5900el-unknown-linux- That flag is good only for a goal to get the system of hard-float; currently this flag fixes some issueses but not the all of them, so using this flag or using this flag not, the FPU emulator steel needs to be activated in linux. Do I understood that moment right?

NaN (not a number) and plus/minus infinities are not supported; exception mechanisms are not fully supported; denormalized numbers are not supported; rounding towards nearest and plus/minus infinities are not supported; computed results usually differs in the least significant bit; saturating instructions can differ more than the least significant bit.

may I ask you to tell some details about that? - I always wanted to know it, - where are the exact places of the all of that issueses? In binutils/gcc/kernel? glibc?.. Are they a functions to fix? Are they an opcodes to fix? From the other side, I think this question is stupid, cause I'm looking from the noob side... Here as I determine it - I determine them as a functions of gcc. Each c-code can be presented... in c-code :D but also it can be compiled to, I'll call it, a middle-compiled file with .s extension (which can be edited and further compiled so we'll get the "usual" .o compiled file), where each of the function was written in C will be presented in the assembler code. Here in such .s file we might can search for the unsupported opcodes and might for the wrong logic situations like RDHWR. As simple I determine it in my imagination, as simple I answered to myself how it can be fixed - to rewrite the functions with assembler inlines (asm volatile ....) in C-code or directly rewrite the assembler code in .s file representations. If the kernel somehow emulates the unsupported ones then there IS the way the system somehow handles them. What you'll say? How far I am from the truth in my imaginations?))

But let's return to the MIPSel based operating systems launching on the real PS2 discussion. Am I correct in that I made a conclusion from your post that it is actually do not matters whenever ISA that system is of (MIPS-I/II/III), whenever FP that system is of (hard/soft but not double) - that kind of MIPSel based operating system will be working with your current kernel with FPU emulator? And so that's why we can use the Gentoo'es cross-compiler built for the target=mipsel-unknown-linux-* AND Gentoo as an OS itself, installing it's already compiled packages?

[frno7]

@frno7, many, many thanks for that deep describition of the situation with FP.

You’re welcome!

After reading all of this, the thought appeared in my head... that FP will never be supported...

Issue #3 is our best chance at it, I’d say.

-mfix-r5900

is this flag set by default for the target mipsr5900el-unknown-linux- ? I could see it in gcc's specs though... According to the short-loop erratum article, that flag is set by default for mipsr5900el-unknown-linux-

Yes, -mfix-r5900 is implied for mipsr5900el targets. The -mfix-r5900 option must be given on the command line if one compiles o32 ABI MIPS I and II for other targets, that one would like to be compatible with the R5900 as well. -mfix-r5900 can be safely used with all MIPS I, II and III hardware.

That flag is good only for a goal to get the system of hard-float; currently this flag fixes some issueses but not the all of them, so using this flag or using this flag not, the FPU emulator steel needs to be activated in linux. Do I understood that moment right?

-mfix-r5900 only fixes the R5900 short loop erratum, and is unrelated to the floating-point unit instructions. So it can be used to compile R5900 compatible programs that have FPU instructions in them, which on the R5900 will be trapped and emulated by the kernel. Other MIPS processors may of course run these FPU instructions fast and easy in hardware, but the R5900 will have them emulated, along with everything else that the R5900 needs to emulate for MIPS I, II and III.

• NaN (not a number) and plus/minus infinities are not supported;
• exception mechanisms are not fully supported;
• denormalized numbers are not supported;
• rounding towards nearest and plus/minus infinities are not supported;
• computed results usually differs in the least significant bit;
• saturating instructions can differ more than the least significant bit.

may I ask you to tell some details about that? - I always wanted to know it, - where are the exact places of the all of that issueses? In binutils/gcc/kernel? glibc?.. Are they a functions to fix? Are they an opcodes to fix? From the other side, I think this question is stupid, cause I'm looking from the noob side...

The R5900 FPU omissions are in the R5900 hardware, which cannot be ‘fixed’ in itself because we can’t modify it. But we can do effective workarounds in software, as in #3. This isn’t particularly easy, though: tooling, library and kernel updates are needed to make the best of it, and we’d like to avoid misunderstandings causing nasty bugs and surprises with preexisting MIPS I, II and III software. We really want the tooling and kernel to be safe and easy to use.

Presumably the R5900 FPU was ‘simplified’ to be ‘good enough’ for games and multimedia. Numerical accuracy, for example, is often not a great concern for such applications. This simplification meant fewer processor transistors had to be used for the FPU. The transistors could instead be used for other features of the R5900, for example the 128-bit multimedia instructions (issue #5), the VPU vector co-processors (issue #14), and so on. The engineers at the time surely had to make difficult choices, and take shortcuts, facing the limited transistor budget of the R5900.

One can note that the Linux kernel is happy to run a mix of MIPS I, II and III software, simultanenously, including the possibility of running hardware R5900 FPU extensions for some processes. This means one could make a base distribution for MIPS II, for example, that may be a bit sluggish due to having its programs run under FPU software emulation, but one may still compile and run for example a particular game or other application making complete use of R5900 hardware extensions and run it at maximum speed in the same system.

Am I correct in that I made a conclusion from your post that it is actually do not matters whenever ISA that system is of (MIPS-I/II/III), whenever FP that system is of (hard/soft but not double) - that kind of MIPSel based operating system will be working with your current kernel with FPU emulator?

Yes, except we only have o32 ABI MIPS I and II at the moment, including double-precision FPU emulation for MIPS II. MIPS III with n32 may come later.

And so that's why we can use the Gentoo'es cross-compiler built for the target=mipsel-unknown-linux-* AND Gentoo as an OS itself, installing it's already compiled packages?

Yes, PlayStation 2 Linux can run Gentoo compiled for o32 ABI MIPS II (having -mfix-r5900 as previously noted), but Gentoo can also easily be compiled specifically for the R5900, with a proper mipsr5900el target from the ground up. That’s what I have.

[Arch91]

The R5900 FPU omissions are in the R5900 hardware, which cannot be ‘fixed’ in itself because we can’t modify it. But we can do effective workarounds in software, as in #3.

Yes, if to pay attention to the litteral terminology, saying "fix" I mean the software workarounds.

This isn’t particularly easy, though: tooling, library and kernel updates are needed to make the best of it, and we’d like to avoid misunderstandings causing nasty bugs and surprises

So... that means it is precisely unknown for sure where are the certain places of the functions and etc which are responsible for that or another FPU issue, right?..

one may still compile and run for example a particular game or other application making complete use of R5900 hardware extensions and run it at maximum speed in the same system.

That game/app can not be based on the newest original glibc, I think that by saying that you mean compiling with PS2SDK. If you are not, then, as I think, even if there will be some example app which somehow consists from a specific r5900 hardware extensions only, if to compile it using the current cross-compiler built for target=mipsr5900el-unknown-linux-* (we manually setting that flag --with-float=soft while building the cross-compiler; I tried --with-float=hard and got the problem of compiling glibc, met the unsupported upcode), the compiled program will be of soft-float which, again, triggers the kernel's FPU emulator. Eventually, the speed is not be maximum. The FPU emulator do not work like it working only when the system meets the unsupported opcode to proccess only THAT opcode - if there was a trigger to activate the FPU emulator, it will softwarely proccess ALL of the opcodes indiscriminately whenever they are supported by hardware or they are not.

PlayStation 2 Linux can run Gentoo compiled for o32 ABI MIPS II (having -mfix-r5900 as previously noted)

What will happens if to try to run the existing Gentoo compiled without -mfix-r5900 ? - the system crashes sooner or later? I still don't get it - can we use the existing/ready/already compiled O32 MIPS-II Gentoo OR we have to build everything of it from the scratch.

[frno7]

So... that means it is precisely unknown for sure where are the certain places of the functions and etc which are responsible for that or another FPU issue, right?..

Well, the kernel, libraries and tools are certainly responsible for maintaining o32 ABI and MIPS I, II and III compatibility, which in the case of the R5900 FPU means that the kernel emulates it, as previously explained. Otherwise R5900 Linux would be broken.

Issue #3 is about defining a new, officially supported, ELF ABI annotation, specific to the R5900 where we can define other rules that won’t break the preexisting o32 ABI for MIPS I, II and III.

That game/app can not be based on the newest original glibc,

As mentioned, the kernel, tools, and libraries most likely including GNU libc would need to be updated for issue #3. I told you, it won’t be easy. :-) Regardless of FPU emulation, you may try Musl, as an alternative, mentioned in https://github.com/frno7/linux/issues/33#issuecomment-750902872. Musl is expressly designed to allow programs to be statically linked, which is often useful in this particular case, since one might not want to link with system libc. Musl is also much smaller.

I think that by saying that you mean compiling with PS2SDK.

No, PS2SDK doesn’t work with Linux. It’s designed to run without an enclosing operating system. “On the metal”, as they say.

I tried --with-float=hard and got the problem of compiling glibc, met the unsupported upcode),

Yes, GAS refuses instructions that it knows aren’t compatible with the target, o32 ABI MIPS II. That’s because the R5900 doesn’t have a FPU that’s good enough for o32 ABI MIPS II.

the compiled program will be of soft-float which, again, triggers the kernel's FPU emulator.

Actually, no. “Soft float” means that floating-point operations are emulated by GCC, so the programs don’t make use of any FPU instructions. “Hard float” on the other hand means that there are FPU instructions, and with an R5900 Linux kernel, those would be trapped and emulated by the kernel. Currently there’s no way to compile a program and avoid FPU emulation: either GCC emulates them (‘soft float’), or the kernel (‘hard float’). By the way, ‘soft’ is slightly faster than ‘hard’, because there’s more overhead for kernel emulation with less opportunity for compiler optimisations.

The FPU emulator do not work like it working only when the system meets the unsupported opcode to proccess only THAT opcode - if there was a trigger to activate the FPU emulator, it will softwarely proccess ALL of the opcodes indiscriminately whenever they are supported by hardware or they are not.

Yes, all FPU instructions are emulated by the kernel, as if the processor didn’t have any FPU at all. That’s because the R5900 Linux kernel turns off the FPU in hardware.

What will happens if to try to run the existing Gentoo compiled without -mfix-r5900 ? - the system crashes sooner or later?

It may crash, or it might corrupt your data, or cause any other imaginable or unimaginable disaster. Ooh, the horror!

I still don't get it - can we use the existing/ready/already compiled O32 MIPS-II Gentoo OR we have to build everything of it from the scratch.

I don’t think the o32 MIPS II packages for Gentoo were compiled with -mfix-r5900, sadly. If they were, I’d expect them to work fine. Since it’s relatively easy to build R5900 Gentoo oneself, I haven’t asked the Gentoo MIPS maintainers to update their compilation flags with -mfix-r5900. Would you be interested in trying it? :-)

[Arch91]

I still don't get it - can we use the existing/ready/already compiled O32 MIPS-II Gentoo OR we have to build everything of it from the scratch. I don’t think the o32 MIPS II packages for Gentoo were compiled with -mfix-r5900, sadly. If they were, I’d expect them to work fine. Since it’s relatively easy to build R5900 Gentoo oneself, I haven’t asked the Gentoo MIPS maintainers to update their compilation flags with -mfix-r5900

Looks like I am beginning to blown up your mind with my silly understanding, so you used a little sarcasm :D So the answer was simple - wanna Gentoo? - don't use the existed one, compile it from the scratch by your own hands! And forget about 'emerge' package downloader.

Currently there’s no way to compile a program and avoid FPU emulation

In time of testing the Juergen Urban's 2.6.35.4 kernel, I was able to build gcc-svn-20130804 (still keep the source) with flags --target=mipsr5900el-unknown-linux-uclibc --with-float=hard . To force to not trigger the kernel's FPU emulator, there was a definition cpu_has_fpu 1 in the file arch/mips/include/asm/mach-ps2/cpu-feature-overrides.h

That was the fastest combo I ever saw. The system of hard float based on uClibc, I tested the next - X'es, SDL, some usual linux apps such like 'file', nano, the apps for ethernet inteface - I am listing these to say that using that version of gcc and so I never saw a message about that the FPU emulator was triggered on (that warning message actually exists when it is triggered). Everything I tested was working stable.

Actually, no. “Soft float” means that floating-point operations are emulated by GCC, so the programs don’t make use of any FPU instructions. “Hard float” on the other hand means that there are FPU instructions, and with an R5900 Linux kernel, those would be trapped and emulated by the kernel... ‘soft’ is slightly faster than ‘hard’, because there’s more overhead for kernel emulation with less opportunity for compiler optimisations... all FPU instructions are emulated by the kernel, as if the processor didn’t have any FPU at all.

If the FPU emulator is so fantastic great in handling everything with the numerical accuracy, then can it be that the FPU emulator is the answer for the issue #3?.. Instead of creating some special ABI (and moreover accepting it further officially) and making workarounds in countless unknown places, maybe it is needed to just cut-off FPU emulator proccessing? - why FPU emulator have to emulate ALL of the FPU instruction? - is it possible to set for it that "these instructions are excluded from your emulation proccess, let the hardware handle them. You may proccess any others which not in your known list with sets."

[frno7]

So the answer was simple - wanna Gentoo? - don't use the existed one, compile it from the scratch by your own hands! And forget about 'emerge' package downloader.

The relevant MIPS II Gentoo packages I’ve seen, under (experimental) / other experimental architectures are somewhat old, 5+ years, so you probably wouldn’t want to use them anyway. The point is that now with the -mfix-r5900 option, we finally have the tooling to make the best of it. Gentoo, Debian, and the rest of them could do very well compiling a generic MIPS II Linux distribution that would work with the PlayStation 2. In the meantime, you might want to build R5900 Gentoo yourself, build from your own sources, or wait for someone to come and deliver a distribution for you. :-) See also issue #34.

Everything I tested was working stable.

There’s no doubt some perhaps most programs will work quite nicely, but will you thoroughly test all packages in the major distributions like Gentoo and Debian? Issue #3 would let you choose which programs you’d dare to run with the FPU, and which you’d prefer to be on the safe side.

If the FPU emulator is so fantastic great in handling everything with the numerical accuracy, then can it be that the FPU emulator is the answer for the issue #3?.. Instead of creating some special ABI (and moreover accepting it further officially) and making workarounds in countless unknown places, maybe it is needed to just cut-off FPU emulator proccessing?

Well, FPU emulation is very slow. Some people will want to run at least a few programs with FPU hardware for maximum speed, and issue #3 would make it officially possible.

why FPU emulator have to emulate ALL of the FPU instruction? - is it possible to set for it that "these instructions are excluded from your emulation proccess, let the hardware handle them. You may proccess any others which not in your known list with sets."

The R5900 hardware can turn the FPU either on or off. It’s all or nothing, the hardware can’t turn on or off a selection of instructions...

[Arch91]

The relevant MIPS II Gentoo packages I’ve seen, under (experimental) / other experimental architectures are somewhat old, 5+ years, so you probably wouldn’t want to use them anyway.

Why not? The people like to watch the show) Rather than reading how to build something, where to place this or that thing... Or it might be another situation - for example, I am compiling some package but it not works like I want. I am installing the OS with that package included and configured, and study how it is configured. Since I understood, I am using that experience to configure that package right in the nowadays OS. Anyway, I understood how the precompiled MIPSel distribution will be working with the current 5.X kernel.

you might want to build R5900 Gentoo yourself, build from your own sources, or wait for someone to come and deliver a distribution for you. :-)

Well... I don't see a defference between a Gentoo built from the sources and the "unnamed" system built from the same sources. Literally >don't see a difference< - if you know that, please, share with your opinion. The only difference I can note is the way they are building a compiled package in the from it can be downloaded and installed and the tools to install that kind of packages. I still not sure in the exact name of the tool in the Gentoo (I previously called it 'emerge') which installes that packages, but in a Red Hat based Linux it's called 'yum' (currently 'dnf') to install .rpm .

See also issue #34.

If someone want a system to test, I have one which was built from the sources using the appropriate already wrote tutorial. The one little problem is that I have no place to keep it constantly for downloading from, so everyone who not consider me as "a random noob" can ask me to share with it :D Oncely. I can either make an USB image of my 16Gb USB flash stick using USBit aka USB Image Tool OR to pack into the tar archive my partition with the system (except the dev folder, I may place a devcreator.sh script there). However, I prefer to specially not doing that because of the next reason - to test the system it is good, a pleasure, of course, but what's next? That's all? One was seeing the PS2Linux thingies and satisfied? - no, I don't think so. I think the people would want further to compile something for more tests (or higher goal). To compile something, they totally end up with the cross compiler needence and the packages which eventually might be a base for their system. So building the cross-compiler from the sources from that wiki tutorial bloodybathly kills both birds - to setup a cross-compiler and to make a base system. Also, to get some experience in the compiling if to follow the tutorial by own hands (without using the .sh scripts).

will you thoroughly test all packages in the major distributions

To test everything is impossible, and the test of the packages should be not just the way "./executable", but they might do something as a result. That requires involving into the each package, into the each executable... Besides, we already have the issueses to "fix" :S And currently playing with the tests of the soft-float and hard-float compiled binaries will one way or another result to the system freezes, kernel panics or SegFaults. I just was telling about the .s middle-compiled files (which can be compiled with a cross-compiler built for soft-float and final-compiled .s->.o using a cross-compiler built for hard-float) and about the existence of the way to compile the newest packages with a hard-float which can be tested at least to differ how it works comparing to the same package compiled with soft-float.

The R5900 hardware can turn the FPU either on or off. It’s all or nothing, the hardware can’t turn on or off a selection of instructions...

Yes, FPU will be hardwarely on. While unsupported/wrong working ones (redefined as unsupported, absent in R5900) will be emulated by GCC. Easely said, simple to realize :p

It's not mips>>r5900<<el, so everything of it was compiled not for a mipsr5900el target... the base of it is mostly a busybox. That means that "mount/umount -a" will not read the /etc/fstab file, that ethernet abilities are some way cut off, and else... Am I right?

Just wanted to sum up how the question about the some Gentoo installing details raised up into a major FPU problems description :D In the comments of a patch pull request :] Which will be closed, and everything will gone... :/

---

Yesterday'es evening and today's morning I was building a cross-compiler with FP from sources for mipsr5900el target. binutils and gcc are compiled without any problem, but to build two glibc packages required some actions. First one, and the primary one, is the unsupported opcodes "l.d" and "s.d". I tried to solve it by injecting ".set mips2/3" into the asm volatile piece of code, but it "answered" me that these opcodes are not supported on these proccessors either. That problem appears in compiling stdlib/{__longjmp.c,setjmp.c,setjmp_aux.c} into .o and .os . To procceed the compilation, I was manually entering the large compiling command I saw while their compilation (with 'make' command) with an addition of the -U__mips_hard_float flag. Next one - "_Float32" is not supported to our target, edited the places with them rewritting them to "float" typedefs. Another one is while compiling math, I met a lot of places where the use of unsupported infinity needed, and with the flag -Werror they becomes an errors. Just removed that flag to proceed :/ And a next one for glibc-2.30 - could not compile e_sqrt.c while I have it compiled for glibc-2.13. Cause they are same, just copyed the compiled e_sqrt.o to the build directory of glibc-2.30. Any other further packages I compiled with such cross-compiler with FP was compiled without any problem. Tested. Chrooted into the mounted target-system. Bash operating normally. Executed pwconv, grpconv. When executed "groupadd" - got SegFault. I never saw such large description of the Segmentation Fault before) - looks like that's your work that it is so large and detailed (useless for my mind though :S). I had a suppose that it might be that if I'd compiled those executable binaries statically, then maybe there would not be any SegFault executing "groupadd". Could not test that theory, because the package "shadow" (motherpackage of the "groupadd" executable binary) just can not be compiled the way their executable binaries resulted statically. From the other hand, I had the glibc compiled with my intrudes... End of test. BUT! I did not saw a warning message of the kernel's FPU emulator activated.

By the way, which issue from the #3 list the omissions of the "l.d" and "s.d" opcodes is belongs to?

---

Wanted to try to build R5900 Gentoo, but it obviously needs crossdev for doing that. Is it available for Gentoo Linux only OR I can somehow install it? Quickly searching I can not find the info how it may be installed to non-Gentoo systems. Wanted to try to build cross compiler with FP based on musl, but met the same problems like I was compiling glibc with FP - s.d/l.d opcodes omission and can not compile scrt.c musl is just the one of the libc alternatives. There are many suchlike (but I know only about uClibc and tried only it).

[frno7]

Why not? The people like to watch the show)

Older packages are progressively more difficult to update, so if you have a choice, it’s easier to start with recent ones.

Rather than reading how to build something, where to place this or that thing... Or it might be another situation - for example, I am compiling some package but it not works like I want. I am installing the OS with that package included and configured, and study how it is configured. Since I understood, I am using that experience to configure that package right in the nowadays OS. Anyway, I understood how the precompiled MIPSel distribution will be working with the current 5.X kernel.

I’d recommend studying Gentoo on your workstation, or PC, that is your host computer first. If you, say, run Debian, the schroot package is very nice for this purpose. Later on, in fact, once you’ve managed to cross-compile a system for R5900, you can easily run that too within another schroot in combination with R5900 QEMU.

So building the cross-compiler from the sources from that wiki tutorial bloodybathly kills both birds - to setup a cross-compiler and to make a base system. Also, to get some experience in the compiling if to follow the tutorial by own hands (without using the .sh scripts).

Well, yes, as in the proverb _give a man a fish and you feed him for a day; teach a man to fish and you feed him for a lifetime_. :-)

Just wanted to sum up how the question about the some Gentoo installing details raised up into a major FPU problems description :D In the comments of a patch pull request :] Which will be closed, and everything will gone... :/

Oh, not gone, it’s just one extra mouse click to find all closed pull requests: try clicking on the button that says Closed!

Yesterday'es evening and today's morning I was building a cross-compiler with FP from sources for mipsr5900el target. binutils and gcc are compiled without any problem, but to build two glibc packages required some actions. First one, and the primary one, is the unsupported opcodes "l.d" and "s.d". I tried to solve it by injecting ".set mips2/3" into the asm volatile piece of code, but it "answered" me that these opcodes are not supported on these proccessors either. That problem appears in compiling stdlib/{__longjmp.c,setjmp.c,setjmp_aux.c} into .o and .os . To procceed the compilation, I was manually entering the large compiling command I saw while their compilation (with 'make' command) with an addition of the -U__mips_hard_float flag.

Nice! You’re progressing on fixing some advanced libc issues. Would you like to work it through and eventually submit patches for it? I guess we need to fix GCC first, which would be the ABI annotation mentioned in #3. I haven’t worked out the details on this topic yet, so I’m unable to help very much at this time.

I never saw such large description of the Segmentation Fault before) - looks like that's your work that it is so large and detailed (useless for my mind though :S).

What did it say?

BUT! I did not saw a warning message of the kernel's FPU emulator activated.

What would you expect?

By the way, which issue from the #3 list the omissions of the "l.d" and "s.d" opcodes is belongs to?

Have a look at the TX System RISC TX79 Core Architecture manual (the processor manual for the R5900), revision 2.0, Toshiba Corporation, page A-94. I believe it says that SD is MIPS III, so the ISA setting in the assembler might not have it even though the R5900 hardware certainly does.

Wanted to try to build R5900 Gentoo, but it obviously needs crossdev for doing that. Is it available for Gentoo Linux only OR I can somehow install it? Quickly searching I can not find the info how it may be installed to non-Gentoo systems.

The crossdev package uses Gentoo-specific features, so I don’t think you’ll find it elsewhere. But it’s certainly worth the effort, very powerful. It helps to be familiar with Gentoo in general, so as mentioned I do recommend playing with Gentoo in general first, within for example schroot on your workstation.

Wanted to try to build cross compiler with FP based on musl, but met the same problems like I was compiling glibc with FP - s.d/l.d opcodes omission and can not compile scrt.c musl is just the one of the libc alternatives. There are many suchlike (but I know only about uClibc and tried only it).

Hmm. I’m not familiar with all the problems we’re going to face with the FPU and issue #3. You’re ahead of me on this topic now. :-)

[Arch91]

Nice! You’re progressing on fixing some advanced libc issues. Would you like to work it through and eventually submit patches for it? I guess we need to fix GCC first, which would be the ABI annotation mentioned in #3. I haven’t worked out the details on this topic yet, so I’m unable to help very much at this time.

The things I done as the actions of the blind kitty. The test showed that there is no a stability, I got a SegFault in the place where everything fine in the same system of soft-float. It is unknown why - because of the doubtful edits I made in glibc, or it might be that the edits do the trick but the case in something another. Or, as I noted, it might be that if I'd compiled that binary statically, then I'd not got the SegFault launching that static binary.

I remembered that I had no any issue related to l.d/s.d in times of testing kernel 2.6, so I decided to return to the old binutils-2.24 and gcc-6.3.0. And I was correct; the problem is in binutils-2.34! The developers who accepted the Juergen's patch in their source code long time ago somehow do not correctly updated the binutils, so the newest version have the opcodes broken as l.d/s.d are not belongs to R5900 anymore! To test FPU from now on I am using binutils-2.24/gcc-9.2.0/glibc-2.30 combo. I successfully compiled glibc-2.30 with FP; I did not met an error about s.d/l.d, but the problem with e_sqrt.c still persists... I analysed how it is built in glibc-2.13 and found out that there it somewhy builds that file from sysdeps/ieee754/dbl-64/ instead of from sysdeps/mips/fpu/ directory. So, as a workaround, for glibc-2.30 I made the same - just replaced the file from the described directory. And these workarounds I like better. Already recompiled the target-system and tested - got the same SegFault by executing groupadd (see the photo below).

Also I was able to build musl with FP too, but I had to make the related replace for sqrt.c file too - from src/math/ to src/math/mips/

I never saw such large description of the Segmentation Fault before) - looks like that's your work that it is so large and detailed (useless for my mind though :S).

What did it say?

By the way, you did not answer me what is the device node for the current graphic console driver? I would like to know it, for example, for making a screenshot like cat /dev/fb0 > image.raw

BUT! I did not saw a warning message of the kernel's FPU emulator activated.

What would you expect?

From the FPU tests in kernel 2.6 times, I can say that I saw it message only in the moment when the fedora12_mips64el (ABI N32) just starting to load. And I never saw that message with the target-system which was compiled with hard-float. So, I think, I can make a conclusion that it is only triggered in cases of executing the libs/binaries compiled with soft-float

[frno7]

The things I done as the actions of the blind kitty. The test showed that there is no a stability, I got a SegFault in the place where everything fine in the same system of soft-float. It is unknown why - because of the doubtful edits I made in glibc, or it might be that the edits do the trick but the case in something another. Or, as I noted, it might be that if I'd compiled that binary statically, then I'd not got the SegFault launching that static binary.

It looks nasty, indeed. Does it crash like that with R5900 QEMU too? QEMU is very helpful to examine problems of these kinds. It’s highly recommend if you haven’t installed it already. :-) However, note that QEMU and real R5900 hardware don’t behave exactly the same. The R5900 is much too complicated to emulate perfectly for QEMU in the foreseeable future. Nevertheless, it’s worth a try and is often useful for debugging, compiling, and so on.

The developers who accepted the Juergen's patch in their source code long time ago somehow do not correctly updated the binutils, so the newest version have the opcodes broken as l.d/s.d are not belongs to R5900 anymore!

I’m not so sure Binutils is broken. Hmm, I’ve made a mistake, S.D is an alias for SDC1, see page D-34 in the previously mentioned TX System RISC TX79 Core Architecture manual. The R5900 doesn’t have double (64-bit) floats, only nonstandard single (32-bit) floats. So S.D won’t work. But the assembler will under some circumstances replace instructions that aren’t supported with ones that are. And MIPS has many different aliases for various instructions and registers. So, for example, S.D (SDC1) can sometimes be replaced by two S.S (SWC1):

% echo ' s.d $f0,($0)' >test.s
% mipsr5900el-unknown-linux-gnu-as test.s
% mipsr5900el-unknown-linux-gnu-objdump -d a.out
00000000 <.text>:
   0:   e4000000    swc1    $f0,0(zero)
   4:   e4010004    swc1    $f1,4(zero)
    ...

A proper SDC1 is rejected by the assembler:

% echo ' sdc1 $f0,($0)' >test.s
% mipsr5900el-unknown-linux-gnu-as test.s
test.s: Assembler messages:
test.s:1: Error: opcode not supported on this processor: r5900 (mips3) `sdc1 $f0,($0)'

Anyway, I believe any problem (like issue #3) involving a compiler (GCC) is almost always best solved by fixing the compiler first, and proceed from there. You’re best chance of obtaining FPU instructions in your programs right now is likely to compile for any other MIPS II processor that actually does have a good and proper FPU, have Gnu libc FPU support, etc. The R5900 isn’t there yet. But remember to use the -mfix-r5900 option. :-)

By the way, you did not answer me what is the device node for the current graphic console driver? I would like to know it, for example, for making a screenshot like cat /dev/fb0 > image.raw

There is no (virtual) frame buffer, as explained on the Graphics Synthesizer wiki page and in issue #10. But in this case one might as well do dmesg >dmesg.txt, which is even better, because it’s plain text. :-)

[Arch91]

Does it crash like that with R5900 QEMU too? ... However, note that QEMU and real R5900 hardware don’t behave exactly the same.

I prefer to test everything on the real hardware. Indeed, because the behavior may differs. It is not so hard to test on the real - I have my PS2 always connected to TV, it is connected "comfortable" so it is not laying on the floor like everyone stepping around - no, it is laying on the shelf of the cabinet (cupboard), the cabinet which TV is staying on. I just connect the USB keyboard and flash stick and I'm in.

Another side of the question is that whenever I got SegFault or I got not while executing groupadd in QEMU - that would gave me nothing, because, even if there would not be a SegFault, I already know that it IS there on the real hardware, and it is needed it to be researched and fixed.

And the most bad thing is that I do not have any conception of the debugging process known by me. I'm looking on the photo of SegFault like if I would seeing the art "La persistencia de la memoria" by Salvador Dalí - "picture is a picture, and so?.." All these ciphers, aren't they gives/says something to someone?!? The only that I got from the SegFault is that it might be the problem is in the memory/page allocation. Again, and what if it's so? - what's next? How to be sured? How to test, where to fix... an endless questions, the questions only they will not help to the project. However, that memory/page allocation problem (if to think that the problem is actually in it) appeared only when the target-system was compiled with hard-float, and when there is a soft-float there is no such problem...

S.D won’t work

So, the binutils version 2.24 is that which is the bugged one... s.d, store double, is not supported by r5900. You made a test and saw that sdc(1) can be rewritten with the two swc1... if only there could be an on-the-fly way to do the same trick while compiling the packages... Fot the certain example, __longjmp.c piece of code:

ifdef __mips_hard_float

asm volatile ("l.d $f20, %0" : : "m" (env[0].fpregs[0])); asm volatile ("l.d $f22, %0" : : "m" (env[0].fpregs[1])); asm volatile ("l.d $f24, %0" : : "m" (env[0].fpregs[2])); asm volatile ("l.d $f26, %0" : : "m" (env[0].fpregs[3])); asm volatile ("l.d $f28, %0" : : "m" (env[0].fpregs[4])); asm volatile ("l.d $f30, %0" : : "m" (env[0].fpregs[5]));

endif

_ is there a way to make a test with them only to see how they will be rewritten into those swc(1) (and the same with l.d)? So eventually we could write the results back into the code, at least manually by our own hands. Or it is not right - to cut the piece of code, rewrite it and think that the whole code will be working with those rewritions?.. That's the conflicted issue, the logic is going to compile there because of the branch of the united options mips3 and hard-float. MIPS-III support doubles, gcc emulately handles them in C code but when there are an assembler inlines, it can do nothing with them on it's own, it needs help)

to compile for any other MIPS II processor that actually does have a good and proper FPU, have Gnu libc FPU support, etc. The R5900 isn’t there yet. But remember to use the -mfix-r5900 option

The gcc source is nowadays constructed the way so when building it for target mipsr5900el--linux- it provides such flags like -march=r5900 and -mfix-r5900 by default. Do you know where in gcc source to edit the default flags for the target mipsel-unknown-linux-gnu? For the FPU purposes then it is needed to change mips1 to mips2 and add -mfix-r5900. The specs file can be somehow edited for that purpose as well. However, I dumped the specs and it is not obvious where to add them to they be used by default in there. I'd prefer to not using the way of playing with setting the CFLAGS while compiling packages. Not every package with Makefiles is constructed like it is accepting CFLAGS, there might be a rules in them to compile something without CFLAGS.

There is no (virtual) frame buffer, as explained on the Graphics Synthesizer wiki page and in issue https://github.com/frno7/linux/issues/10. But in this case one might as well do dmesg >dmesg.txt, which is even better, because it’s plain text. :-)

Well, I know how to get the output into the file, but I wanted a screenshot) I compiled ImageMagick... and it provides the SegFault too))

[AKuHAK]

@Arch91 as you are testing things on the real console, could you please comment on this issue https://github.com/frno7/linux/issues/34 if you have some working binaries, or, better, step-by-step guide how to compile them.

[Arch91]

Hello, @AKuHAK ! Yes, the guide is already done, read the wiki. Since it is on the wiki, I think there is no reason to propagate it everywhere, my opinion. If you want to build it - go ahead an read it. You'll obtain that there are the .sh scripts to build everything much faster. Also, pay attention to the final stage, the actions which have to be done manually. Building and installing PlayStation 2 Linux -> building programs for PlayStation 2 Linux

If you'll have the questions on some stages, share them.

[AKuHAK]

Oh, ok, i can try again. The guide is quite old (March, 2020). And I remember that I faced critical problems when trying to compile it 2 years ago. But if you state that following guide 1:1 will produce working setup without facing some non-standart problems during that process...

[Arch91]

The guide about how to build the whole system from the sources was "birthed" about a week ago) I am using the same files/additions/patches by myself. Used it yesterday to try to build the system with hard-float... in that case, from step 10 (ncurses) everything was compiled fine...

Note! Read the .sh scripts file, there might be a hints to solve some issueses which may appear (they may needs an attention which can not be automatized in scripts such like to install something to host PCLinux).

[frno7]

I prefer to test everything on the real hardware. Indeed, because the behavior may differs. It is not so hard to test on the real

@Arch91, R5900 QEMU lets you pretend you had a 32 gigabyte 32 multicore 32 terabyte disk PlayStation 2. That’s useful sometimes. But no hurry, R5900 QEMU will be waiting for you when you need it. :-)

I have my PS2 always connected to TV, it is connected "comfortable" so it is not laying on the floor like everyone stepping around - no, it is laying on the shelf of the cabinet (cupboard), the cabinet which TV is staying on. I just connect the USB keyboard and flash stick and I'm in.

Nice setup! I’ve added the USB wifi adapter Ralink Technology RT3070 to the mix, to login remotely via ssh etc. I had to reduce the driver memory requirements a bit though, see commit 844ad983a20ea374589ffc033933c0dedd65bc70, as explained on the USB wiki page.

Another side of the question is that whenever I got SegFault or I got not while executing groupadd in QEMU - that would gave me nothing, because, even if there would not be a SegFault, I already know that it IS there on the real hardware, and it is needed it to be researched and fixed.

QEMU is often helpful as a comparison in these cases. If QEMU and the hardware behaves the same, it’s less likely a bug in the kernel. If they behave differently, things become more interesting.

And the most bad thing is that I do not have any conception of the debugging process known by me. I'm looking on the photo of SegFault like if I would seeing the art "La persistencia de la memoria" by Salvador Dalí - "picture is a picture, and so?.." All these ciphers, aren't they gives/says something to someone?!?

Oh, yes, some things appear cryptic now, but you’ll learn more with time, bug-by-bug, by studying the fixes for them. Eventually you’ll learn to read and recite segmentation faults like poetry by Александр Сергеевич Пушкин.

is there a way to make a test with them only to see how they will be rewritten into those swc(1) (and the same with l.d)?

objdump is often useful to disassemble and examine the instructions that are actually used, but I’m not familiar with all the rules on mnemonic substitutions by the assembler...

That's the conflicted issue, the logic is going to compile there because of the branch of the united options mips3 and hard-float. MIPS-III support doubles, gcc emulately handles them in C code but when there are an assembler inlines, it can do nothing with them on it's own, it needs help)

Yes, it seems likely we’ll need to fix libc in some ways.

Do you know where in gcc source to edit the default flags for the target mipsel-unknown-linux-gnu?

Unfortunately I haven’t studied GCC enough to change these defaults myself.

I'd prefer to not using the way of playing with setting the CFLAGS while compiling packages. Not every package with Makefiles is constructed like it is accepting CFLAGS, there might be a rules in them to compile something without CFLAGS.

Ah, indeed, there may be broken packages to deal with. But high-quality packes should work fine. I suppose it’s most convenient to try them first, for example Busybox?

[Arch91]

I found out how to build ISA MIPS-II ABI O32 mipsel-unknown-linux-gnu cross-compiler with default flags -msingle-float and -mfix-r5900. Additions to binutils configure: --with-arch=r6000 --with-cpu=r6000 additions to gcc: --with-arch=r6000 --with-fpu=single edit file gcc/config/mips/mips.c - add the next lines under the function with a related comment:

• / Default to working around R5900 errata /
• if (strcmp (mips_arch_info->name, "r6000") == 0)
• target_flags |= MASK_FIX_R5900;

Explanations. According to gcc/config/mips/mips-cpus.def the only available target for that purpose as to build with MIPS-II is R6000. About the default flag -mfix-r5900 . It is a doubtful addition to the mips.c file because there is a comparison to the zero... anyway, it was added, checked with the command mipsel-unknown-linux-gnu-gcc -Q --help=target

---

@frno7, inspirited with your s.d asm "melting" into a two sdc1, I "invented" a patch for glibc-2.30 . I noticed that the next one is bigger for a 4 in a comparison to the previous one. For to get that correct, I made a change as to define fpregs as float instead of the original double together with expansion of the number of the... regs?.. from 6 to 12. The last one triggered the change of the values for the variables in sysdeps/unix/sysv/linux/mips/mips32/jmp_buf-macros.h . I corrected the values of the variables here playing with the related Static_assertion . I think that all of these are reasonable, ANYWAY, I feel myself a blind kitty again, because I can't/I don't know a way how to test anything of that to be sure. sqrt.d is the opcode which is also not supported on our target. I do not found a way to "melt" it) I only found a mention about sqrt.s in the .pdf you was mentioning.

I even wanted to ask the question to the GNU developers, but I forgot what nickname I am registered there with...

Recompiled target-system... and got the same SegFault in the same place.

With those fpregs edits, from now on I am noticing the warnings from time to time while compiling: Warning: float register should be even, was (some uneven value) And those uneven values from the warnings are matching to the edits in getcontext.S/setcontext.S/swapcontext.S where I added an uneven fs*f registers 21,23,25,27,29,31. Except value 1, I did not add that.

---

Compiled gdb. Got SegFault launching only it. Compiled it statically, got it's static binary. Got SegFault launching only it. My conclusion for myself is to forget about FP for a long period of time.

[frno7]

Explanations. According to gcc/config/mips/mips-cpus.def the only available target for that purpose as to build with MIPS-II is R6000.

I completely agree, as I’ve learned that the R6000 is the ideal choice for MIPS II. The hardware itself was apparently a rarity, but it functioned as a lowest common denominator for many other systems. According to Wikipedia it was the first implementation of the MIPS II ISA and had few users. Control Data Systems (CDS) used an 80 MHz version in their high-end 4680-300 Series InforServer server at the time.

About the default flag -mfix-r5900 . It is a doubtful addition to the mips.c file because there is a comparison to the zero... anyway, it was added, checked with the command mipsel-unknown-linux-gnu-gcc -Q --help=target

I’ve made a tool r5900check to analyse machine code and check whether an ELF has the short loop hazard. It can be used to verify that -mfix-r5900 works as intended, and that all cases have been catched. Sometimes noreorder and other directives in assembly source code are used to override command and architecture settings. Analysing the final machine code may be the best alternative to handle those.

sqrt.d is the opcode which is also not supported on our target. I do not found a way to "melt" it) I only found a mention about sqrt.s in the .pdf you was mentioning. I even wanted to ask the question to the GNU developers, but I forgot what nickname I am registered there with...

I believe Maciej W. Rozycki is happy to guide you on how to get started, should you decide to contribute fixes and extensions to these GNU packages. He maintains some of these things. That’d be very helpful of you! You’ve made some progress already, and previously had conversions with him by email.

Compiled gdb. Got SegFault launching only it. Compiled it statically, got it's static binary. Got SegFault launching only it. My conclusion for myself is to forget about FP for a long period of time.

Hmm. What kind of segfault was it? Last time I attempted to start GDB on the hardware it crashed badly too, but then it had run out of memory. When #18 is completed, a swap can be added easily, relieving memory pressure. Modern SSD devices have excellent performance and do fairly well according to my benchmarks. And, well, as a reminder R5900 QEMU should be able to verify whether your GDB is complete crap or actually working. :-)

[Arch91]

I’ve made a tool r5900check to analyse machine code and check whether an ELF has the short loop hazard. It can be used to verify that -mfix-r5900 works as intended, and that all cases have been catched. Sometimes noreorder and other directives in assembly source code are used to override command and architecture settings.

Compiled by manually edit it's Makefile by adding variables AR,AS,CC,LD,RM there. By the way is there a way to compile it automatically like some command from the root of the kernel source? Like "make ARCH=... ... tools" ? Launched r5900check on some binaries and on libc shared lib. In each launch I saw an outputs with "erratum shortloop path /path/to/that/I/launched" and some addresses with a values with -/+'1/2 and zero between them. As for me, I do not see any hint for a -mfix-r5900 presence) Or that erratum shortloop is the confirmation about it?)

Well, the kernel, libraries and tools are certainly responsible for maintaining o32 ABI and MIPS I, II and III compatibility

So that's the tools you were mentioning) I thought about PSX console devices... :D I know one guy which owns a pare of them at once.

R5900 QEMU should be able to verify whether your GDB is complete crap or actually working. :-)

QEMU, QEMU... Since I am building everything from the sources, to load the system well I have to chroot into it and launch the commands "pwconv" and "grpconv". If to not do that once before the full functional use, the system will stack somewhere while loading. However, those commands passed well... so it might be I can launch QEMU passing the path to the mounted USB flash stick...

whether your GDB is complete crap or actually working.

The case is not in GDB... there is something in glibc source or even in gcc compiled code provided. By the way, since swap area (from /dev/sda2) activated, the SegFault messages becomes about 2-3 lines output. @frno7, you were talking something about swap somewhere... If I not mistake you were noting to activate the swap partition from the internal HDD. But what about activating it from USB? As an owner of SLIM console device I can mount swap partition only in two ways - from USB flash stick or as a loop_file from some shared directory through (currently unavailable) ethernet. Did you mean about that PS2 RAM is not enough to handle USB I/O operations so activating the swap partition from USB flash stick is not recommended? Or did I understand your context wrong? I am an owner of the FAT console device either... but I aware of triggering the one more big discussion about how to create linux acceptable partitions in the HDD connected to expansion bay, the partitions which would be existing together with the partition for the OPL/HDL games. I bought the FAT PS2 relatively not so a long period of time ago and almost did not use it at all. It's current purpose if to absorb the dust :D

---

I currently have a two cross compilers one in /usr/local/ps2 directory, that's mipsr5900el soft float another one in /usr/local/ps3 directory, that's the described mipsel R6000 with hard single float and -mfix-r5900 flag and with my doubtful glibc asm edits.

I compiled gdb statically using soft float mipsr5900el cross compiler and tried to launch it on the system of hard-float. I expected to see the related output right after it's launch: Algorithmics MIPS FPU emulator but that did not happened. Weird. The static gdb binary launch comes to a SegFault. And a no to compare, I did not tried to launch gdb on the system of soft-float.

[frno7]

Compiled by manually edit it's Makefile by adding variables AR,AS,CC,LD,RM there.

That’s correct!

By the way is there a way to compile it automatically like some command from the root of the kernel source? Like "make ARCH=... ... tools" ?

No, I just put r5900check in the Linux repo for convenience, to have it around somewhere. It might just as well be in a completely separate repo. I should probably do that eventually!

Launched r5900check on some binaries and on libc shared lib. In each launch I saw an outputs with "erratum shortloop path /path/to/that/I/launched" and some addresses with a values with -/+'1/2 and zero between them. As for me, I do not see any hint for a -mfix-r5900 presence) Or that erratum shortloop is the confirmation about it?)

My apologies, r5900check really is poorly documented. Let’s look at an example by analysing Busybox:

% linux/tools/r5900check/r5900check initramfs/ps2/sbin/busybox
erratum shortloop path initramfs/ps2/sbin/busybox
code 00403058 -3 00021040
code 0040305c -2 0047302b
code 00403060 -1 00602021
code 00403064  0 14c0fffc
code 00403068  1 2463ffff

Column 2 is the address, column 3 is the offset relative to the jump instruction (with the actual jump at the zero offset), and column 4 the machine code. We can easily disassemble this piece with objdump by printing the relevant addresses using a small sed script,

% mipsr5900el-unknown-linux-gnu-objdump -d initramfs/ps2/sbin/busybox | sed -n '/403058:/,${p;/403068:/q}'
  403058:   00021040    sll v0,v0,0x1
  40305c:   0047302b    sltu    a2,v0,a3
  403060:   00602021    move    a0,v1
  403064:   14c0fffc    bnez    a2,0x403058
  403068:   2463ffff    addiu   v1,v1,-1

and find that we have this backwards branching BNEZ (branch on not equal zero) instruction. Now let’s review the conditions for the R5900 short loop erratum, which says

The short loop bug under certain conditions causes loops to execute only once or twice. We must ensure that the assembler never generates loops that satisfy all of the following conditions:

• a loop consists of less than or equal to six instructions (including the branch delay slot);
• a loop contains only one conditional branch instruction at the end of the loop;
• a loop does not contain any other branch or jump instructions;
• a branch delay slot of the loop is not NOP (EE 2.9 or later).

As we can see, the loop only has five instructions, it has only one conditional branch (BNEZ), with no other branches or jumps, and no NOP in the branch delay slot (it’s ADDIU). Danger, indeed! So that’s what the r5900check tool is about. Of course, the next step is to find out why Busybox has this piece of problematic code...

Well, the kernel, libraries and tools are certainly responsible for maintaining o32 ABI and MIPS I, II and III compatibility

So that's the tools you were mentioning) I thought about PSX console devices... :D I know one guy which owns a pare of them at once.

Tooling to me is mostly GCC, GAS, etc. Sometimes also screwdrivers and suchlike. :-)

QEMU, QEMU... Since I am building everything from the sources, to load the system well I have to chroot into it and launch the commands "pwconv" and "grpconv". If to not do that once before the full functional use, the system will stack somewhere while loading. However, those commands passed well... so it might be I can launch QEMU passing the path to the mounted USB flash stick...

R5900 QEMU is user mode emulation. There’s currently no system mode emulation. Let’s look at Busybox again. It identifies itself as a MIPS R3000 program:

% readelf -h initramfs/ps2/sbin/busybox | grep Machine:
  Machine:                           MIPS R3000

Now let’s run the Busybox built-in Awk with a small floating point calculation, via qemu-mipsr5900el on a POWER9 host system (for avoidance of doubt, this is not on PlayStation 2 hardware):

% qemu-mipsr5900el initramfs/ps2/sbin/busybox awk 'END { print 1.23 + 4.56 }' </dev/null
5.79

My Gentoo overlay has an ebuild for the app-emulation/qemu-mipsr5900el package, so it’s simple to install it with emerge. Of course, one could also build R5900 QEMU from sources too. The relevant configure option is --target-list=mipsel-linux-user. As one can see, it really is easy to run R5900 Linux programs on the host computer via R5900 QEMU.

By the way, since swap area (from /dev/sda2) activated, the SegFault messages becomes about 2-3 lines output.

What does it say?

If I not mistake you were noting to activate the swap partition from the internal HDD.

True!

But what about activating it from USB? As an owner of SLIM console device I can mount swap partition only in two ways - from USB flash stick

That works too, but it’d be very slow. Also, USB sticks tend to wear out and become destroyed after many write-cycles. SSDs are much more capable of handling a lot of writing.

or as a loop_file from some shared directory through (currently unavailable) ethernet.

Yes, that might work too, but even slower probably. :-)

Did you mean about that PS2 RAM is not enough to handle USB I/O operations so activating the swap partition from USB flash stick is not recommended? Or did I understand your context wrong?

GDB requires a lot of memory, sometimes much more than is available on the PlayStation 2. It might be possible to make GDB use less memory, I don’t know. So the swap was intended to manage all the memory GDB would use. If GDB doesn’t get enough memory, it most likely crashes. It should be possible to figure that out from the printed segmentation fault message, possibly also via dmesg.

I am an owner of the FAT console device either... but I aware of triggering the one more big discussion about how to create linux acceptable partitions in the HDD connected to expansion bay, the partitions which would be existing together with the partition for the OPL/HDL games. I bought the FAT PS2 relatively not so a long period of time ago and almost did not use it at all. It's current purpose if to absorb the dust :D

Ah, do you have a harddisk in your expansion bay? An SSD? I suppose issue #39 is about having Linux play nicely with those partitions. :-)

[Arch91]

Danger, indeed! So that’s what the r5900check tool is about.

Well... I understood only that that will be a tool to remember of when some executable binary will be not working. To check by it too.

R5900 QEMU is user mode emulation. There’s currently no system mode emulation.

What if I do not go to launch emulately anything compiled for R5900 specific?) In other words, the "usual" qemu-system-mipsel is enough, when anything was compiled by that R6000 mipsel-unknown-linux-gnu cross compiler. So I installed qemu and followed this guide to install malta's mipsel-linux debian-11.2.0. As I foresaw I copied the target-system from my USB flash stick to the qemu's image disk; chrooted into it. And got the same segfaults in the same places I was getting on the real hardware. Once again, I am noting that it was a test of the system of hard-float with my glibc doubtful patch. And in case of when the system is of soft-float, there is no any (doubtful) patch needed for glibc, and there is no those SegFaults in those places.

USB sticks tend to wear out and become destroyed after many write-cycles.

I wish that after the PS2Linux kernel will be fully optimized and "cured" from the current issues, this will be a single problem :D In other words, currently the USB will be enough for the tests as they are pretty rare, they are not "many cycles", so USB flash stick will be living) Besides, that's my special USB stick, for PS2Linux only.

or as a loop_file from some shared directory through (currently unavailable) ethernet.

Yes, that might work too, but even slower probably. :-)

On the contrary, it is faster than USB both in the theory and in the practice. Theory - USB 1.1 R/W operations speed is much lower than ethernet's. USB 1.1 R/W operations speed is faster only than the memory card's R/W :p Practice - I used it in PS2Linux 2.4/2.6, but the one little thing was each time in the moment when I activated the swap loop-file, there was such message like "swap file ... holes" (makes/contains/includes/uses... I forgot the exact word). And using it that way in 2.4 was pretty stable and that swap RAM achievable.

Did you mean about that PS2 RAM is not enough to handle USB I/O operations so activating the swap partition from USB flash stick is not recommended? Or did I understand your context wrong?

GDB requires a lot of memory...

No, I mean another... You were saying about that you reduced a memory allocated by the driver specially for your WI-Fi USB stick. And thought that USB memory allocation for the USB flash stick with swap partition activated will be unstable. But looks like I am incorrect here, nothing to worry about.

Ah, do you have a harddisk in your expansion bay? An SSD?

I have a chinese IDE interface to SATA replacement replaced in the official network adapter. 3.5" SATA HDD 500Gb connected into it.

issue https://github.com/frno7/linux/issues/39 is about having Linux play nicely with those partitions

Well... do not want to obtain those things empirically, at least yet. I would like to follow the fine guide - a program, an example, etc.

By the way ... the SegFault messages becomes about 2-3 lines output.

do_page_fault(): sending SIGSEGV to gdb for invalid write access to 7f114fec epc = 00b2f364 in gdb[400000+9bf000] ra = 00b2f374 in gdb[400000+9bf000] Segmentation fault

Any SegFault looks same. Invalid write access.

[frno7]

Well... I understood only that that will be a tool to remember of when some executable binary will be not working. To check by it too.

I’ve updated the wiki on verifying programs and libraries.

What if I do not go to launch emulately anything compiled for R5900 specific?) In other words, the "usual" qemu-system- mipsel is enough, when anything was compiled by that R6000 mipsel-unknown-linux-gnu cross compiler.

Sure, QEMU emulates several MIPS systems!

So I installed qemu and followed this guide to install malta's mipsel-linux debian-11.2.0.

There’s no need to install Debian (or any other distribution) to run R5900 QEMU. Statically linked programs simply runs from the command line, for example qemu-mipsr5900el initramfs/ps2/sbin/busybox uname -a. It should be possible to define LD_LIBRARY_PATH to run dynamically linked programs with QEMU too, if memory serves me. One can also use schroot, as previously mentioned, with INITRAMFS to test it before booting it on the actual hardware. binfmt_misc can be used to run R5900 programs as if they were normal for the host:

echo 0 >/proc/sys/fs/binfmt_misc/qemu-mipsel
echo ':qemu-mipsr5900el:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/local/bin/qemu-mipsel:' >/proc/sys/fs/binfmt_misc/register

As I foresaw I copied the target-system from my USB flash stick to the qemu's image disk; chrooted into it. And got the same segfaults in the same places I was getting on the real hardware. Once again, I am noting that it was a test of the system of hard-float with my glibc doubtful patch.

Some malfunction with the patch then.

And in case of when the system is of soft-float, there is no any (doubtful) patch needed for glibc, and there is no those SegFaults in those places.

Good, but does GDB run on hardware then? I thought it would run out of memory anyway. R5900 QEMU should work since your host (most likely) has much more memory.

I wish that after the PS2Linux kernel will be fully optimized and "cured" from the current issues, this will be a single problem :D In other words, currently the USB will be enough for the tests as they are pretty rare, they are not "many cycles", so USB flash stick will be living) Besides, that's my special USB stick, for PS2Linux only.

Installing Linux images a couple of times per day for a few years is likely no problem whatever. I was worried about turning it to a memory paging (swap) device, which could subject it to massive amounts of writing that would destroy the poor flash stick in mere hours.

do_page_fault(): sending SIGSEGV to gdb for invalid write access to 7f114fec
epc = 00b2f364 in gdb[400000+9bf000]
ra = 00b2f374 in gdb[400000+9bf000]
Segmentation fault

Any SegFault looks same. Invalid write access.

With a soft-float GDB?

[Arch91]

With a soft-float GDB?

No, every SegFault I wrote here was an output from a launched gdb on the system of hard-float (with my silly patch and without it, original glibc), even the static soft-float gdb. I did not test the soft-float system and it's gdb yet.

---

I even wanted to ask the question to the GNU developers, but I forgot what nickname I am registered there with...

I was asking the help from the bugzilla team by filling a bug. https://sourceware.org/bugzilla/show_bug.cgi?id=28872

All these experienced people of the whole their good great gnu libc library just SCREWED with it. Just as I foresaw, the answerer just read my article which I filled with my noob understanding without explanation/terminology and etc. smart, and he just answered that "you are doing something that is not related to us. Compile as we provided or get lost." However, it might be that both he is a wrong person to discuss that with and I filled the bug wrongly, not in the purposed category. If you know what should be written there, please, share. Because with my English and terminology it's a dead end.

[frno7]

No, every SegFault I wrote here was an output from a launched gdb on the system of hard-float (with my silly patch and without it, original glibc), even the static soft-float gdb. I did not test the soft-float system and it's gdb yet.

Did any of these GDBs start successfully with QEMU emulating the R5900 or the R6000? If any of them did, that GDB is probably golden, and a good one for actual hardware, except I very much suspect it runs out of memory, and crashes badly for that reason, as you’ve probably noticed already. So a smaller, simpler, program than GDB would be easier to try, really. :-)

However, it might be that both he is a wrong person to discuss that with and I filled the bug wrongly, not in the purposed category. If you know what should be written there, please, share.

The R5900 is quite strange, so I would forgive them for not understanding much about this. I would almost certainly fail trying to file a similar bug report in Bugzilla. For the FPU, issue #3, I believe we should try to figure out where to begin. For example:

1. do we want to run a ‘hard’-float R6000 program on the R5900, expecting its FPU instructions to be emulated by the kernel?
2. or, make GNU libc for the R5900 support the R5900 FPU?

I think (1) is easier to try, and I hope it already works. For (2) I’m not really sure where to begin, perhaps GCC and GAS need to be patched first, before proceeding with libc. Maciej W. Rozycki can probably indicate some good starting points. He knows a lot about the oddities of the R5900 too. The next step is most likely to write a patch, extending or fixing some part (of GAS, for instance), and submit it (to the Binutils mailing list) for review and inclusion.

Because with my English and terminology it's a dead end.

I think you’ll do alright if we can figure out where to begin this effort with the FPU. It’s probably easier to focus on a more specific and hopefully simpler part (GAS, for example) to begin with, and proceed to more complex parts later on.

[Arch91]

Did any of these GDBs start successfully with QEMU emulating the R5900 or the R6000?

No, any of the hard-float compiled gdb failed on QEMU either, and the launches shows the same SegFaults.

I very much suspect it runs out of memory, and crashes badly for that reason, as you’ve probably noticed already.

I'm sure it will be OK. I was launching it previously in kernel 2.6 tests. Actually, I was launching a lot in the system of hard-float successfully, when my toolchain was binutils-2.24 (almost original, not specific for r5900 bfd patch) gcc-6.3.0 (gcc-6.3.0 with Juergen's specific patch) and uClibc-ng-1.0.22 (original with not specific for r5900 patches) I'm sure it is a glibc what is ruining all my FP plans!)) Which is responsible for all these SegFaults. I could check that suppose by building from the described toolchain based on your kernel headers... But you denied me to use gcc older than 9.2.0) Also I need that the libc base be glibc. My goal is to successfully run some newer (than in the original PS2Linux) firefox with hard-float. The successful run will be enough)

1. do we want to run a ‘hard’-float R6000 program on the R5900, expecting its FPU instructions to be emulated by the kernel?
2. or, make GNU libc for the R5900 support the R5900 FPU?

I think neither it, nor that. My opinion is

3. To rely the FPU instructions to be emulated by GCC, which means they are reworked into a supported ones by it. And in each place of the met unsupported opcode in assembler inlines must be manually rewritten (I mean in the packages we are trying to compile, not in GCC; in GCC we'll rely to the FPU emulation currently realized, for single float).

In that bug request I would like to they help us. I think we came to a consensus there about ABI, doubles and the logic branch. Now it's up to them to involve into the problem practically, not with the links and theories to study. The experience ungivers, do not want to share their experience... Anyway, I'm shooting a shots in the random directions in the hope that some of the bullets will hit the target)

I would almost certainly fail trying to fill a similar bug report in Bugzilla. For the FPU, issue #3 At least we know the differences in r5900 FPU. To start fixing them I think we should get the fail first related to that or another issue. And trace where it starts from.

I believe the gdb compiled for soft-float will be working OK. I'm already run out of free days now, so such kind of test becames rarely.

[frno7]

No, any of the hard-float compiled gdb failed on QEMU either, and the launches shows the same SegFaults.

What version of GDB is it? I concede that GDB was nimble in the olden days. Old enough will be small enough to fit within the memory constraints. :-)

I'm sure it is a glibc what is ruining all my FP plans!)) Which is responsible for all these SegFaults. I could check that suppose by building from the described toolchain based on your kernel headers... But you denied me to use gcc older than 9.2.0)

Are you prepared to go low-level, into MIPS assembly, with your FPU tests? If so, I suggest compiling a trivial C function, such as float f(float a, float b) { return a + b; }, and examine the resulting assembly and object machine code, with various compiler settings. Similar to what Adhemerval Zanella suggested for a f.c.

Next, you could write a very small program entirely in MIPS assembly, without using libc at all. Its exit code integer could be used as a means to return computational ‘results’ of the tests. That’s much simpler than invoking kernel system calls (to for example print text on the console), which of course is a possibility in assembly too, but I suggest skipping such complications to begin with. The only dependency on tooling would be the MIPS assembler, GAS, which is straight-forward to precisely control with ISA settings etc. The kernel then, does it behave as expected, with respect to the floating-point operations in your test program?

3. To rely the FPU instructions to be emulated by GCC, which means they are reworked into a supported ones by it. And in each place of the met unsupported opcode in assembler inlines must be manually rewritten (I mean in the packages we are trying to compile, not in GCC; in GCC we'll rely to the FPU emulation currently realized, for single float).

Hmm, do mean that GCC would emit a combination of real FPU instructions, in some cases, and FPU emulation in other cases?

In that bug request I would like to they help us. I think we came to a consensus there about ABI, doubles and the logic branch. Now it's up to them to involve into the problem practically, not with the links and theories to study. The experience ungivers, do not want to share their experience...

If they’re volunteer and enthusiast maintainers I wouldn’t expect them to write the patches for you. The two main options are:

1. write the FPU patches yourself (with some help and guidance along the way), or
2. patiently wait for someone else to do it (most likely a volunteer and enthusiast, having an interest in the R5900).

[Arch91]

What version of GDB is it? I concede that GDB was nimble in the olden days. Old enough will be small enough to fit within the memory constraints. :-)

You underestimate the possibilities of our good old console devices :D Compiled gdb-11.2 for the system of soft-float and it is working with swap partition activated. But to debug float point issues with gdb, it must be compiled for the system of hard-float. If I not mistake, Juergen also had a patch for gdb... I remember it's existence but I don't it's includings, what it is fixing in. Maybe even nothing critical at all... Maybe ll/sc...

1. write the FPU patches yourself (with some help and guidance along the way), or
2. patiently wait for someone else to do it (most likely a volunteer and enthusiast, having an interest in the R5900). Are you prepared to go low-level

No, I am not :p Simplest to say, that's beyond my mentality.

The kernel then, does it behave as expected, with respect to the floating-point operations in your test program?

Can't write a test program, can test a program :] When program will be malfunctioning, then there will be an actions to do. Including to test it in more stable MIPSel linux system, such as which is based on r6000. There are some projects of the test-files only. I think it is reasonable to compile it for the system of soft-float and hard-float, and to compare their output results. Maybe some day.

If they’re volunteer and enthusiast maintainers I wouldn’t expect them to write the patches for you.

I not actually consider that it should be done especially for me - nowadays it is impossible to get any libc compiled (because of the files structure is same; I mean the files noted in the patch) with fp enabled if your MIPSel target have only single float at the disposal. glibc is not an exception here. So, let they get their lazy... hands... into thinking-out the forgotten branch) Of ABI 32 hard float, now with single float. But seriously, yes, I agree with you - when this is a giant project, the person who will want to handle some trifle will be considered rather as an enthusiast than a bugfixer.

Hmm, do mean that GCC would emit a combination of real FPU instructions, in some cases, and FPU emulation in other cases?

Yes. But I don't think it works exactly with assembler instructions. I think, looking from the aspect of compiling the code objectly, GCC can rewrite the C code into a suitable output with the supported opcodes generated in it. Cross compiler based on binutils-2.24 can compile a .c file which contains s.d in asm volatile (and if to see objdump of the file.o compiled by gcc, there are swc1 opcodes will be in the places of where the s.d ones are in .c code; as well as in your presentation with the file a.out generated with as). If the cross compiler is based on binutils>2.24 - GCC rumbles that s.d is the unsupported opcode for our target without double-float.

[Arch91]

@frno7, I got an executable binary which loads continuously (that's ImageMagick's import; I was waiting about 2hours and, according to the flashing led of my USB flash stick, it still was loading). I checked it with r5900check.

code 0040055c -4 82040000
code 00400560 -3 26100001
code 00400564 -2 308300ff
code 00400568 -1 2462fff7
code 0040056c  0 1065fffb
code 00400570  1 2c420005

  40055c:   82040000    lb  a0,0(s0)
  400560:   26100001    addiu   s0,s0,1
  400564:   308300ff    andi    v1,a0,0xff
  400568:   2462fff7    addiu   v0,v1,-9
  40056c:   1065fffb    beq v1,a1,40055c <parse_boolean+0xa4>
  400570:   2c420005    sltiu   v0,v0,5
  400574:   5440fffa    bnezl   v0,400560 <parse_boolean+0xa8>
  400578:   82040000    lb  a0,0(s0)
  40057c:   1080000e    beqz    a0,4005b8 <parse_boolean+0x100>

You were saying that six outputs found by r5900check is still fine. Did I met some FPU issue? If yes then which one?

[frno7]

You were saying that six outputs found by r5900check is still fine.

No, I didn’t. Less than or equal to six instructions, is a potential problem, says the R5900 short loop erratum. :-)

Did I met some FPU issue? If yes then which one?

Does it work with QEMU?

[Arch91]

Less than or equal to six instructions

Oooh... <=6 is BAD, and >6 is good :D (I firstly understood vice versa)

Does it work with QEMU?

I compiled a toolchain from binutils-2.24 gcc-9.2.0 and musl with your ll/sc patch for a target mipsr5900el-unknown-linux-musl, so I can't test it behavior in qemu because of when testing in qemu it shows "Illegal Instruction" on launch. Though, which instruction is illegal for the qemu's debian ABI o32 ISA mips32r2 is unknown; besides according to that debian's gcc, it has the flags -mdouble-float set by default, so there should be nothing unacceptable, and moreover mipsr5900el-unknown-linux-musl target do not provides anything of r5900 special instruction, right?.. Anyway, that was not about to begin to involve into it. I just was interesting whenever I met something from FPU issue or I did not. I am still searching for the more or less stable toolchain which provides fine and correct FPU instructions... And since there is no a workaround for the any mipsel libc to build it with fp without using l.d/s.d, I'm like a walking along the bottom of the chinese great wall searching for an entrance.

[frno7]

Though, which instruction is illegal for the qemu's debian ABI o32 ISA mips32r2 is unknown

Would you be able to find out, using for example GDB?

[Arch91]

@frno7, gdb says it is mult v1,a1,v1

Can mult operate with the three registers?.. According to that .pdf manual:

The C790 supports three-operand multiply instructions that store the multiply result to a general purpose register in addition to the LO register. These instructions, as such, don’t have to use the MFLO instruction to move data from the LO register to a general purpose register.

I analyzed all .o compiled files by "_objdump -d | grep mult > somefile.txt", and there is no such triple registered mult's instructrions in them, but the result magick executable binary file do have them a lot. Some of them the related type when the first operand is same as the last operand.

[frno7]

Can mult operate with the three registers?

Oh, sure, see [Qemu-devel] [PULL 17/34] target/mips: Support R5900 three-operand MULT. :-)

[Arch91]

@frno7, thanks for a direct link for a patch. I was rebuilding qemu-3.1.1 from source .rpm for my PCLinux. There I witnessed that that patch is already implemented there. So, qemu-system-mipsel can not handle the output binaries from the cross-compiler built for the target mipsr5900el. And either the original qemu-mipsel can not launch the related output binaries as it says that it do not knows about R5900 model. Compiled your r5900 qemu. Launched that magick using it. Got these:

qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault

[frno7]

So, qemu-system-mipsel can not handle the output binaries from the cross-compiler built for the target mipsr5900el.

There’s no QEMU system emulation of the PlayStation 2 (imagine that!), only R5900 Linux user emulation.

And either the original qemu-mipsel can not launch the related output binaries as it says that it do not knows about R5900 model.

QEMU MIPS maintenance was a mess during the years 2018-2019, sadly. Maybe they’ve sorted things since then, maybe not. Anyway, I reverted to maintain my own R5900 QEMU to avoid the fuss.

Compiled your r5900 qemu. Launched that magick using it. Got these:

qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault

What would GDB say about this? Where, exactly, does it crash? Would you be able to inspect the call stack at the time of the segmentation fault?

[Arch91]

What would GDB say about this?

I have no a working static version of gdb (which could be launched further with qemu-mipsel user mode) because of the new glibc version - it prevents from doing that OR glibc should be recompiled with some additional flag. I tried to build a static gdb in the qemu's debian, but it is sooooo looong.... To do things faster I should try to build it with another one cross compiler based, for example, on musl but without fp.

There’s no QEMU system emulation of the PlayStation 2

GCC even for target mipsr5900el do not provides r5900 specific instructions (right?..), so there is no needence in the whole system of PS2 emulation. Yet. I thought that handling those triple MULTs would be enough for qemu-system-mipsel... but the patch is applied and the Instruction is Invalid :/

Where, exactly, does it crash? Would you be able to inspect the call stack at the time of the segmentation fault?

I prefer to be aware of involving deeper yet. Because I have no a cross compiler with fp I am sure in which. And that's because of those files with s.d/l.d opcodes...

[frno7]

I have no a working static version of gdb (which could be launched further with qemu-mipsel user mode)

A statically linked GDB isn’t necessary, if LD_LIBRARY_PATH is appropriately set up, I think. See also https://github.com/frno7/linux/pull/42#issuecomment-1032846839.

GCC even for target mipsr5900el do not provides r5900 specific instructions (right?..), so there is no needence in the whole system of PS2 emulation.

System emulation with QEMU would involve the processor (the R5900), various hardware peripherals (the IOP, the Graphics Synthesizer, and the rest of it), essentially the whole machine. User emulation, on the other hand, is essentially only a subset of the processor (the R5900) and the Linux kernel system calls. R5900 QMEU currently doesn’t emulate any PlayStation 2 hardware except the R5900 processor. GCC emits R5900-specific instructions, and those must be emulated in both cases.

Yet. I thought that handling those triple MULTs would be enough for qemu-system-mipsel...

Again, system emulation for the R5900 has never existed. We’ve only got R5900 user emulation. The Wikipedia page for QEMU explains its operating modes.

but the patch is applied and the Instruction is Invalid :/

MIPS QEMU maintenance was a mess at the time, unfortunately, and most if not all R5900 QEMU emulation was disabled and then dismantled in the official distribution, beginning with commit https://github.com/qemu/qemu/commit/823f2897bdd78185f3ba33292a25105ba8bad1b5. I don’t know why.

[Arch91]

Well... I was manually enumerately building a toolchain combos in the searching of the ability to compile a static gdb which uses hardware FP and which could be eventually launched in qemu-system-mipsel. And I failed with that task - that static hard-float'ed gdb is that way or another not working (it could be launchable but it freezes at the stage of loading the program to debug OR when the program is loaded, then "run" results to abort from gdb).

Another fact is that the static gdb will not be working with qemu user mode: https://stackoverflow.com/questions/68435791/warning-ptrace-function-not-implemented-during-startup-program-exited-with-cod

To use gdb in qemu right: https://qemu.readthedocs.io/en/latest/system/gdb.html

However, I'm not sure what kind of gdb should be used for that purpose - PCLinux'es, means not cross-compiled at all? And what should be used to launch the program to debug - qemu (user mode, r5900) or gdb->run ?..

To have gdb in qemu-system-mipsel, it have to be compiled natively which is sooo looong.... Using distcc could speed up the compilation processes, but to build the proper cross-compiler for MIPSel gentoo requires the proper and same toolchain the native compilator was built from.

Doubtful patch triggered doubtful workaround :/ That Zanella girl supposed that replacing everywhere s.d/l.d to swc1/lwc1 should do the trick saying that those FP registers are used to keep the machine state... Also she was noting that the linux target for r6000 was never thought-out... So I'm totally confused about which toolchain and flags to trust, which should be used to build a proper stable cross-compiler with hardware fp...

This is a NaN related binutils patch. I used it today as to apply when was trying to build uClibc-ng-1.0.40 with binutils-2.23.2, so it might be those NaNs flags were introduced in the first place.

[frno7]

Another fact is that the static gdb will not be working with qemu user mode:

https://wiki.qemu.org/Features/gdbstub seems to indicate what you want: QEMU support a gdbserver known as the gdbstub. With this you can debug programs in both linux-user and system emulation modes. I haven’t used this myself, though.

To use gdb in qemu right: https://qemu.readthedocs.io/en/latest/system/gdb.html

As previously mentioned, we don’t have R5900 system emulation (as of now), only R5900 user emulation.

So I'm totally confused about which toolchain and flags to trust, which should be used to build a proper stable cross-compiler with hardware fp...

Diving head first into libc is similar to attempting a triple pirouette in figure skating on the first day exercise. It’s a difficult proposition to pull off, no doubt. :-) I’ve updated #3 with a suggestion to begin with a small test suite, with parts in MIPS assembly, whereby FPU machine instructions can be tested without libc or GCC support.

[Arch91]

attempting a triple pirouette in figure skating on the first day exercise

Well... If before that first exercise, the person was good enough in ice-skating like a hockey player and he/she is strong and dexterous enough to jump and not to fall, then that attempt should be quite successful :D I agree with a suggestion of a small testsuites for searching a wrongly working opcodes, but if you ask me I think that the problem may be in gcc functions (such as libgcc's) which may not include an assembly inlines at all but generates an unsupported combo of opcodes. And that "generator" location will be hard to trace rather than to fix.

As previously mentioned, we don’t have R5900 system emulation (as of now), only R5900 user emulation.

Yes, you mentioned that several times, and I got that from the first one :) Maybe from the second one... :D But I surely did) Of course I mean that if I am testing with original qemu's system mode then I am testing the binaries which were built using the cross-compiler built for r6000 (with -mfix-r5900 gcc flag enabled by default).

A statically linked GDB isn’t necessary, if LD_LIBRARY_PATH is appropriately set up, I think.

I tried to launch a dynamically linked mipsel program using your r5900 qemu. Before to do that, a symlink to cross-compiler's ld.so has to be created in the host PC's lib directory. Particularly in my case I had to create a symlink here /lib/ld.so.1 to /usr/local/ps3/mipsel-unknown-linux-gnu/lib/ld.so.1 Next, qemu should be launched like this: LD_LIBRARY_PATH=/path/to/cross-compiler's_folder_before_lib_folder qemu-mipsel program_to_launch In my case, an example: LD_LIBRARY_PATH=/usr/local/ps3/mipsel-unknown-linux-gnu qemu-mipsel magick When done with the in-qemu tests, that symlink must be removed. for a safely purposes.

Now I can get my "lovely" SegFault in qemu's user mode without compiling everything static! :S Either without building a whole target-system for the real testing. I prefer a real testing, but at this stage of unresolved SegFault, I agree that that is very usable, rational.

Still, I did not debug anything. I do not getting qemu's gdbstub, either I have no a stable working native mipsel gdb.

[frno7]

Now I can get my "lovely" SegFault in qemu's user mode without compiling everything static! :S

Progress! Congratulations.

Either without building a whole target-system for the real testing. I prefer a real testing, but at this stage of unresolved SegFault, I agree that that is very usable, rational.

Indeed. Now with R5900 QEMU you’ve got a powerful tool at your disposal. :-)

Still, I did not debug anything. I do not getting qemu's gdbstub, either I have no a stable working native mipsel gdb.

I don’t know much about gdbstub either, but there appear to be some guides about it. Maybe you could ask the GDB mailing list about it?

[Arch91]

I don’t know much about gdbstub either

So am I... So going the old way) Finally I understood how to prepare the things for to be debugged with gdb and where to make those debugs themselves. Cross-compiler for r6000 target, based on uClibc (because glibc denies to build static binaries); uClibc-ng-1.0.40 patched with those edits suggested by Zanella girl. Also the gcc flag -mfix-r5900 enabled by default. gdb-7.12-6, just installed using apt-get install in mipsel debian. Statically compiled ImageMagick, got it's main static binary called "magick".

gdb magick
run import scr.jpg
Program received signal SIGSEGV, Segmentation fault.
0x008c30e4 in __divdf3 (x=0, y=4) at ../../../libgcc/config/hardfp.c:37
37  ../../../libgcc/config/hardfp.c: No such file or directory.

gcc-9.2.0/libgcc/config/hardfp.c - lines 36-37:

#elif defined (OP_div3)
TYPE FUNC (TYPE x, TYPE y) { return x / y; }

frno7, what'll do you say?

---

Well, I believe it is someway related to this: https://gcc.gnu.org/legacy-ml/gcc-patches/2014-02/msg00420.html Also it might be the file hardfp.c contains the functions which are not supported for the single float targets. I found out the quick workaround:

    # All MIPS targets provide a full set of FP routines.
    cpu_type=mips
    tmake_file="mips/t-mips"
-   if test "${libgcc_cv_mips_hard_float}" = yes; then
-       tmake_file="${tmake_file} t-hardfp-sfdf t-hardfp"
-   else
+#  if test "${libgcc_cv_mips_hard_float}" = yes; then
+#      tmake_file="${tmake_file} t-hardfp-sfdf t-hardfp"
+#  else
        tmake_file="${tmake_file} t-softfp-sfdf"
-   fi
+#  fi

Now I have a hard'n'single float'ed mipsr5900el cross-compiler abi o32 MIPS-III based on the glibc-2.32 with the Zanella girl's patch. I recompiled the target-system from scratch, everything seems alright, quite stable.

frno7, if you'll be minding about fixing the issue about using hard-fp for libgcc floating-point routines, I think you should take a look into a gcc files libgcc/config/t-hardfp, libgcc/config/t-hardfp-sfdf and libgcc/config/hardfp.c