mikaku
commented
1 year ago An initrd of 1700K or larger will push low kernel memory over the 4MB mark, but the memory over 4MB is not mapped by setup_minmem so the kernel crashes when trying to access it.
Yes, this is because the kernel is initially setup to accept only an initrd image of 1.44MB (a floppy disk). If anyone wants a bigger initrd image, then it will have to modify the setup_minmem() function, as you well said. I'm agree this is uncomfortable.
At least #1 should be fixed. This can be done by determining how big an initrd can be accommodated and checking that it can fit on startup. If it cannot fit then display an error message.
Agreed.
In my PR I have stopped using ramdisksize to control the size of the initrd ram drive. Using the ramdisksize parameter for this purpose is unnecessary and can lead to either an error (if too small) or wasted memory (if too large). Instead, the ram drive is sized to match the size of the provided initrd file system image. This allows a small initrd to be combined with a large ram drive (or the reverse). If ramdisksize controlled the initrd size then ramdisksize could not exceed 120MB as previously explained, and so all ram drives would be need to be limited to 120MB.
I can't think of a reason to give your initrd ram drive a size other than the size of the initrd file system anyway. In general, requiring all ram drives, including initrd, to be the same size is inflexible and while this change does not resolve that completely, it is a significant improvement, in my opinion.
I totally agree on this.
The forthcoming PR will allow mapping the initrd ram disk into a much larger memory area. However, at some point the initrd module may bump into another problem: it will start to conflict with memory mapped by user processes. User processes typically start their mapping at 0x08048000. If the initrd is large enough, its mapping will overlap process memory mappings. So, without changing the kernel initrd memory mapping design, an initrd must be limited to approximately 120MB. But this is still much better than 1.6MB.
Actually virtual memory solves this problem. You are able to set a RAMdisk drive of 512MB without any problem, as long as there is enough physical memory, of course. This is because the memory used for RAMdisk drives is mapped in kernel space, and so there is no conflict at all with the virtual memory used by the user-space applications. In fact, this is one of the reasons I removed the option RAMDISK_MAXSIZE recently.
See the following example:
$ qemu-system-i386 \
-cdrom FiwixOS-3.2-i386.iso \
-kernel fiwix \
-append "ro root=/dev/hdc rootfstype=iso9660 ramdisksize=512000" \
-m 1024This screen shot shows the kernel messages during the boot up:
As you can see the 500MB RAMdisk drive is mapped in 0xc034c000:
ram0 0xc034c000-0xdf74c000 RAMdisk of 512000KB size, 1KB blocksizeand the system boots finely. But of course, there will be only 500MB (aprox.) left for the user:
@rick-masters: In all, there are parts of this PR that I agree but there are others that might be unnecessary. I think we could make a little rewrite of this PR removing those unnecessary parts.
If you want me to modify your PR let me say you that I have never modified the code of a PR before merging it. So I don't know if I can edit it here on GitHub or I need to do something locally. I'm a bit lost here, sorry.
rick-masters
An initrd of 1700K or larger will push low kernel memory over the 4MB mark, but the memory over 4MB is not mapped by
setup_minmemso the kernel crashes when trying to access it.Details on how to reproduce this are described later.
This Issue can be divided into two separate issues:
At least
#1should be fixed. This can be done by determining how big an initrd can be accommodated and checking that it can fit on startup. If it cannot fit then display an error message. Currently it just hangs or seg faults.For
#2, currently Fiwix supports an initrd large enough to boot the kernel and mount a root partition. So, opinions may vary on how important it is to support a larger initrd drive. However, the Fiwix documentation indocs/kexec.txtseems to anticipate a 48MB initrd so it seems like larger drives were intended to be supported. With a modest number of changes a larger initrd can be supported.The following explains the cause of the problem.
Very early in the boot process, the kernel calls
setup_minmemto map enough memory to boot the kernel. Later, the kernel places critical data structures such as page tables in low memory, after the kernel image and multiboot modules. And importantly, the stack (%esp) is set to a location after the multiboot modules: https://github.com/mikaku/Fiwix/blob/e4c1e7ef94f400d21227dbab4c0dca0da55baa33/kernel/boot.S#L140-L146However, if an initrd module is large enough, it will push the stack pages beyond the 4MB of low mapped memory and the kernel will crash when the first value is pushed on the stack here: https://github.com/mikaku/Fiwix/blob/e4c1e7ef94f400d21227dbab4c0dca0da55baa33/kernel/boot.S#L146-L148
Even if the stack fits within the first 4MB, the page tables which are allocated after the stack may not fit in 4MB and so the kernel may crash later when accessing page table memory in
map_kaddr.A proper solution seems to require mapping more low memory in the boot process to fit the multiboot modules + page_tables + kernel data structures, etc. But how much memory must be mapped is not calculated exactly until much later in the boot process (by
mem_init). But we cannot wait until this late stage to map the stack page because the stack is used earlier to pass local variables tostart_kernel.Perhaps the amount of low memory to be mapped could be estimated and mapped earlier in the boot (perhaps in setup_minmem). But consider that setup_minmem would not have access to
kstatwhich holds the size of physical memory because that structure has not been populated yet. So calculating the size of the page tables would be difficult. There might be a way to make this work but it would be difficult and would require ongoing maintenance when the size of kernel data structures change.For these reasons, I am proposing a different idea: simply leave the stack where it was initially set on boot, below the kernel rather than above where it would take signficant work to map it properly. Then, the boot process can stay the same for the most part. After the kernel starts and determines how much low memory needs to be mapped (in the
mem_initfunction), the kernel can map additional memory pages as needed in the normal page tables for the initrd module and kernel data structures. Note that my PR only reserves one stack page because the kernel doesn't appear to need more than that but more pages could easily be reserved if necessary.The forthcoming PR will allow mapping the initrd ram disk into a much larger memory area. However, at some point the initrd module may bump into another problem: it will start to conflict with memory mapped by user processes. User processes typically start their mapping at 0x08048000. If the initrd is large enough, its mapping will overlap process memory mappings. So, without changing the kernel initrd memory mapping design, an initrd must be limited to approximately 120MB. But this is still much better than 1.6MB.
The role of ramdisksize= kernel parameter
In my PR I have stopped using ramdisksize to control the size of the initrd ram drive. Using the ramdisksize parameter for this purpose is unnecessary and can lead to either an error (if too small) or wasted memory (if too large). Instead, the ram drive is sized to match the size of the provided initrd file system image. This allows a small initrd to be combined with a large ram drive (or the reverse). If ramdisksize controlled the initrd size then ramdisksize could not exceed 120MB as previously explained, and so all ram drives would be need to be limited to 120MB.
I can't think of a reason to give your initrd ram drive a size other than the size of the initrd file system anyway. In general, requiring all ram drives, including initrd, to be the same size is inflexible and while this change does not resolve that completely, it is a significant improvement, in my opinion.
Notes on Reproducing the Issue and Testing the PR
How to reproduce:
To simplify testing, I have created initrd.img files of various sizes that can be booted standalone with qemu. In other words, they do not require a hard drive.
I have started with an initrd.img originating from
fiwix.org/downloads.html. There is an initrd.img inside the FiwixOS-3.2-initrd-i386.img file system. However, this initrd.img has an fstab that mounts root from a hard drive. So, I modify the initrd /etc/fstab file to mount root from /dev/ram0.Here is a script I used to extract initrd.img from FiwixOS-3.2-initrd-i386.img:
Here is a script I used to create initrd-${SIZE}.img:
Example usage:
Note these scripts use
sudoso ifsudois not setup properly then you may need to run the script as root usingsuor another method.