Build environment for LOADLIN / embedded Linux version missing from the repository

[skrzyp]

Code of Conduct & Contributing Guidelines

• [X] I agree to follow the code of conduct and the contributing guidelines.

Have you checked that no other similar question(s) already exists?

• [X] I have searched and didn't find any similar issues.

Question

Most likely related to @grapeli beyond the standard development team scope.

So, I've seen there's a "LOADLIN version" of DOSBox-X on the website, which is basically a bundle of Linux kernel + initramfs containing the dosbox-x binary + LOADLIN + BAT file for convenience. Turns out that /etc/os-release file clearly states it was made from Buildroot, which is very nice. Actually, it's the quite handy and great way of distributing dosbox-x more than an experiment.

Turns out that I'm probably duplicating the efforts due to low discoverability, took me a few days to found out that your development is basically same as http://github.com/skrzyp/boot2dos - using exact same technology and exact same purpose :)

(Currently I was using regular DOSBox SVN builds as a placeholder, but eventually planned to move onto dosbox-x)

The problem is that I was looking for your sources / BR2_EXTERNAL environment for building the "LOADLIN version" and found nothing except prebuilt artifacts shared over and over. Aside from elaborate and philosophical licensing questions, I think it would be great to have that build environment on the table, maybe in separate repository with submodule to prevent additional noise, so I could contribute there.

My goals are pretty much same to these being represented by current state of "LOADLIN version", but I think it might be expanded to its own kind of representative solution for bootable DOSBox-X on old/abandoned/legacy/embedded/repurposed x86/x86_64 PCs. Especially with addition of more quality-of-life improvements (for example, automatic mounting of every found FAT32 devices), development (full CI builds, possibly) or UX (merging the kernel + initrd and reducing the size to make it a single file as a result to be placed everywhere and called from grub4dos, other loaders or maybe even as an EFISTUB).

Personally, I know buildroot environment pretty much, worked with it at $dayjob for ~2 years and while I don't really want to "take over" the current state I think there might be some added value in contribution.

Let me know about your thoughts, questions and decisions.

[grapeli]

The HX version is also prepared separately (an appropriate toolchain is needed) by @Wengier.

Similarly in this case.

My build was made purely as a curiosity to build a minimal Linux x86 version of DOSBox-X with the ability to run under DOSBox-X and QEMU. Nothing more. A thing of very little use. Out of curiosity, it can be run under https://copy.sh/v86/.

Why would someone run it on computers that already have Windows Xp, 7, 8, 8.1, 10, 11, any Linux, macOS, FreeBSD, OpenBSD installed? After all, DOSBox-X works on them without any problems. Much better functioning. I really don't know and I'm wondering what this is for.

LOADLIN is a bootloader, just like grub, syslinux and others, the only difference is that it loads linux directly from DOS.

For the most part, I prepare it by hand. Buildroot configuration. buildroot.config.txt

Busybox configuration. busybox.config.new.txt

Linux-4.19.x configuration. config-linux-4.19.x.txt

Dirty patch for dosbox-x. 0001-dirty-fix-for-dosbox-x-buildroot.patch.txt

I am building dosbox-x (cross compilation), copying necessary files for a mock initrd (rootfs) and using mksquashfs to prepare the final image.

[Wengier]

I agree that the LOADLIN package was mainly made to run inside DOSBox-X, as the HX package was not able to run inside DOSBox-X (at least at the time, see #1830). On the other hand, the HX version was not really prepared by me originally. I think @yksoft1 and @joncampbell123 made the initial HX-DOS build, and I simply improved the build script to make it self-contained as requested in #591. But compared with builds for other platforms, it is also a minimal build for running in DOS. You cannot expect more really (no advanced functions etc). I hope that it works for such users (who run it in DOS), but if one wants advanced functionalities they can certainly try DOSBox-X builds for other platforms or perhaps other DOSBox branches instead.

[Wengier]

Meanwhile, I think @skrzyp was looking for a Discord channel for DOSBox-X, and there is indeed one. Not many people are there though since DOSBox-X is apparently mostly GitHub based and the Discord channel is not publicly linked from the project website, but some people may find it useful for chatting since communications via Discord is real-time and thus may be more effectively. There is Discord app for smart phones and tablets too, which may notify you when there are new messages. The DOSBox-X Discord channel is available from:

https://discord.gg/5cnTmcsTpG

@skrzyp and @grapeli can both join too.

[grapeli]

I'm not interested in creating another live linux pretending to be a DOS system. I do not have enough experience for this.

@Wengier I don't use discord. I'm so sorry.

Building such a minimal system on your own, even for your own needs and based on Buildroot, is not difficult. If it is to run on relatively modern hardware without Xorg, Wayland should then use KMS/DRM (SDL2 only). Thanks to OpenGL, scaling will work, 3DFx will work and it will be an efficient solution. Using a VESA frambuffer (fbdev) is definitely an unsuitable solution on modern equipment.

There is a fork of SDL 1.2 with KMS/DRM (no opengl). https://github.com/OpenDingux/SDL/tree/fix-integer-scaled/src/video/kmsdrm https://github.com/libsdl-org/SDL-1.2/issues/817

[Wengier]

@grapeli The HX-DOS CI target has been recently added in the code, and I hope there is also a LOADLIN CI target so that things can be automated. If possible, can you post (or make) a working script (along with required files) for the LOADLIN package (perhaps in a repo like @skrzyp did above for boot2dos), so that a CI target can be made from it as well? Thanks!

P.S. I am personally certainly less familiar with Linux than you are, so I hope the build script can work out of the box so that the CI can be implemented more easily.

[grapeli]

@Wengier Unfortunately, I am not able to do this. I'm not a developer. I'm a regular user who experimented with running dosbox-x in dosbox-x.

Scripting it all and adding it to your automated CI infrastructure takes more experience. After that, it requires constant supervision and constant improvement. Not only is the dosbox-x code changing, buildroot is also constantly evolving and changing.

So far, I have done it completely differently. I have limited CPU resources (my hardware is a cheap 2010 laptop). I build the kernel by using the archlinux toolchain, because it shortens and simplifies the build (buildroot). The kernel can be compiled with a completely different toolchain (maybe with a small exception if you would later try to add an external module, building it against a different compiler).

curl -JOL https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.19.241.tar.xz
tar xf linux-4.19.241.tar.xz && cd linux-4.19.241
cp ../config-linux-4.19.x.txt .config
KBUILD_BUILD_USER=grapeli23 KBUILD_BUILD_HOST=myhost KBUILD_BUILD_VERSION=0 ARCH=x86 CC=gcc-10 HOSTCC=gcc-10 make -j3

I build dosbox-x with profiling and LTO. Using the bundled SDL1.x library.

The first step. CFLAGS/CXXFLAGS="...-O3 -march=i586 -mtune=generic -fprofile-arcs -ffat-lto-objects -flto=4..." (The sdl library is also profiled).

The second stage is collecting samples (under the qemu). I do this in one run, it requires manual intervention multiple times.

The third step is the final compilation using the collected samples. CFLAGS/CXXFLAGS=...-O3 -march=i586 -mtune=generic -fbranch-probabilities -fprofile-use -fprofile-correction -Wno-missing-profile -ffat-lto-objects -flto=4... This makes dosbox-x in dosbox-x much faster than a regular build. All of this is just for fun.

[grapeli]

@Wengier I'm downloading buildroot and building.

curl -JOL https://buildroot.org/downloads/buildroot-2022.02.1.tar.xz
tar xf buildroot-2022.02.1.tar.xz && cd buildroot-2022.02.1
cp ../buildroot.config.txt .config
cp ../busybox.config.new.txt package/busybox/busybox.config.new
make -j4

I build dosbox-x manually due to profiling using such a long command. From outside the buildroot directory.

./autogen.sh
sed -i "s|O2|O3|g" configure

and

PATH="/home/wojtek/src/buildroot-2022.02.1/output/host/bin:/home/wojtek/src/buildroot-2022.02.1/output/host/sbin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl" AR="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-ar" AS="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-as" LD="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-ld" NM="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-nm" CC="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-gcc" GCC="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-gcc" CPP="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-cpp" CXX="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-g++" RANLIB="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-ranlib" READELF="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-readelf" STRIP="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-strip" OBJCOPY="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-objcopy" OBJDUMP="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-objdump" AR_FOR_BUILD="/usr/bin/ar" AS_FOR_BUILD="/usr/bin/as" CC_FOR_BUILD="/usr/bin/gcc" GCC_FOR_BUILD="/usr/bin/gcc" CXX_FOR_BUILD="/usr/bin/g++" LD_FOR_BUILD="/usr/bin/ld" CPPFLAGS_FOR_BUILD="-I/home/wojtek/src/buildroot-2022.02.1/output/host/include" CFLAGS_FOR_BUILD="-O2 -I/home/wojtek/src/buildroot-2022.02.1/output/host/include" CXXFLAGS_FOR_BUILD="-O2 -I/home/wojtek/src/buildroot-2022.02.1/output/host/include" LDFLAGS_FOR_BUILD="-L/home/wojtek/src/buildroot-2022.02.1/output/host/lib -Wl,-rpath,/home/wojtek/src/buildroot-2022.02.1/output/host/lib" FCFLAGS_FOR_BUILD="" DEFAULT_ASSEMBLER="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-as" DEFAULT_LINKER="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/i586-buildroot-linux-musl-ld" CPPFLAGS="-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64" LDFLAGS="" FCFLAGS=" -O2 " FFLAGS=" -O2 " PKG_CONFIG="/home/wojtek/src/buildroot-2022.02.1/output/host/bin/pkg-config" STAGING_DIR="/home/wojtek/src/buildroot-2022.02.1/output/host/i586-buildroot-linux-musl/sysroot" INTLTOOL_PERL=/usr/bin/perl ac_cv_lbl_unaligned_fail=no ac_cv_func_mmap_fixed_mapped=yes ac_cv_func_memcmp_working=yes ac_cv_have_decl_malloc=yes gl_cv_func_malloc_0_nonnull=yes ac_cv_func_malloc_0_nonnull=yes ac_cv_func_calloc_0_nonnull=yes ac_cv_func_realloc_0_nonnull=yes lt_cv_sys_lib_search_path_spec="" ac_cv_c_bigendian=no gt_cv_func_gnugettext1_libc=yes gt_cv_func_gnugettext2_libc=yes CONFIG_SITE=/dev/null CFLAGS=".............." CXFLAGS="................" ./configure --target=i586-buildroot-linux-musl --host=i586-buildroot-linux-musl --build=x86_64-pc-linux-gnu --prefix=/usr --exec-prefix=/usr --sysconfdir=/etc --localstatedir=/var --program-prefix="" --enable-core-inline --disable-debug --prefix=/usr --enable-screenshots --disable-avcodec --disable-opengl --disable-alsatest --enable-sdl --disable-x11 --enable-freetype --enable-printer --enable-scaler-full-line

Oh first I go to the cd vs/sdl directory and build (using the same environment variables).

[Wengier]

@grapeli I know you are a user rather than developer, but I asked you because I had hoped a CI exists for it so that you don't need to do it manually any more, and I could try to create the CI myself if there are detailed build steps. But it looks like you may find it somewhat difficult to post the complete steps (such as the steps for collecting samples as you mentioned above), and of course in such case I also cannot make a CI target myself. The CIs are made in the hope to check that the code can be compiled in different platforms (if the code does not build, then of course they need to be fixed) and to make actual DOSBox-X binary packages (resembling the release packages), and while I created the CI targets, the workflows can be further improved of course (perhaps incrementally to fix issues etc), and indeed everyone can come to update them, not just me. And PR #3489 is a good example of CI improvements by other contributors. Apparently CIs are used in various other GitHub project too, not just DOSBox-X.

[Torinde]

other loaders or maybe even as an EFISTUB

Related discussion about UEFI DOS via DOSbox-X, libsdl-org/SDL/issues/8232