IMGMAKE FAT32 support

[rderooy]

Is your feature request related to a problem? Please describe. Now that there is support for FAT32 from the DOSBox-X shell, it should also be possible to use IMGMAKE to create FAT32 volumes.

Describe the solution you'd like If your creating a volume larger then 2GB, it should automatically use FAT32. imgmake hdd.img -t hd_4gig

In addition there should be a new flag to force FAT32 on smaller volumes imgmake hdd.img -t hd_520 -fat32

[Wengier]

Both IMGMAKE and IMGMOUNT commands need to be updated accordingly for FAT32 support, i.e. creating and auto-detect geometry for FAT32 drives.

[rderooy]

I was trying to understand what the minimum FAT32 size is supposed to be.

• Win98 FAT converter will refuse to convert a drive smaller then 512MB
• I did not test this, but supposedly the smallest drive that Windows will let you format as FAT32 is about 260MB.

But according to other other data I found online it was either

• 65,525 clusters * 512 bytes = 33,548,800 bytes (32,763 KB)
• 65,527 clusters * 512 bytes = 33,549,824 bytes (32,763.5 KB)

Also found some interesting details here. Apparently the FAT32 limits changed from Win95 OSR2 to Win95 OSR2.5 http://www.mdgx.com/secrets.htm#FAT32

It also mentions the undocumented FDISK /FPRMT and FORMAT /Z:x switch for creating smaller FAT32 volumes. http://www.mdgx.com/secrets.htm#FORMAT-Z

[joncampbell123]

@rderooy It's not just Windows, the mkdosfs tool here on Linux also refuses to make FAT32 volumes less than about 260MB, which is highly relevant because most UEFI BIOSes only support FAT32 for the EFI system partition.

In any case I suggest IMGMAKE should make a FAT12 drive if 32MB or less, FAT16 if 512MB or less, and FAT32 otherwise, if DOS version 7.10 or higher.

If DOS version 7.0 or lower, then the limit to create FAT32 should be 2GB (the limit of FAT16).

I also suggest a command line option to let the user force FAT12/FAT16/FAT32 within reasonable size limits.

That will require changes to the partition type code and default FAT entries. The code could use some cleanup and better decision making on how to arrange the FAT filesystem than the fixed code there is now.

[joncampbell123]

@rderooy I wrote some experimental FAT formatting tool for Linux some time back that might be a useful reference. At the very least when I used it to format various pen drives and external drives Windows XP, 7, and 10 CHKDSK were perfectly happy with the FAT filesystem it generated.

https://github.com/joncampbell123/datamtoolbox2/blob/master/datamtoolbox-v2/libmsfat/libmsfmt.c

[rderooy]

The Windows 98 FAT32 conversion tool (CVT.EXE) has a /MIN parameter to allow smaller volumes to be converted. In the Win98 Resource Kit manual it states:

[ /MIN ] This parameter overrides the minimum volume size logic that the converter uses to determine whether converting a volume to FAT32 is feasible. The conversion will take place even if the volume is too small to economically convert to FAT32. The logic the converter uses to determine if a volume is below the minimum size follows.

if (((FAT16_clusters sectors_per_cluster)/2) < 512 1024) ( do not convert volume)

[joncampbell123]

I just finished IMGMAKE FAT32 support, as well as plenty of code cleanup in the IMGMAKE tool.

So far, Windows 98 is happy with any FAT12, FAT16 and FAT32 image made by it.

IMGMAKE seems to have a 8GB limit, which I'll work on tomorrow to lift if I can.

EDIT: I just removed the limit. Windows 98 seems happy with FAT32 even if IMGMAKE is asked to generate a 80GB hard disk image.

[joncampbell123]

Ha, my hack to enable oversized FAT12 partitions works with Windows 98.

The pure MS-DOS kernel is perfectly fine with it, and SCANDISK.EXE is fine with it.

The graphical desktop on the other hand makes the drive read only, SCANDSKW.EXE and DEFRAG won't touch it.

[rderooy]

@joncampbell123 great job! I'm compiling now and will give it a go here. After that I will need to update the Windows install guide :-)

[rderooy]

SDL1 compile failure, but not with the FAT32 code.:

g++  -g -std=gnu++11  -O2 -msse  -Wall   -Wextra   -Wunused   -pedantic   -Wlogical-op   -Wsign-promo   -Wconversion-null   -Wno-deprecated-declarations   -Wno-implicit-fallthrough   -Wno-strict-aliasing   -Wno-missing-field-initializers   -Wno-format-zero-length   -Wno-address-of-packed-member   -Wno-int-to-void-pointer-cast  -I/home/robert/github/dosbox-x/vs2015/sdlnet/linux-host/include -I/home/robert/github/dosbox-x/vs2015/sdlnet/linux-host/include/SDL -D_XOPEN_SOURCE=700 -D_POSIX_C_SOURCE=200809L   -L/home/robert/github/dosbox-x/vs2015/sdlnet/linux-host/lib -o dosbox-x dosbox.o  debug/libdebug.a dos/libdos.a shell/libshell.a builtin/libbuiltin.a ints/libints.a misc/libmisc.a hardware/serialport/libserial.a hardware/parport/libparallel.a libs/porttalk/libporttalk.a gui/libgui.a libs/gui_tk/libgui_tk.a hardware/libhardware.a cpu/libcpu.a hardware/reSID/libresid.a fpu/libfpu.a gui/libgui.a output/liboutput.a hardware/mame/libmame.a libs/zmbv/libzmbv.a aviwriter/libaviwriter.a  xBRZ/libxbrz.a mt32/libmt32.a -lasound -lm -ldl -lpthread -L/home/robert/github/dosbox-x/vs2015/sdl/linux-host/lib -Wl,-rpath,/home/robert/github/dosbox-x/vs2015/sdl/linux-host/lib -lSDL -lX11 -lXext -lvga -lpthread -ltinfo -lfreetype  -lSDL_net -lz -lpng -lz -lpcap -lX11 -lXrandr -lxkbfile -lfluidsynth -lGL
/usr/bin/ld: cpu/libcpu.a(core_normal_8086.o): in function `CPU_Core8086_Normal_Run()':
/home/robert/github/dosbox-x/src/cpu/core_normal/prefix_none.h:111: undefined reference to `mustCompleteInstruction'
/usr/bin/ld: /home/robert/github/dosbox-x/src/cpu/core_normal/prefix_none.h:127: undefined reference to `mustCompleteInstruction'
/usr/bin/ld: /home/robert/github/dosbox-x/src/cpu/core_normal/prefix_none.h:143: undefined reference to `mustCompleteInstruction'
/usr/bin/ld: /home/robert/github/dosbox-x/src/cpu/core_normal/prefix_none.h:159: undefined reference to `mustCompleteInstruction'
/usr/bin/ld: /home/robert/github/dosbox-x/src/cpu/core_normal/prefix_none.h:291: undefined reference to `mustCompleteInstruction'
/usr/bin/ld: cpu/libcpu.a(core_normal_8086.o):/home/robert/github/dosbox-x/src/cpu/core_normal/prefix_none.h:294: more undefined references to `mustCompleteInstruction' follow
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:470: dosbox-x] Error 1
make[3]: Leaving directory '/home/robert/github/dosbox-x/src'
make[2]: *** [Makefile:530: all-recursive] Error 1
make[2]: Leaving directory '/home/robert/github/dosbox-x/src'
make[1]: *** [Makefile:390: all-recursive] Error 1
make[1]: Leaving directory '/home/robert/github/dosbox-x'
make: *** [Makefile:327: all] Error 2

[joncampbell123]

Never tested the code against non-debugger-enabled builds, fixing now.

[joncampbell123]

Try the latest commit. With and without the debugger.

[Wengier]

@joncampbell123 Thanks! I tried to create a FAT32 disk image using IMGMAKE (with "-fat 32"), and it does work, which is great.

However, I also tried to create a regular 1.44MB floppy disk, and it no longer works.

Z:\>imgmake fd.img -t fd_1440
Creating an image file with 80 cylinders, 2 heads and 18 sectors

Z:\>imgmount a fd.img
Cannot create drive from file.

Tried with imgmake fd.img -t fd_1440 -fat 12, same result.

Also tried with imgmake fd.img -t fd_1440 -fat 16 and imgmake fd.img -t fd_1440 -fat 32, and there is no error message shown even though I believe FAT16 and FAT32 do not support disk image with such a small size.

[joncampbell123]

Good catch! I wrote the code last night focused only on hard disk images 4MB or larger.

If a disk image's size is too small for a filesystem, there is code to catch that and report an error. It's tied to the cluster count, which according to Microsoft's FAT32 whitepaper is crucial in deciding whether a filesystem is FAT12, FAT16, or FAT32.

There is also code to deal with a format with too many clusters compared to the format you want as well, but most MS-DOS implementations are happy with it if the total sector fields are smaller than the partition to make it work. In that case you get a partition that is far larger than the actual filesystem.

[joncampbell123]

Please try the latest commit. Non-hdd images failed to set a crucial variable reflecting the volume size.

[Wengier]

@joncampbell123 Yes, creating a floppy image with a command like imgmake fd.img -t fd_1440 now works.

Also, I noticed that the command imgmake fd.img -t fd_1440 -fat 16 now triggers the error message "Error: Generated filesystem has too few clusters given the parameters". Yet the disk image named fd.img is generated anyway, which is not mountable by the imgmount command. Perhaps IMGMAKE should automatically delete the file in such case?

[joncampbell123]

I agree, it should delete the file.

A floppy disk has too few sectors to ever support a FAT16 filesystem. The sector to cluster math cannot make a value large enough for FAT16.

[rderooy]

SDL1 compile succeeded (SDL2 is still broken, but that is the SDLNet issue).

Will try a few things and report any problems.

[rderooy]

I tried the following;

imgmake hdd.img -t hd -size 700000

And it worked. Although I still need to try to install Win98 and boot from it :-)

The only thing is the message, both on the dos prompt and in the log.

Creating an image file with 1023 cylinders, 16 heads and 63 sectors

Which obviously is not the case, as a 700GB image will not fit in CHS.

[joncampbell123]

Beyond a certain point, the C/H/S values are meaningless because the image is too large to represent the actual size. By that point (when hard drives were about 2-4GB) most systems went to use LBA anyway and ignored the C/H/S geometry stuff.

[rderooy]

Yeah I know, just the message can be a bit confusing.

Trying to install Win98SE to my 700GB drive now. xcopy is VERY slow, it seems like it may be trying to check disk space for every file it copies. Even with Turbo mode on, it takes a long time to copy even a single file.

[joncampbell123]

DOSBox-X inherits DOSBox SVN's very inefficient FAT driver, yes.

[rderooy]

I forgot, the Win98 IDE driver supposedly has issues with volumes greater than 128GB, so my 700GB install will fail in the end. Since it is still crawling along with xcopy I will kill it and create a smaller volume.

[rderooy]

Not sure if this is related to the Win98 IDE driver issue, but my install to a 128GB volume failed with a SU0013.

Log shows these messages:

LOG: 1178989898 ERROR DOSMISC:DOS:IOCTL Call  5 unhandled
LOG: 1178996097 ERROR DOSMISC:DOS:INT 2F Unhandled call AX=5600
LOG: 1179030777 ERROR DOSMISC:DOS:IOCTL Call  5 unhandled
LOG: 1179036594 ERROR DOSMISC:DOS:INT 2F Unhandled call AX=5600

edit sorry, not after a reboot, but after the first scan of the disk after running setup.exe

[joncampbell123]

Windows 98 and FAT32 images beyond a certain size (64GB?) seem to trigger memory overruns and problems with SETUP. Not sure why.

[rderooy]

Indeed. booting from a msdos71 floppy and running fdisk against the hdd shows it as a 62464 MB volume using the entire drive, while it is actually a 128GB volume.

[joncampbell123]

:laughing: I've heard about this... FDISK.EXE in Windows 98 FE is said to have a bug where it cannot properly represent disk capacities larger than 64GB (65535MB?). Probably used an unsigned int (16-bit) somewhere. Do you have a later updated version of FDISK.EXE? Perhaps from Windows 98 SE?

[Wengier]

As mentioned in the DOSBox-X Wiki page (https://github.com/joncampbell123/dosbox-x/wiki/Guide%3ADOS-Installation-in-DOSBox%E2%80%90X), FDISK from MS-DOS 7.1/Windows 98 requires patch to support HDD size greater than 64GB.

[rderooy]

I just grabbed what should be a Win98SE bootdisk from WinWorld, and it did not change anything.

In any case, according philscomputerlab the issue with fdisk is only a reporting problem due to it being a 16bit application. The maximum supported bootable drive that Win98 supports with 28 bit LBA is 137438953472 bytes (= 128GiB). https://www.philscomputerlab.com/windows-98-maximum-hard-drive-capacity.html#:~:text=127%20GB%20%2F%20128%20GB%20%2F%20137,for%20a%20bootable%20hard%20drive.

[Wengier]

@rderooy FDISK from Windows 98 SE does not fix the 64GB problem. There is however an official patch from Microsoft to fix the 64GB problem of FDISK (it is not because of being a 16-bit application).

[Wengier]

The original Microsoft page is now gone, but it can still be accessed from:

https://web.archive.org/web/20170610205053/https://support.microsoft.com/en-us/help/263044/fdisk-does-not-recognize-full-size-of-hard-disks-larger-than-64-gb

[joncampbell123]

Ah, Microsoft's corporate amnesia strikes again. Just like in the late 1990s when they willfully forgot anything about the 16-bit Windows world despite being the foundation of Windows 9x/ME.

[joncampbell123]

I like how the page at your link actively refreshes itself to make sure it's not readable on the Archive. Gotta hide that article!

[Wengier]

@joncampbell123 Interesting. The first time I opened that link on the Archive it did work, but seems to refresh itself later on. Let me see if there is a better link that does not have this problem.

[rderooy]

I get the instant refresh also. I just see it a fraction of a second and it's gone.

[rderooy]

My google-fu still works https://web.archive.org/web/20000817184504/http://support.microsoft.com/support/kb/articles/Q263/0/44.ASP

[joncampbell123]

There you go! Back when web pages were web pages that didn't need 20 frameworks and 1MB of fiddly JavaScript to exist. :laughing: :+1:

[Wengier]

@joncampbell123 and @rderooy As a backup you can also try this link (from archive.is instead of archive.org): https://archive.is/eVLi

[joncampbell123]

@Wengier That link is fine, if you're willing to use Firefox's web developer tools to actively delete that popup sitting right on top of the article.

[joncampbell123]

There's also another one about Windows 95 and a 32GB limit. If you do the math, 32GB is 0x40000000 sectors and there are "major architectural changes required" to lift the limit. Did someone in Microsoft use the upper 2 bits of the sector number for flags? :laughing:

EDIT: Also noticed that if you consider 32GB as a count of MB, 32GB is 32768MB. So maybe FDISK or some Windows 95 component used a signed 16-bit integer somewhere? But then, if this is anything prior to OSR2, the DOS kernel doesn't support the INT 13h extensions and cannot go beyond 4-8GB anyway (limited by BIOS).

[rderooy]

Found a newer link with the fixes listed: https://web.archive.org/web/20011119000442/http://support.microsoft.com:80/support/kb/articles/q263/0/44.asp

[Wengier]

@joncampbell123 The above link found by @rderooy is indeed better. archive.is seems to have issues handling website popups.

[rderooy]

Calling it a day, will try to do some more testing tomorrow.

In any case, the fixed fdisk can be found here: https://www.eastbuttress.com/windows-98-microsoft-patches

[rderooy]

Not sure if this will cause problems, but it always appears as though the partition was formatted by MSDOS5.0, even for FAT32 volumes.

Also, with these changes to IMGMAKE, perhaps remove or disable the boot menu it writes to the MBR? I don't think it is needed, and it just confuses people.

[rderooy]

I checked it on a Win98 formatted volume, and it contains instead "MSWIN4.1"

[rderooy]

Trying to install Win98SE on a 32768MB volume throws a SU0650 Out of Memory error. DOSBox-X is configured with 64MB.

Looking online I find some claims that this can be caused by a too-small cluster size for FAT32, which eats too much memory.

Default cluster sizes for Win98: http://www.helpwithwindows.com/windows98/fat32.html

[joncampbell123]

@rderooy That's interesting they choose multiples of 4KB. Perhaps Windows 98 ties that into the swap/paging system. Will update.

[joncampbell123]

The latest commit picks FAT32 cluster sizes in 4KB increments as documented.

[rderooy]

The same install on a 32GB volume now worked with the latest update.

[sikthehedgehog]

Those are powers of two, not multiples of 4KB :P (I mean, technically they are a subset of those multiples but that's not the point here). Admittedly I'd advise against smaller clusters anyway.

I can see how 4KB could be tied to paging tho (since that's what Protected Mode uses).

[rderooy]

Install to a 64GB volume also worked.