search

dosemu2 / fdpp

FreeDOS plus-plus, 64bit DOS
GNU General Public License v3.0
198 stars 18 forks source link

Installed block device unavailable under FDPP #85

Closed andrewbird closed 5 years ago

andrewbird commented 5 years ago

After installing a block device driver with device=emudsk.sys the drive is unavailable under FDPP, but is fine under FreeDOS

$ cat test-imagedir/dosemu.conf
$_lpt1 = ""
$_hdimage = "dXXXXs/c:hdtype1 +1"
$_floppy_a = ""

Current FDPP

Based on FreeDOS sources (C) Pasquale J. Villani and The FreeDOS Project.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

C: HD1, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=    10 MB
D: HD2, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
E: HD3, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
F: HD4, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
EMUFS host file and print access available
Dosemu EMUDSK installed as drive G:
Process 0 starting: C:\command.com /P /E:256

FreeCom version 0.84-pre2 XMS_Swap [Aug 28 2006 00:29:00]
C:\>prompt $P$G
C:\>path c:\bin;c:\gnu;c:\dosemu
C:\>unix -s DOSEMU_VERSION
UNIX: option -s is deprecated, use system.com instead
C:\>unix -e
UNIX: option -e is deprecated, use system.com instead
C:\>g:
Invalid drive G:.

You can see that the driver is loaded as G:

dosdebug> devs
dosdebug> 
DOS Devices

0000:0dd8 Char 'NUL     '
  Attributes: 0x8004 (Char, NULDEV)
  Routines: Strategy(0000:0e78), Interrupt(0000:0e7d)

02a8:0000 Block (1 Units)
  Attributes: 0x0000 (Block)
  Routines: Strategy(02a8:006b), Interrupt(02a8:0084)

029f:0000 Char 'EMUFS$  '
  Attributes: 0x8000 (Char)
  Routines: Strategy(029f:0036), Interrupt(029f:0041)

<snip>

Device driver was loaded okay and remains resident in memory

dosdebug> mcbs
dosdebug> 

ADDR(LOW) PARAS  OWNER
0290:0000 0x04db [DOS]
  => ADDR      PARAS TYPE USAGE
     0291:0000 0x000c [F] Files
     029e:0000 0x0008 [D] Driver (EMUFS)
     02a7:0000 0x000d [D] Driver (EMUDSK)
     02b5:0000 0x0004 [E] Driver Extension
     02ba:0000 0x0020 [B] Buffers
     02db:0000 0x029e [B] Buffers
     057a:0000 0x00cf [F] Files
     064a:0000 0x008f [L] CDS Array
     06da:0000 0x0080 [S] Stacks
     075b:0000 0x0010 [B] Buffers
076c:0000 ------ [LINK]
0817:0000 0x0006 [FREE]
081e:0000 0x12c1 [COMMAND]
1ae0:0000 0x7e63 [FREE]
9944:0000 0x0619 [COMMAND]
9f5e:0000 0x0090 [COMMAND]
9fef:0000 0x0010 [COMMAND] (END)

CDS entry seems to have the flags zeroed

064b:0210 47 3A 5C 00 00 00 00 00 00 00 00 00 00 00 00 00  G:\.............
064b:0220 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
064b:0230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
064b:0240 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
064b:0250 00 00 00 00 00 00 00 00 00 FF FF FF FF FF FF 02  ..........
064b:0260 00 00 00 00 00 00 00 00

And DPB for drive G: is missing

Now FreeDOS

$ mv test-imagedir/dXXXXs/c/fdppconf.sys test-imagedir/dXXXXs/c/config.sys
$ cp /tmp/kernel.sys test-imagedir/dXXXXs/c/.
Ask for help in mail list: linux-msdos@vger.kernel.org

FreeDOS kernel build 2036 cvs [version Aug 18 2006 compiled Aug 18 2006]
Kernel compatibility 7.10 - WATCOMC - 80386 CPU required - FAT32 support

(C) Copyright 1995-2006 Pasquale J. Villani and The FreeDOS Project.
All Rights Reserved. This is free software and comes with ABSOLUTELY NO
WARRANTY; you can redistribute it and/or modify it under the terms of the
GNU General Public License as published by the Free Software Foundation;
either version 2, or (at your option) any later version.

C: HD1, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=    10 MB
D: HD2, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
E: HD3, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
F: HD4, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
EMUFS host file and print access available
Dosemu EMUDSK installed as drive G:

FreeCom version 0.84-pre2 XMS_Swap [Aug 28 2006 00:29:00]
C:\>prompt $P$G
C:\>path c:\bin;c:\gnu;c:\dosemu
C:\>unix -s DOSEMU_VERSION
UNIX: option -s is deprecated, use system.com instead
C:\>unix -e
UNIX: option -e is deprecated, use system.com instead
C:\>g:
G:\>copy c:\autoexec.bat
c:\autoexec.bat => .\autoexec.bat
G:\>dir
 Volume in drive G has no label
 Directory of G:\

AUTOEXEC BAT            76  06-29-19  1:00a
         1 file(s)             76 bytes
         0 dir(s)     134,066,176 bytes free
G:\>
stsp commented 5 years ago

Would be cool if you can debug this a bit...

andrewbird commented 5 years ago

For reference the block driver is at https://github.com/andrewbird/dosemu2/tree/emudsk01-sys, but I expect the same thing to occur with other block drivers.

Yes, I can have a look, do you an idea where to start?

stsp commented 5 years ago

Yes, I can have a look, do you an idea where to start?

In init_device(), update_dcb() and all that.

andrewbird commented 5 years ago

Please remind me what I need to do to use symbols in FDPP? I'm doing

(gdb) p * (dhp*) &lowmem_base[(dhp.ptr.seg<<4)+dhp.ptr.off]
A syntax error in expression, near `) &lowmem_base[(dhp.ptr.seg<<4)+dhp.ptr.off]'.

I think there's some knob in FDPP build I need to tweak?

stsp commented 5 years ago

EXTRA_DEBUG=1

andrewbird commented 5 years ago

Thanks, that's great.

stsp commented 5 years ago

Interesting driver btw. But I hate adding anything that is not going to be in the default config/autoexec. It will rot.

andrewbird commented 5 years ago

Yes, I'd hoped the driver would be faster than MFS, but as it's not there's no point in adding it.

stsp commented 5 years ago

Any progress?

andrewbird commented 5 years ago

I got distracted, will have another look tomorrow.

stsp commented 5 years ago

Could you please attach pre-built binary?

andrewbird commented 5 years ago

I could, but it won't load without its dosemu doshelper routine in place. You'd be better off checking out my emudsk01-sys branch.

stsp commented 5 years ago

OK then, waiting for your another look tomorrow. :) Its too late here already anyway.

andrewbird commented 5 years ago

This can be reproduced using devel, with the xmsdsk.exe loaded from fdppconfig.sys as a device

lastdrive=Z                                                                     
device=dosemu\emufs.sys /tmp                                                    
device=dosemu\ems.sys                                                           
device=xmsdsk.exe 8000

FDPP

C: HD1, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=    10 MB
D: HD2, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
E: HD3, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
F: HD4, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
EMUFS host file and print access available
dosemu XMS 3.0 & UMB support enabled
dosemu EMS driver rev 0.8 installed.

*** XMS RAMdisk v1.9I (FU - 08/98): Installed as drive G:.

Process 0 starting: C:\command.com /P /E:256

FreeCom version 0.84-pre2 XMS_Swap [Aug 28 2006 00:29:00]
C:\>prompt $P$G
C:\>path c:\bin;c:\gnu;c:\dosemu
C:\>unix -s DOSEMU_VERSION
UNIX: option -s is deprecated, use system.com instead
C:\>unix -e
UNIX: option -e is deprecated, use system.com instead
C:\>dir g:
File not found. - 'g:'
C:\>g:
Invalid drive G:.
C:\>

FreeDOS

(C) Copyright 1995-2006 Pasquale J. Villani and The FreeDOS Project.
All Rights Reserved. This is free software and comes with ABSOLUTELY NO
WARRANTY; you can redistribute it and/or modify it under the terms of the
GNU General Public License as published by the Free Software Foundation;
either version 2, or (at your option) any later version.
C: HD1, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=    10 MB
D: HD2, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
E: HD3, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
F: HD4, Pri[ 1], CHS=    0-1-1, start=     0 MB, size=  2000 MB
EMUFS host file and print access available
dosemu XMS 3.0 & UMB support enabled
dosemu EMS driver rev 0.8 installed.

*** XMS RAMdisk v1.9I (FU - 08/98): Installed as drive G:.

FreeCom version 0.84-pre2 XMS_Swap [Aug 28 2006 00:29:00]
C:\>prompt $P$G
C:\>path c:\bin;c:\gnu;c:\dosemu
C:\>unix -s DOSEMU_VERSION
UNIX: option -s is deprecated, use system.com instead
C:\>unix -e
UNIX: option -e is deprecated, use system.com instead
C:\>g:
G:\>
stsp commented 5 years ago

Are you looking into that yourself? :)

andrewbird commented 5 years ago

To be honest I'm finding it difficult and I'm about to run out of time for today.

stsp commented 5 years ago

Yeah, sometimes I am asking myself if debugging fdpp with gdb is any easier than debugging freedos with dosdebug. Though as I have never done the later, I don't know the answer.

andrewbird commented 5 years ago

I see it trying to add the new device to the DPBs, but for some reason it doesn't appear on the end (this is before the CDS and DPB relocation to final location at the end of config.sys processing).

Does this line make sense in an FDPP world? https://github.com/stsp/fdpp/blob/61dc2e70bf857c96365393c58246968765ec3987/kernel/main.c#L559

stsp commented 5 years ago

It should, there are lots of ptr arithmetic around.

andrewbird commented 5 years ago

I see this

(gdb) p/x  LoL->_CDSp[0].cdsDpb
$32 = {ptr = {off = 0x19e8, seg = 0xd9}}
(gdb) p/x  LoL->_CDSp[1].cdsDpb
$33 = {ptr = {off = 0x1a25, seg = 0xd9}}
(gdb) p/x  LoL->_CDSp[2].cdsDpb
$34 = {ptr = {off = 0x1a62, seg = 0xd9}}
(gdb) p/x  LoL->_CDSp[3].cdsDpb
$35 = {ptr = {off = 0x1a9f, seg = 0xd9}}
(gdb) p/x  LoL->_CDSp[4].cdsDpb
$36 = {ptr = {off = 0x1adc, seg = 0xd9}}
(gdb) p/x  LoL->_CDSp[5].cdsDpb
$37 = {ptr = {off = 0x1b19, seg = 0xd9}}
(gdb) p/x  LoL->_CDSp[6].cdsDpb
$38 = {ptr = {off = 0x0, seg = 0x2d4}}

Then with my tweaked dosdebug

dosdebug> dpbs 00d9:1b19
dosdebug> 
DPBs (compiled for DOS v4+ format)

00D9:1B19 (F:)
  driver unit: 5
  bytes_per_sect = 0x0
  last_sec_in_clust = 0x0
  sec_shift = 0x0
  reserv_secs = 0x0
  num_fats = 0x0
  root_ents = 0x0
  data_start = 0x0
  max_clu = 0x0
  sects_per_fat = 0x0
  first_dir_off = 0x0
  device driver = 0070:0618
  media_id = 0x0
  accessed = 0xff
  next_DPB = FFFF:FFFF
  first_free_clu = 0x0
  fre_clusts = 0x0

dosdebug> dpbs 02d4:0000
dosdebug> 
DPBs (compiled for DOS v4+ format)

02D4:0000 (G:)
  driver unit: 0
  bytes_per_sect = 0x0
  last_sec_in_clust = 0x0
  sec_shift = 0x0
  reserv_secs = 0x0
  num_fats = 0x0
  root_ents = 0x0
  data_start = 0x0
  max_clu = 0x0
  sects_per_fat = 0x0
  first_dir_off = 0x0
  device driver = 02B2:0000
  media_id = 0x0
  accessed = 0xff
  next_DPB = FFFF:FFFF
  first_free_clu = 0x0
  fre_clusts = 0x0

See that the next pointer of F: is still 0xfffffff

andrewbird commented 5 years ago

BTW is there any way to just service dosdebug requests when stopped in gdb, as I find myself having to queue up a dosdebug command, then 'next' in gdb until it's been serviced?

andrewbird commented 5 years ago

I have to stop on this for today now, so feel free to fix it :smile:. I've pushed my dosdebug tweak in https://github.com/andrewbird/dosemu2/tree/dosdebug-39 if you need it.

stsp commented 5 years ago

So if you inspect _dpb->dpb_next right after it was changed, do you still see that? You can do that with the lowmem_base printing trick.

stsp commented 5 years ago

Thanks for pointing to this place, the bug was just 5 lines above. :) You saved me a debugging time. Please see how your driver works. xmsdsk still doesn't, so you can continue this quest. :)

andrewbird commented 5 years ago

That fix worked well, early dpb chain looks good now

00D9:1B19 (F:)
  driver unit: 5
  bytes_per_sect = 0x0
  last_sec_in_clust = 0x0
  sec_shift = 0x0
  reserv_secs = 0x0
  num_fats = 0x0
  root_ents = 0x0
  data_start = 0x0
  max_clu = 0x0
  sects_per_fat = 0x0
  first_dir_off = 0x0
  device driver = 0070:0618
  media_id = 0x0
  accessed = 0xff
  next_DPB = 02D4:0000
  first_free_clu = 0x0
  fre_clusts = 0x0

02D4:0000 (G:)
  driver unit: 0
  bytes_per_sect = 0x0
  last_sec_in_clust = 0x0
  sec_shift = 0x0
  reserv_secs = 0x0
  num_fats = 0x0
  root_ents = 0x0
  data_start = 0x0
  max_clu = 0x0
  sects_per_fat = 0x0
  first_dir_off = 0x0
  device driver = 02B2:0000
  media_id = 0x0
  accessed = 0xff
  next_DPB = FFFF:FFFF
  first_free_clu = 0x0
  fre_clusts = 0x0

I guess that's uncovered something else as XMSDISK installation is now hanging, is that what you see?

stsp commented 5 years ago

I see the crash. Does your driver now work?

andrewbird commented 5 years ago

I just checked,yes it loads and G: is functional.

G:\>c:\iostoned

Wait - IOSTONE is performing disk IO with varying blocksizes.

Total elapsed time is 20 sec.
Files were opened 9504 times. 15579 kb was read, and 7461 kb written.
Average read/write troughput was 1152 kb/sec, including open/close overhead.

This machine benchmarks at 100000 iostones/sec.

G:\>c:\iostoned

Wait - IOSTONE is performing disk IO with varying blocksizes.

Total elapsed time is 20 sec.
Files were opened 9504 times. 15579 kb was read, and 7461 kb written.
Average read/write troughput was 1152 kb/sec, including open/close overhead.

This machine benchmarks at 100000 iostones/sec.

G:\>

I think we can close this ticket now, thanks.

BTW XMSDSK crashes both FreeDOS and FDPP (ISTR MS-DOS 6.22 is okay) when loaded as an exe from the command line, so there's probably some unrelated incompatibility in there.

stsp commented 5 years ago

From the command line? But we load it from device= line, no?

andrewbird commented 5 years ago

Yes, that's one of those funky dual mode drivers.

stsp commented 5 years ago

So we load it from device=. In this case it doesn't matter that it crashes "when loaded as an exe from the command line". So why it is not a bug that in fdpp it crashes even with device=?

andrewbird commented 5 years ago

It's not directly related to this ticket as XMSDSK was a secondary example to avoid using an experimental driver, so that's why I closed it. But I suppose that even though it's generally not great on FreeDOS it's still not right that loading with FDPP crashes.

stsp commented 5 years ago

Exactly. It works perfectly under freedos with device=.

andrewbird commented 5 years ago

Chasing this down, it seems the crash occurs in InitializeAllBPBs() when trying to open a non existent file on G:, that should be okay as it worked for all the preceding drives. open is defined to init_DosOpen() but I don't seem to be able to find the source for that, where should I look?

stsp commented 5 years ago

In asm files. Another thing to check is if maybe its called after init_text is being purged (something not done in freedos, I added purge to catch bugs)

stsp commented 5 years ago

Though if it was done for other drives in the same call, then its definitely not purged.

andrewbird commented 5 years ago

So I end up in dosdebug tracing kernel code. After a while I see a suspicious return to what looks like uninitialized data

dosdebug> 
dosdebug> 
system state: stopped
AX=0003  BX=0008  CX=02b2  DX=03ca  SI=0000  DI=0001  SP=039e  BP=039e
DS=00d9  ES=0060  FS=0000  GS=0000  FL=000a3346
CS:IP=078b:03aa       SS:SP=00d9:039e

078b:03aa 5F               pop  di
dosdebug> 
dosdebug> 
system state: stopped
AX=0003  BX=0008  CX=02b2  DX=03ca  SI=0000  DI=f800  SP=03a0  BP=039e
DS=00d9  ES=0060  FS=0000  GS=0000  FL=000a3346
CS:IP=078b:03ab       SS:SP=00d9:03a0

078b:03ab 5E               pop  si
dosdebug> 
dosdebug> 
system state: stopped
AX=0003  BX=0008  CX=02b2  DX=03ca  SI=0ca7  DI=f800  SP=03a2  BP=039e
DS=00d9  ES=0060  FS=0000  GS=0000  FL=000a3346
CS:IP=078b:03ac       SS:SP=00d9:03a2

078b:03ac 5D               pop  bp
dosdebug> 
dosdebug> 
system state: stopped
AX=0003  BX=0008  CX=02b2  DX=03ca  SI=0ca7  DI=f800  SP=03a4  BP=02fa
DS=00d9  ES=0060  FS=0000  GS=0000  FL=000a3346
CS:IP=078b:03ad       SS:SP=00d9:03a4

078b:03ad C3               ret
dosdebug> 
dosdebug> 
system state: stopped
AX=0003  BX=0008  CX=02b2  DX=03ca  SI=0ca7  DI=f800  SP=03a6  BP=02fa
DS=00d9  ES=0060  FS=0000  GS=0000  FL=000a3346
CS:IP=078b:0001       SS:SP=00d9:03a6

078b:0001 0000             add  [bx+si],al
dosdebug> 
dosdebug> 
system state: stopped
AX=0003  BX=0008  CX=02b2  DX=03ca  SI=0ca7  DI=f800  SP=03a6  BP=02fa
DS=00d9  ES=0060  FS=0000  GS=0000  FL=000a3306
CS:IP=078b:0003       SS:SP=00d9:03a6

078b:0003 0000             add  [bx+si],al

Any idea how I can convert memory addresses in kernel.map file to seg:ofs so I could have symbols?

stsp commented 5 years ago

kernel.map is quite easily convertible to seg:off. There are load addresses to which you can add 0x600 (our load address) to match the dosdebug address.

Also you can use -D9+f to see in the log what freedos functions are being called. Seeing the last func before crash, may give some clue. Also you can breakpoint the do_asm_call func and see the stacktrace on every call. The last one before crash would be very very useful.

stsp commented 5 years ago

0x600 (our load address) to match the dosdebug address.

Not quite. The relocation also happens. After adding 0x60 to segment, you need to see the -D9+f log to check if that segment was relocated. If so, it is printed there, and you see its new value.

andrewbird commented 5 years ago

I think I'm going to work on adding the dosdebug symbol relocation you asked for.

BTW do you think it would be possible to run the mhpdbg stuff in a separate thread, as I stumbled across this gdb reference.

stsp commented 5 years ago

In gdb you usually can do call or print and call any target function w/o involving any threads. So you can just queue some dosdebug commands in the pipe and then call mhp_pool perhaps or something like that with gdb. Haven't tried this myself, but looks much easier than involving threads.

stsp commented 5 years ago

Symbol relocation would be nice, but I wonder if this particular bug needs it that badly. :)

andrewbird commented 5 years ago

So you can just queue some dosdebug commands in the pipe and then call mhp_pool perhaps or something like that with gdb.

That's interesting, I didn't know that was possible.

Symbol relocation would be nice, but I wonder if this particular bug needs it that badly. :)

Probably not, but I'm doing that to avoid finding the bug :)

stsp commented 5 years ago

Probably not, but I'm doing that to avoid finding the bug :)

Deal. :)

stsp commented 5 years ago

So in gdb things like p foo(2) usually work fine. I don't even know why the one would use call if print just works unless foo returns void.

stsp commented 5 years ago

Works even when foo returns void it seems.

andrewbird commented 5 years ago

cool!

stsp commented 5 years ago

The crash happens in CriticalError() fn. To reproduce it w/o any extra drivers, the one can just type a: to provoke the critical error. Sigh. This is definitely a critical error in fdpp. :)

stsp commented 5 years ago

And of course when we fix CriticalError() to not crash, we will return back to see why that driver causes the critical error under fdpp...

stsp commented 5 years ago

Added #88

andrewbird commented 5 years ago

So the crash is gone, but the unavailable g: is still there retry