obarthel / amiga-smbfs

A SMB file system wrapper for AmigaOS, using the AmiTCP V3 API
42 stars 10 forks source link

Small operation performance regression #4

Open patrikaxelsson opened 5 years ago

patrikaxelsson commented 5 years ago

When upgrading from the old SMBFS 1.74 to the current 2.1 version, I ran diskspeed on both to see if anything had changed. Noticed that the ratings for small create and open file had dropped significantly.

Some examples: 50 -> 9 create file/sec, 147 -> 19 open file/sec, 102720Bytes/sec -> 9488Bytes/sec create file with 512B block.

An interesting detail is that for the ratings that has regressed, the CPU % availability is very high for 2.1, so it is not an inefficiency, but it seems like in those cases, the Amiga is spending most of the time idle, waiting for something to happen.

Tests results in issue are from an A3000-030@25MHz, X-Surf-100 and Roadshow 1.13 against a Debian 9.5 server with Samba 4.5.12. Also tested on an A2000-030@40MHz, A2065 and Roadshow 1.13 where I the issue could be replicated.

Test result where SMBFS 1.74 is run first, then 2.1:

8.Ram Disk:> version
Kickstart 40.68, Workbench 40.42
8.Ram Disk:> showconfig
PROCESSOR:      CPU 68030/68882fpu/68030mmu
CUSTOM CHIPS:   ECS PAL Agnus (id=$0020), ECS Denise (id=$00FC)
VERS:   Kickstart version 40.68, Exec version 40.10, Disk version 40.42
RAM:    Node type $A, Attributes $505 (FAST), at $7000000-$7FFFFFF (16.0 meg)
        Node type $A, Attributes $703 (CHIP), at $400-$1FFFFF (~2.0 meg)
BOARDS:
 Board (unidentified):   Prod=4626/100($1212/$64)
     (@$40000000, size 64meg, subsize 64K)
8.Ram Disk:> cpu
System: 68030 68882 (INST: Cache Burst) (DATA: Cache NoBurst)
8.Ram Disk:> version bsdsocket.library
bsdsocket.library 4.332
8.Ram Disk:> version file devs:Networks/x-surf-100.device 
x-surf-100.device 1.12
8.Ram Disk:> version x-surf-100.device
x-surf-100.device 1.11
8.Ram Disk:> version c:smbfs_1.74 
smbfs 1.74
8.Ram Disk:> Run C:smbfs_1.74 WORKGROUP="xyz" USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 6]
8.Ram Disk:> diskspeed DRIVE=SMBFS: DIR SEEK FAST LONG
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1374

Testing directory manipulation speed.
File Create:           50 files/sec  |  CPU Available: 2%
File Open:            147 files/sec  |  CPU Available: 0%
Directory Scan:       189 files/sec  |  CPU Available: 47%
File Delete:           92 files/sec  |  CPU Available: 3%

Seek/Read:            103 seeks/sec  |  CPU Available: 0%

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       102720 bytes/sec  |  CPU Available: 2%
Write to file:     107520 bytes/sec  |  CPU Available: 0%
Read from file:    111424 bytes/sec  |  CPU Available: 0%

Testing with a 4096 byte, MEMF_FAST, LONG-aligned buffer.
Create file:        70990 bytes/sec  |  CPU Available: 72%
Write to file:      73033 bytes/sec  |  CPU Available: 72%
Read from file:    419840 bytes/sec  |  CPU Available: 0%

Testing with a 32768 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       132193 bytes/sec  |  CPU Available: 32%
Write to file:     138571 bytes/sec  |  CPU Available: 31%
Read from file:    567924 bytes/sec  |  CPU Available: 0%

Testing with a 262144 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       334522 bytes/sec  |  CPU Available: 10%
Write to file:     370085 bytes/sec  |  CPU Available: 5%
Read from file:    533067 bytes/sec  |  CPU Available: 10%

Average CPU Available: 17%  |  CPU Availability index: 234
8.Ram Disk:> break 6
8.Ram Disk:> version c:smbfs_2.1
smbfs 2.1
8.Ram Disk:> Run C:smbfs_2.1 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 6]
8.Ram Disk:> diskspeed DRIVE=SMBFS: DIR SEEK FAST LONG
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1371

Testing directory manipulation speed.
File Create:            9 files/sec  |  CPU Available: 73%
File Open:             19 files/sec  |  CPU Available: 77%
Directory Scan:       225 files/sec  |  CPU Available: 18%
File Delete:            9 files/sec  |  CPU Available: 81%

Seek/Read:            171 seeks/sec  |  CPU Available: 0%

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9488 bytes/sec  |  CPU Available: 74%
Write to file:       9703 bytes/sec  |  CPU Available: 74%
Read from file:     97664 bytes/sec  |  CPU Available: 1%

Testing with a 4096 byte, MEMF_FAST, LONG-aligned buffer.
Create file:        64351 bytes/sec  |  CPU Available: 68%
Write to file:      67757 bytes/sec  |  CPU Available: 67%
Read from file:    392192 bytes/sec  |  CPU Available: 1%

Testing with a 32768 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       291005 bytes/sec  |  CPU Available: 24%
Write to file:     423864 bytes/sec  |  CPU Available: 0%
Read from file:    563838 bytes/sec  |  CPU Available: 1%

Testing with a 262144 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       251457 bytes/sec  |  CPU Available: 37%
Write to file:     330686 bytes/sec  |  CPU Available: 24%
Read from file:    524288 bytes/sec  |  CPU Available: 10%

Average CPU Available: 37%  |  CPU Availability index: 507
patrikaxelsson commented 5 years ago
raziel- commented 5 years ago

Try with CACHE=1024 in your smbfs line. (That will set the "Buffers" to 1024 and you'll get your i/o speed back)

v2.1 uses all the bandwidth it can get and i found that this setting is a very good compromise between wasting cache space and speed gain.

The speed went down for me too while testing the different version, but that is because smbfs2.1 is using far more complex i/o operations (and error proof) than 1.74.

DiskSpeed 4.4, OS4 version Copyright © 1989-92 MKSoft Development Copyright © 2003-04 Daniel J. Andrea II & Stéphane Guillard Modified December 2015 by A. W. Wyatt for VP DOS API


CPU: 68020 AmigaOS Version: 53.89 Normal Video DMA Device: Buffers: 1024

Testing directory manipulation speed. File Create: 111 files/sec File Open: 1229 files/sec Directory Scan: 53.16 kfiles/sec File Delete: 181 files/sec

Seek/Read: 1591 seeks/sec

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer. Create file: 12.35 kiB/sec Write to file: 12.07 kiB/sec Read from file: 754.25 kiB/sec

Testing with a 4096 byte, MEMF_FAST, LONG-aligned buffer. Create file: 96.04 kiB/sec Write to file: 98.63 kiB/sec Read from file: 1.91 MiB/sec

Testing with a 32768 byte, MEMF_FAST, LONG-aligned buffer. Create file: 6.28 MiB/sec Write to file: 6.59 MiB/sec Read from file: 2.12 MiB/sec

Testing with a 262144 byte, MEMF_FAST, LONG-aligned buffer. Create file: 2.82 MiB/sec Write to file: 3.54 MiB/sec Read from file: 2.63 MiB/sec

patrikaxelsson commented 5 years ago

What are your results with standard CACHE settings?

raziel- commented 5 years ago

hehe, they are actually faster, guess i was wrong;-)

DiskSpeed 4.4, OS4 version Copyright © 1989-92 MKSoft Development Copyright © 2003-04 Daniel J. Andrea II & Stéphane Guillard Modified December 2015 by A. W. Wyatt for VP DOS API


CPU: 68020 AmigaOS Version: 53.89 Normal Video DMA Device: Buffers: 170

Testing directory manipulation speed. File Create: 212 files/sec File Open: 1253 files/sec Directory Scan: 54.14 kfiles/sec File Delete: 503 files/sec

Seek/Read: 1866 seeks/sec

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer. Create file: 12.41 kiB/sec Write to file: 12.43 kiB/sec Read from file: 848.76 kiB/sec

Testing with a 4096 byte, MEMF_FAST, LONG-aligned buffer. Create file: 99.07 kiB/sec Write to file: 82.89 kiB/sec Read from file: 2.81 MiB/sec

Testing with a 32768 byte, MEMF_FAST, LONG-aligned buffer. Create file: 5.32 MiB/sec Write to file: 4.68 MiB/sec Read from file: 1.80 MiB/sec

Testing with a 262144 byte, MEMF_FAST, LONG-aligned buffer. Create file: 3.40 MiB/sec Write to file: 3.34 MiB/sec Read from file: 3.01 MiB/sec

patrikaxelsson commented 5 years ago

Yes, it seems it does not help, "Directory Scan" is twice as fast, but otherwise more or less the same - comparison between std and CACHE=1024:

7.Ram Disk:> version
Kickstart 40.68, Workbench 40.42
7.Ram Disk:> showconfig
PROCESSOR:      CPU 68030/68882fpu/68030mmu
CUSTOM CHIPS:   ECS PAL Agnus (id=$0020), ECS Denise (id=$00FC)
VERS:   Kickstart version 40.68, Exec version 40.10, Disk version 40.42
RAM:    Node type $A, Attributes $505 (FAST), at $7000000-$7FFFFFF (16.0 meg)
        Node type $A, Attributes $703 (CHIP), at $400-$1FFFFF (~2.0 meg)
BOARDS:
 Board (unidentified):   Prod=4626/100($1212/$64)
     (@$40000000, size 64meg, subsize 64K)
7.Ram Disk:> cpu
System: 68030 68882 (INST: Cache Burst) (DATA: Cache NoBurst)
7.Ram Disk:> version bsdsocket.library
bsdsocket.library 4.332
7.Ram Disk:> version file devs:Networks/x-surf-100.device 
x-surf-100.device 1.12
7.Ram Disk:> version x-surf-100.device
x-surf-100.device 1.11
7.Ram Disk:> version C:smbfs_2.1 
smbfs 2.1
7.Ram Disk:> Run C:smbfs_2.1 USER=patrik PASSWORD=xyz SERVICE=//megaburken
/patrik
[CLI 6]
7.Ram Disk:> diskspeed DRIVE=SMBFS: DIR SEEK FAST LONG
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
Error:  Could not create test directory.
7.Ram Disk:> diskspeed DRIVE=SMBFS: DIR SEEK FAST LONG
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
Error:  Could not create test directory.
7.Ram Disk:> diskspeed DRIVE=SMBFS: DIR SEEK FAST LONG
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1372

Testing directory manipulation speed.
File Create:            9 files/sec  |  CPU Available: 73%
File Open:             19 files/sec  |  CPU Available: 77%
Directory Scan:       220 files/sec  |  CPU Available: 20%
File Delete:            9 files/sec  |  CPU Available: 81%

Seek/Read:            171 seeks/sec  |  CPU Available: 1%

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9512 bytes/sec  |  CPU Available: 75%
Write to file:       9728 bytes/sec  |  CPU Available: 75%
Read from file:     97856 bytes/sec  |  CPU Available: 0%

Testing with a 4096 byte, MEMF_FAST, LONG-aligned buffer.
Create file:        64351 bytes/sec  |  CPU Available: 69%
Write to file:      67926 bytes/sec  |  CPU Available: 68%
Read from file:    394240 bytes/sec  |  CPU Available: 1%

Testing with a 32768 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       293865 bytes/sec  |  CPU Available: 24%
Write to file:     423864 bytes/sec  |  CPU Available: 1%
Read from file:    563838 bytes/sec  |  CPU Available: 1%

Testing with a 262144 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       252668 bytes/sec  |  CPU Available: 37%
Write to file:     330686 bytes/sec  |  CPU Available: 24%
Read from file:    531795 bytes/sec  |  CPU Available: 8%

Average CPU Available: 37%  |  CPU Availability index: 508
7.Ram Disk:> break 6
7.Ram Disk:> Run C:smbfs_2.1 CACHE=1024 USER=patrik PASSWORD=xyz SERVICE=/
/megaburken/patrik
[CLI 6]
7.Ram Disk:> diskspeed DRIVE=SMBFS: DIR SEEK FAST LONG
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 1024

CPU Speed Rating: 1375

Testing directory manipulation speed.
File Create:            9 files/sec  |  CPU Available: 73%
File Open:             19 files/sec  |  CPU Available: 77%
Directory Scan:       512 files/sec  |  CPU Available: 3%
File Delete:            9 files/sec  |  CPU Available: 81%

Seek/Read:            171 seeks/sec  |  CPU Available: 1%

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9512 bytes/sec  |  CPU Available: 75%
Write to file:       9664 bytes/sec  |  CPU Available: 75%
Read from file:     97536 bytes/sec  |  CPU Available: 1%

Testing with a 4096 byte, MEMF_FAST, LONG-aligned buffer.
Create file:        64191 bytes/sec  |  CPU Available: 68%
Write to file:      67926 bytes/sec  |  CPU Available: 68%
Read from file:    394752 bytes/sec  |  CPU Available: 0%

Testing with a 32768 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       293865 bytes/sec  |  CPU Available: 24%
Write to file:     424921 bytes/sec  |  CPU Available: 1%
Read from file:    565248 bytes/sec  |  CPU Available: 0%

Testing with a 262144 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       252061 bytes/sec  |  CPU Available: 37%
Write to file:     331446 bytes/sec  |  CPU Available: 24%
Read from file:    544798 bytes/sec  |  CPU Available: 7%

Average CPU Available: 36%  |  CPU Availability index: 495
raziel- commented 5 years ago

I'm not sure about the real life potential of those benchmark figures anyway.

I did a lot of speed comparision with diskspeed 1.3 and 4.4 ( to have another program for cross comparision) during development and there were two significant speed jumps.

The first was when Olaf introduced a new and polished i/o access method with the (now fixed) correct access and error handling and the second one when my hardware developer rolled out a new firmware for the NAS device.

So, it's depending on more factors than only smbfs, i believe.

I know that you checked the same line with 1.74 and 2.1, so something is draining your access rate, what that is i can only imagine.

Then again, did you check with "real life" tests? i.e.copy a big file back and forth, copy lots of small files back and forth?

I did that too extensively during development by copying a dir with lots of .flac files from the nas to ram: and back to the nas while measuring the time. I post the figures, but it's not to be taken for any comparision for other hardware, because while i did those testing a lot of stuff changed (firmware, cables etc.)...but as you can see the speed isn't really going up or down. At some point in development it simply stalled and stayed at a very good rate (imho).

I get 6.5 MBit max from my LAN based Amiga hardware (the bottleneck remaining is Roadshow ... "Hi Olaf" waves :-D )

Plus, it's not all about speed, i'd rather have an error prone and (i/o) secure smbfs i can work with than something really fast that trashes my data.

1.74 copy music#? all to ram: took 1.25,252,198 seconds copy ram:music#? all to "" took 1.19,643,812 seconds delete music#? all took 1,340,351 seconds

1.160 copy music#? all to ram: took 45,889,226 seconds copy ram:music#? all to "" took 42,388,385 seconds delete music#? all took 0,460,635 seconds

1.170 copy music#? all to ram: took 45,316.105 seconds copy ram:music#? all to "" took 41,115.139 seconds delete music#? all took 0,452.75 seconds

1.181 copy music#? all to ram: took 48,795.096 seconds copy ram:music#? all to "" took 39,442.621 seconds delete music#? all took 0,417.365 seconds

2.1 copy music#? all to ram: took 49,125.952 seconds copy ram:music#? all to "" took 33,035.445 seconds delete music#? all took 0,251.316 seconds

patrikaxelsson commented 5 years ago

I have no complaint with say the top read speed with large buffers. The A3000 I tested on with its 030@25MHz manages just under 700kB/sec of raw TCP receive and this result is not far from that.

What I find interesting with most of the small operation tests with smbfs 2.1 is that there is a very high CPU availabilty percentage, indicating that during those tests the Amiga is mostly spending time waiting and adding a really fast CPU would not make a difference - which explains why I got no better score at those test when I tested on my A1200 with 060@50Mhz.

To get some real life comparison, I did a test with smbfs 1.74 and 2.1 copying a collection of files (~4.3MB with mixed file sizes, but a lot of small ones). With smbfs 2.1, more than 4 times the time was required to copy to the SMBFS: volume compared to 1.74 and more than 2.5 times was required to copy from the SMBFS: volume:

7.Ram Disk:> version
Kickstart 40.68, Workbench 40.42
7.Ram Disk:> showconfig
PROCESSOR:      CPU 68030/68882fpu/68030mmu
CUSTOM CHIPS:   ECS PAL Agnus (id=$0020), ECS Denise (id=$00FC)
VERS:   Kickstart version 40.68, Exec version 40.10, Disk version 40.42
RAM:    Node type $A, Attributes $505 (FAST), at $7000000-$7FFFFFF (16.0 meg)
        Node type $A, Attributes $703 (CHIP), at $400-$1FFFFF (~2.0 meg)
BOARDS:
 Board (unidentified):   Prod=4626/100($1212/$64)
     (@$40000000, size 64meg, subsize 64K)
7.Ram Disk:> cpu
System: 68030 68882 (INST: Cache Burst) (DATA: Cache NoBurst)
7.Ram Disk:> version bsdsocket.library
bsdsocket.library 4.332
7.Ram Disk:> version FILE DEVS:Networks/x-surf-100.device 
x-surf-100.device 1.12
7.Ram Disk:> version x-surf-100.device 
x-surf-100.device 1.11
7.Ram Disk:> aget whdload.de/whdload/WHDLoad_usr.lha QUIET
7.Ram Disk:> md5sum WHDLoad_usr.lha
e9b78e5c0ed2e04f3ec7eea2452f1b22  WHDLoad_usr.lha
7.Ram Disk:> lha -q x WHDLoad_usr.lha
7.Ram Disk:> aget uhc.megaburken.net/time QUIET
7.Ram Disk:> stack 65536

7.Ram Disk:> version c:smbfs_1.74 
smbfs 1.74
7.Ram Disk:> Run C:smbfs_1.74 WORKGROUP=xyz USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 6]
7.Ram Disk:> makedir SMBFS:destdir1
7.Ram Disk:> time copy WHDLoad(|.info) SMBFS:destdir1/ ALL QUIET
40.266105s
7.Ram Disk:> makedir SMBFS:destdir2
7.Ram Disk:> time copy WHDLoad(|.info) SMBFS:destdir2/ ALL QUIET
40.265518s
7.Ram Disk:> break 6
7.Ram Disk:> version c:smbfs_2.1 
smbfs 2.1
7.Ram Disk:> Run C:smbfs_2.1 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 6]
7.Ram Disk:> makedir SMBFS:destdir3
7.Ram Disk:> time copy WHDLoad(|.info) SMBFS:destdir3/ ALL QUIET
167.345548s
7.Ram Disk:> makedir SMBFS:destdir4
7.Ram Disk:> time copy WHDLoad(|.info) SMBFS:destdir4/ ALL QUIET
167.423803s

7.Ram Disk:> break 6
7.Ram Disk:> Run C:smbfs_1.74 WORKGROUP=xyz USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 6]
7.Ram Disk:> delete WHDLoad(|.info) ALL QUIET
7.Ram Disk:> time copy SMBFS:destdir2/WHDLoad(|.info) "" ALL QUIET
30.678721s
7.Ram Disk:> delete WHDLoad(|.info) ALL QUIET
7.Ram Disk:> time copy SMBFS:destdir2/WHDLoad(|.info) "" ALL QUIET
30.349084s
7.Ram Disk:> break 6
7.Ram Disk:> Run C:smbfs_2.1 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 6]
7.Ram Disk:> delete WHDLoad(|.info) ALL QUIET
7.Ram Disk:> time copy SMBFS:destdir4/WHDLoad(|.info) "" ALL QUIET
77.834731s
7.Ram Disk:> delete WHDLoad(|.info) ALL QUIET
7.Ram Disk:> time copy SMBFS:destdir4/WHDLoad(|.info) "" ALL QUIET
77.315453s
obarthel commented 5 years ago

I have an idea on what may be responsible for the small-sized read/write operations having taken a performance hit.

One of the changes in smbfs 2.1 was to reduce the amount of unnecessary data copying to/from memory, as well as trying to limit unnecessary path name comparisons and length recalculations.

To this effect the file read/write operations were broken up into two subsequent read and write operations, respectively, rather than receiving a single lump of data/combining data into a single lump. This saves at least one big memory copying operation, but it has side-effects.

These side-effects turned out to be complex and subtle at the same time, because depending upon volume and throughput, the TCP/IP stack may not choose to send small amounts of data just yet, and an internal timeout has to elapse before this data is finally transmitted. This may explain why there is less CPU load involved than in version 1.74, for example.

To get an idea of the impact of these side-effects, smbfs 2.8 adds the READTHRESHOLD and WRITETHRESHOLD options. Both settings default to 0, but it may be interesting to know if values of 1000-1500, or higher, will improve performance for the small-size read/write operations.

raziel- commented 5 years ago

I was searching high and low for those options until i realized 2.8 is an upcoming version :-)

patrikaxelsson commented 5 years ago

Did some tests with the 2.11 version, but cut down on the variables by limiting the test to 512B create/write/read.

1.74 baseline:

8.Ram Disk:> version C:smbfs_1.74 
smbfs 1.74
8.Ram Disk:> Run C:smbfs_1.74 WORKGROUP="xyz" USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1373

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       102208 bytes/sec  |  CPU Available: 2%
Write to file:     105920 bytes/sec  |  CPU Available: 0%
Read from file:    110528 bytes/sec  |  CPU Available: 0%

Average CPU Available: 1%  |  CPU Availability index: 14
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1370

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:       102272 bytes/sec  |  CPU Available: 2%
Write to file:     106624 bytes/sec  |  CPU Available: 0%
Read from file:    110912 bytes/sec  |  CPU Available: 0%

Average CPU Available: 1%  |  CPU Availability index: 14
8.Ram Disk:> break 7

2.11 with no extra options:

8.Ram Disk:> version C:smbfs_2.11 
smbfs 2.11
8.Ram Disk:> Run C:smbfs_2.11 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1371

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9512 bytes/sec  |  CPU Available: 75%
Write to file:       9703 bytes/sec  |  CPU Available: 75%
Read from file:     96768 bytes/sec  |  CPU Available: 1%

Average CPU Available: 50%  |  CPU Availability index: 685
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1375

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9512 bytes/sec  |  CPU Available: 74%
Write to file:       9639 bytes/sec  |  CPU Available: 74%
Read from file:     97344 bytes/sec  |  CPU Available: 1%

Average CPU Available: 50%  |  CPU Availability index: 688
8.Ram Disk:> break 7

These results are consistent with 2.1.

2.11 with several WRITETHRESHOLD and READTHRESHOLD combinations:

8.Ram Disk:> Run C:smbfs_2.11 WRITETHRESHOLD=1000 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1373

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9512 bytes/sec  |  CPU Available: 74%
Write to file:       9679 bytes/sec  |  CPU Available: 74%
Read from file:     96576 bytes/sec  |  CPU Available: 0%

Average CPU Available: 49%  |  CPU Availability index: 673
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 WRITETHRESHOLD=10000 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1371

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9472 bytes/sec  |  CPU Available: 75%
Write to file:       9703 bytes/sec  |  CPU Available: 75%
Read from file:     96896 bytes/sec  |  CPU Available: 1%

Average CPU Available: 50%  |  CPU Availability index: 685
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 WRITETHRESHOLD=1000 READTHRESHOLD=1000 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1373

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9448 bytes/sec  |  CPU Available: 74%
Write to file:       9728 bytes/sec  |  CPU Available: 74%
Read from file:     96896 bytes/sec  |  CPU Available: 0%

Average CPU Available: 49%  |  CPU Availability index: 673
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 WRITETHRESHOLD=10000 READTHRESHOLD=10000 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1370

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9472 bytes/sec  |  CPU Available: 75%
Write to file:       9728 bytes/sec  |  CPU Available: 74%
Read from file:     96960 bytes/sec  |  CPU Available: 0%

Average CPU Available: 50%  |  CPU Availability index: 685
8.Ram Disk:> break 7

Neither WRITETHRESHOLD or READTHRESHOLD affects the results of this test.

2.11 with TCPNODELAY option:

8.Ram Disk:> Run C:smbfs_2.11 TCPNODELAY USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1373

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9488 bytes/sec  |  CPU Available: 74%
Write to file:       9703 bytes/sec  |  CPU Available: 74%
Read from file:     96704 bytes/sec  |  CPU Available: 1%

Average CPU Available: 50%  |  CPU Availability index: 687
8.Ram Disk:> break 7

TCPNODELAY does not affect the results for this test.

2.11 with SCATTHERGATHER=yes:

8.Ram Disk:> Run C:smbfs_2.11 SCATTERGATHER=on USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright © 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1376

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:        52032 bytes/sec  |  CPU Available: 4%
Write to file:      55040 bytes/sec  |  CPU Available: 1%
Read from file:     97152 bytes/sec  |  CPU Available: 0%

Average CPU Available: 2%  |  CPU Availability index: 28
8.Ram Disk:> break 7

SCATTERGATHER=yes gives a huge improvement. Still not at 1.74 levels, but a huge improvement from 1/10 to 1/2 of the 1.74 performance.

patrikaxelsson commented 5 years ago

Did a run of the earlier real-life example copy-test with the new 2.11 options

1.74 baseline:

8.Ram Disk:> aget whdload.de/whdload/WHDLoad_usr.lha QUIET
8.Ram Disk:> md5sum WHDLoad_usr.lha 
e9b78e5c0ed2e04f3ec7eea2452f1b22  WHDLoad_usr.lha
8.Ram Disk:> lha -q x WHDLoad_usr.lha 
8.Ram Disk:> aget uhc.megaburken.net/time QUIET
8.Ram Disk:> stack 65536
8.Ram Disk:> Run C:smbfs_1.74 WORKGROUP=xyz USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> makedir SMBFS:destdir1
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir1/ ALL QUIET
40.622310s
8.Ram Disk:> makedir SMBFS:destdir2
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir2/ ALL QUIET
40.586848s
8.Ram Disk:> break 7

2.11 with all the options from the previous comment:

8.Ram Disk:> Run C:smbfs_2.11 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> makedir SMBFS:destdir3
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir3/ ALL QUIET
169.025733s
8.Ram Disk:> makedir SMBFS:destdir4
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir4/ ALL QUIET
169.190842s
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 WRITETHRESHOLD=1000 USER=patrik PASSWORD=xyz  SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> makedir SMBFS:destdir6
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir6/ ALL QUIET
168.873734s
8.Ram Disk:> makedir SMBFS:destdir7
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir7/ ALL QUIET
169.025955s
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 WRITETHRESHOLD=10000 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> makedir SMBFS:destdir8
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir8/ ALL QUIET
169.132160s
8.Ram Disk:> makedir SMBFS:destdir9
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir9/ ALL QUIET
169.115331s
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 WRITETHRESHOLD=1000 READTHRESHOLD=1000 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> makedir SMBFS:destdir10
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir10/ ALL QUIET
169.250926s
8.Ram Disk:> makedir SMBFS:destdir11
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir11/ ALL QUIET
169.201319s
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 WRITETHRESHOLD=10000 READTHRESHOLD=10000 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> makedir SMBFS:destdir12
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir12/ ALL QUIET
169.347863s
8.Ram Disk:> makedir SMBFS:destdir13
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir13/ ALL QUIET
169.265387s
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 TCPNODELAY USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> makedir SMBFS:destdir14
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir14/ ALL QUIET
169.485019s
8.Ram Disk:> makedir SMBFS:destdir15
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir15/ ALL QUIET
169.193748s
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.11 SCATTERGATHER=yes USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> makedir SMBFS:destdir16
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir16/ ALL QUIET
161.890515s
8.Ram Disk:> makedir SMBFS:destdir17
8.Ram Disk:> time copy WHDLOAD(|.info) SMBFS:destdir17/ ALL QUIET
161.889506s
8.Ram Disk:> 

The only option which made a difference here, as in the previous test with diskspeed, was the SCATTERGATHER=yes option. However, the improvement was quite small (~4%).

patrikaxelsson commented 5 years ago

With SMBFS 2.12:

8.Ram Disk:> showconfig
PROCESSOR:      CPU 68030/68882fpu/68030mmu
CUSTOM CHIPS:   ECS PAL Agnus (id=$0020), ECS Denise (id=$00FC)
VERS:   Kickstart version 40.68, Exec version 40.10, Disk version 40.42
RAM:    Node type $A, Attributes $505 (FAST), at $7000000-$7FFFFFF (16.0 meg)
        Node type $A, Attributes $703 (CHIP), at $400-$1FFFFF (~2.0 meg)
BOARDS:
 Board (unidentified):   Prod=4626/100($1212/$64)
     (@$40000000, size 64meg, subsize 64K)
8.Ram Disk:> version bsdsocket.library
bsdsocket.library 4.332
8.Ram Disk:> 
8.Ram Disk:> version c:smbfs_2.12
8.Ram Disk:> Run C:smbfs_2.12 USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright ? 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1373

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:         9600 bytes/sec  |  CPU Available: 75%
Write to file:       9743 bytes/sec  |  CPU Available: 75%
Read from file:    108352 bytes/sec  |  CPU Available: 1%

Average CPU Available: 50%  |  CPU Availability index: 687
8.Ram Disk:> break 7
8.Ram Disk:> Run C:smbfs_2.12 SCATTERGATHER=on USER=patrik PASSWORD=xyz SERVICE=//megaburken/patrik
[CLI 7]
8.Ram Disk:> diskspeed DRIVE=SMBFS: LONG FAST BUF1=512 BUF2=0 BUF3=0 BUF4=0
MKSoft DiskSpeed 4.2  Copyright ? 1989-92 MKSoft Development
------------------------------------------------------------
CPU: 68030  AmigaOS Version: 40.68  Normal Video DMA
Device:  SMBFS:    Buffers: 170

CPU Speed Rating: 1376

Testing with a 512 byte, MEMF_FAST, LONG-aligned buffer.
Create file:        55296 bytes/sec  |  CPU Available: 5%
Write to file:      58432 bytes/sec  |  CPU Available: 1%
Read from file:    108096 bytes/sec  |  CPU Available: 1%

Average CPU Available: 2%  |  CPU Availability index: 28
8.Ram Disk:>