search

PiSCSI / piscsi

PiSCSI allows a Raspberry Pi to function as emulated SCSI devices (hard disk, CD-ROM, and others) for vintage SCSI-based computers and devices. This is a fork of the RaSCSI project by GIMONS.
https://piscsi.org
BSD 3-Clause "New" or "Revised" License
545 stars 82 forks source link

Testing improvement: Running initiator and target in the same process #1295

Closed uweseimet closed 1 year ago

uweseimet commented 1 year ago

From a technical perspective it should be possible to run an initiator and a target in the same process, connected by an in-process bus. This would be very helpful for testing piscsi against scsidump and vice versa, for instance, without the need for any PiSCSI board. Testing everything could be completely decoupled from any hardware. A test coverage of almost 100% should be possible.

This is roughly the approach: The target code (piscsi_core) is launched in a thread and connects to an in-process bus. This bus essentially just remembers the state of the bus signals and is otherwise passive. The initiator code (e.g. scsidump_core) is launched in another thread or the main process and connects to the same bus. The initiator can now send SCSI commands, which are executed by the target thread. The initiator does not notice any difference compared to real hardware. If something goes wrong this means that there must be a bug either in the PiSCSI target or initiator code.

uweseimet commented 1 year ago

First essentially working build run on a regular Linux PC, running scsidump against piscsi in process:

>bin/fullspec/in_process_test -t "-id 0 -t scrm /home/us/hatari/zip.hds -id 6 services" -i "-f dump.hds -t 0"
SCSI Target Emulator PiSCSI (Backend Service)
Version 23.10 --DEVELOPMENT BUILD--  (Nov  4 2023 09:56:09)
Powered by XM6 TypeG Technology / Copyright (C) 2016-2020 GIMONS
Copyright (C) 2020-2023 Contributors to the PiSCSI project
Connection type: FULLSPEC
[2023-11-04 15:28:20.041] [info] Set log level to 'info'
[2023-11-04 15:28:20.041] [info] Cleared reserved ID(s)
[2023-11-04 15:28:20.041] [info] Validating: operation=ATTACH, device=0:0, type=SCRM, device params='file=/home/us/hatari/zip.hds', vendor='', product='', revision='', block size=0
[2023-11-04 15:28:20.041] [info] Validating: operation=ATTACH, device=6:0, type=UNDEFINED, device params='file=services', vendor='', product='', revision='', block size=0
[2023-11-04 15:28:20.041] [info] Executing: operation=ATTACH, device=0:0, type=SCRM, device params='file=/home/us/hatari/zip.hds', vendor='', product='', revision='', block size=0
[2023-11-04 15:28:20.041] [info] Attached SCRM 0:0
[2023-11-04 15:28:20.041] [info] Executing: operation=ATTACH, device=6:0, type=UNDEFINED, device params='file=services', vendor='', product='', revision='', block size=0
[2023-11-04 15:28:20.041] [info] Attached SCHS 6:0
[2023-11-04 15:28:20.041] [info] +----+-----+------+-------------------------------------
[2023-11-04 15:28:20.041] [info] | ID | LUN | TYPE | IMAGE FILE
[2023-11-04 15:28:20.041] [info] +----+-----+------+-------------------------------------
[2023-11-04 15:28:20.041] [info] |  0 |   0 | SCRM | /home/us/hatari/zip.hds
[2023-11-04 15:28:20.041] [info] |  6 |   0 | SCHS | Host Services
[2023-11-04 15:28:20.041] [info] +----+-----+------+-------------------------------------
+----+-----+------+-------------------------------------
| ID | LUN | TYPE | IMAGE FILE
+----+-----+------+-------------------------------------
|  0 |   0 | SCRM | /home/us/hatari/zip.hds
|  6 |   0 | SCHS | Host Services
+----+-----+------+-------------------------------------
Note: No PiSCSI hardware support, only client interface calls are supported
SCSI Target Emulator PiSCSI (Hard Disk Dump/Restore Utility)
Version 23.10 --DEVELOPMENT BUILD--  (Nov  4 2023 09:56:09)
Powered by XM6 TypeG Technology / Copyright (C) 2016-2020 GIMONS
Copyright (C) 2020-2023 Contributors to the PiSCSI project
----------------------------------------
PiSCSI board ID is 7
----------------------------------------
Target device is 0:0
Vendor:      PiSCSI
Product:     SCSI HD (REM.)
Revision:    2310
Device Type: Direct Access
Removable:   Yes
Sectors:     1280
Sector size: 512 bytes
Capacity:    0 MiB (655360 bytes)
----------------------------------------

Starting dump, buffer size is 1048576 bytes

100% (655360/655360)
----------------------------------------
Transferred 0 MiB (655360 bytes)
Total time: 1 seconds (0 minutes)
Average transfer rate: 655360 bytes per second (640 KiB per second)
----------------------------------------
uweseimet commented 1 year ago

@rdmark I don't need a Pi anymore in order to test the core functionality :-).

rdmark commented 1 year ago

That’s very handy! Let me try how well this works on native Apple Silicon Linux now (Fedora)

On Sat, Nov 4, 2023 at 11:37 PM, Uwe Seimet @.***(mailto:On Sat, Nov 4, 2023 at 11:37 PM, Uwe Seimet < wrote:

@.***(https://github.com/rdmark) I don't need a Pi anymore in order to test the core functionality :-).

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

uweseimet commented 1 year ago

@rdmark It might even be possible to test on regular macos, or maybe sooner or later. The more we are using standard C++ functioinality and remove Linux-specific functionality, the closer we get. The current sources already compile on FreeBSD, where I sometimes test with a VM. They do not link on FreeBSD, though, but this is an issue not related to PiSCSI. If you can try to compile on macos I would be interested in a logfile if there are compile-time errors. Being able to compile on many platforms can help with testing, but sometimes it's also nice to be able to run scsictl in order to remotely access piscsi. In general I would expect everything to compile on more or less any up to date Linux distribution.

uweseimet commented 1 year ago

@rdmark I know now why linking on FreeBSD does not work: It has to do with an unfortunate compiler and library setup/combination on this platform, which prevents linking against the protobuf library. FreeBSD 14.0 (due this month) is supposed to work better. I once had a macos VM, where I was able to build a working scsictl out of the box. Note that parsing the command line options on macos has a runtime issue, see the respective comment in scsictl_core. All of our binaries are affected by this.

uweseimet commented 1 year ago

@rdmark My guess is that you would be the one to review the PR for this ticket as soon as the first bookworm release is done. This ticket is based on the changes for tickets #1263, #1278 and #1283. Just reviewing/merging this ticket and not reviewing/merging the other three tickets first is less work. This is why I recommend this approach. Note that with this ticket the daynaport and handshake timing changes, even though there are still tests (by others) pending. Nevertheless I would like to commit the modified timing to develop. If need should be I will revert it in time for the first release after 23.11.