Open frno7 opened 5 years ago
I'm waiting on a ABitTop SATA upgrade board and I'd be willing to test out compatibility with it when there's progress with this.
Thanks, @motolav! I believe the most complex issue when making a good driver will be efficient use of DMA, especially when sharing DMA with other IOP drivers, for example networking.
There's already a highly optimized (P)ATA driver for linux v3.8. Fully using linux scatter-gather DMA, and reaching speeds close to the theoretical maximum of UDMA66. The IOP drivers needed can be found here: https://gitlab.com/ps2max/linux/linux-firmware
It consists of:
Also essential for debugging IOP modules, is the eedebug/iopdebug (name is confusing...) driver: https://gitlab.com/ps2max/linux/linux-firmware/tree/master/eedebug https://gitlab.com/ps2max/linux/linux/blob/ps2-v3.8/arch/mips/ps2/iopdebug.c This helps to seend all print/debug messages from the IOP to the EE, so they can then be printed to the default debug output (screen or serial port)
The serial port output is IMO a valabl tool when debugging the kernel. It's been extended to also accept input, so you can start a tty console over serial: https://gitlab.com/ps2max/linux/linux/blob/ps2-v3.8/drivers/tty/serial/ps2_uart.c
The linux implementation for pata can be found here: https://gitlab.com/ps2max/linux/linux/blob/ps2-v3.8/drivers/ata/pata_ps2.c
The linux implementation for ethernet can be found here (files: smap*): https://gitlab.com/ps2max/linux/linux/tree/ps2-v3.8/drivers/ps2
I would like to help out on getting these drivers to work with mailine linux, but I'm not sure if I can get a decent development environment anytime soon since I'm in the middle of moving to another country.
EDIT: For testing I have multiple PATA adapters, a SATA upgraded adapter and a sata clone adapter.
Great, @rickgaiser! Your contributions are most welcome. Take your time, we’ll get there.
This helps to seend all print/debug messages from the IOP to the EE, so they can then be printed to the default debug output (screen or serial port)
I have a similar printk
IOP module here: https://github.com/frno7/iopmod/blob/master/module/printk.c
The serial port output is IMO a valabl tool when debugging the kernel. It's been extended to also accept input, so you can start a tty console over serial:
I’m unable to test this serial driver myself, since I haven’t soldered a serial port. As an alternative, one can use an early putc
with the Graphics Synthesizer, as described in #9. It’s practically equivalent to a true serial port when debugging unmodified hardware, because it will print the same things on the video screen, although it doesn’t accept any input for obvious reasons.
@rickgaiser, I have started reviewing your ATA driver and the links you supplied. This will obviously take some time, as there are significant changes between the 3.8 and 5.4 Linux kernels as well as their corresponding IOP module structures. Nevertheless your driver looks like an excellent starting point.
@rickgaiser, I get a bus error when reading the ATA registers (from the EE), which I suspect is caused by the DEV9 not being powered on properly. So I have a iop-dev9 module that is supposed to power it on, but it seems something is amiss. It follows the steps of ps2dev9 (expbay_init
) of the PS2SDK. And not entirely wrong either, I assume, because it properly detects the expansion device by printing dev9: Expansion device interface
during initialisation. It also claims that the DEV9 is already powered on, by printing dev9: Expansion device already powered on
, which might be wrong, because reading the registers corresponding to SPD_R_REV_1
and SPD_R_REV_3
in the PS2SDK, for example, give bus errors too.
Any ideas on what might be wrong?
There’s quite a bit of obscure hardware poking going on, as can be seen, during expansion device initialisation:
I wrote a minimal dev9 IOP module replacement, and then the initialisation works. It prints
iop: dev9: init start
iop: dev9: rev 31
iop: dev9: power is already on
iop: dev9: spdrev 11
iop: dev9: init exit
in the kernel log, as one would hope for. And then the ATA driver proceeds with
scsi host1: pata-ps2
ata1: PATA max PIO3 irq 77
ata1.00: qc timeout (cmd 0x27)
ata1.00: failed to read native max address (err_mask=0x4)
ata1.00: HPA support seems broken, skipping HPA handling
ata1.00: ATA-11: KINGSTON SA400S37120G, SBFK71E0, max UDMA/133
ata1.00: 234441648 sectors, multi 16: LBA48
scsi 1:0:0:0: Direct-Access ATA KINGSTON SA400S3 71E0 PQ: 0 ANSI: 5
sd 1:0:0:0: [sdb] 234441648 512-byte logical blocks: (120 GB/112 GiB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
which is to be expected, at the moment, because it’s not fully implemented. Although I started out on the EE, it probably is best to do most things on the IOP instead, since it can offload a lot of work for the benefit of the EE. Interesting things can be done if IOP would stream all data through the SIF.
It’s easy to get the driver to work with polling, that is without interrtupts, by ap->flags |= ATA_FLAG_PIO_POLLING
:
ata1: PATA max PIO3 irq 77
ata1.00: ATA-11: KINGSTON SA400S37120G, SBFK71E0, max UDMA/133
ata1.00: 234441648 sectors, multi 16: LBA48
scsi 1:0:0:0: Direct-Access ATA KINGSTON SA400S3 71E0 PQ: 0 ANSI: 5
sd 1:0:0:0: [sdb] 234441648 512-byte logical blocks: (120 GB/112 GiB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sdb: sdb1 sdb2
sd 1:0:0:0: [sdb] Attached SCSI disk
I’ve prepared two partitions that are now detected properly:
# fdisk -l /dev/sdb
Disk /dev/sdb: 112 GB, 120034123776 bytes, 234441648 sectors
14593 cylinders, 255 heads, 63 sectors/track
Units: cylinders of 16065 * 512 = 8225280 bytes
Device Boot StartCHS EndCHS StartLBA EndLBA Sectors Size Id Type
/dev/sdb1 0,0,2 124,254,63 1 2008124 2008124 980M 82 Linux swap
/dev/sdb2 125,0,1 256,254,63 2008125 234436544 232428420 110G 83 Linux
Mounting an EXT4 file system works too:
# mount /dev/sdb2 /mnt
# ls /mnt
lost+found
Of course, with its polling mode, it’s very slow at the moment, as seen with the dd
command:
# dd if=/dev/sdb of=/dev/null bs=4096 count=4096
4096+0 records in
4096+0 records out
16777216 bytes (16.0MB) copied, 9.129766 seconds, 1.8MB/s
@rickgaiser mentioned UDMA/66, which is more than 30× faster than this. The device itself advertises UDMA/133, as shown above. The next task is to get interrupts working. Somehow the registered IRQ_IOP_DEV9
isn’t asserted. Perhaps some part of the DEV9 hardware isn’t enabled, or some special quirk is needed somewhere?
A little offtop: @frno7 just want to mention, that if you want to get a fast answer for some specific question, you can enjoy the discord channel. Almost all active developers are there.
@rickgaiser, I get a bus error when reading the ATA registers (from the EE)
Although I started out on the EE, it probably is best to do most things on the IOP instead, since it can offload a lot of work for the benefit of the EE. Interesting things can be done if IOP would stream all data through the SIF.
The EE can access many things on the IOP bus (memory/devices), but these accesses are always much slower than doing things on the IOP, and transferring the data using DMA. I'm not sure what you mean by "stream all data through the SIF", but it's impossible to stream data directly from HDD -> EE. It always involves 2 DMA transfers (HDD -> IOP -> EE).
@rickgaiser mentioned UDMA/66, which is more than 30× faster than this. The device itself advertises UDMA/133, as shown above. The next task is to get interrupts working. Somehow the registered IRQ_IOP_DEV9 isn’t asserted. Perhaps some part of the DEV9 hardware isn’t enabled, or some special quirk is needed somewhere?
All needed quirks should be in here: https://github.com/ps2dev/ps2sdk/blob/master/iop/dev9/dev9/src/ps2dev9.c
About the speed, there's a few things to note:
To gain max speed:
That being said, 40+ MB/s has been proven possible on ps2/linux.
I'm not sure what you mean by "stream all data through the SIF", but it's impossible to stream data directly from HDD -> EE. It always involves 2 DMA transfers (HDD -> IOP -> EE).
That’s what I had in mind. The SIF can exchange data with several devices too, such as the image processing unit (IPU) in #15 etc. and so isn’t limited to memory transfers only. In principle, I suppose one could make a command such as cat somefile >/dev/ipu
do all data copying in hardware. That could be interesting.
Incidentally, would you happen to know the details on how to acknowledge interrupts with DEV9 (see https://github.com/frno7/iopmod/issues/5), more specifically the ones for SPD_REG16(SPD_R_INTR_STAT)
and SPD_REG16(SPD_R_INTR_MASK)
? I would expect that to be done somewhere near dev9_intr_dispatch
in the PS2SDK, but I couldn’t find it.
Related to https://github.com/frno7/iopmod/issues/5 is also finding documentation on the following DEV9 registers, if someone has figured them out and invented better names for them:
enum _dev9_regnames {
DEV9_R_1460, DEV9_R_1462, DEV9_R_1464, DEV9_R_1466, DEV9_R_1468,
DEV9_R_146A, DEV9_R_POWER,DEV9_R_REV, DEV9_R_1470, DEV9_R_1472,
DEV9_R_1474, DEV9_R_1476, DEV9_R_1478, DEV9_R_147A, DEV9_R_147C,
DEV9_R_147E,
DEV9_R_MAX };
Incidentally, would you happen to know the details on how to acknowledge interrupts with DEV9 (see frno7/iopmod#5), more specifically the ones for SPD_REG16(SPD_R_INTR_STAT) and SPD_REG16(SPD_R_INTR_MASK)? I would expect that to be done somewhere near dev9_intr_dispatch in the PS2SDK, but I couldn’t find it.
Perhaps you're looking for SMAP_REG16(SMAP_R_INTR_CLR), as in this example: https://github.com/ps2dev/ps2sdk/blob/dd9b0b734b1cabf7342c5e2d9278967024745922/iop/network/smap/src/smap.c#L445 I'm not sure why the one is named "SMAP" and the other "SPD", but it seems to use the same flags for both registers in the link above.
Yes, thanks, I’ll try that! With cascading interrupts for DEV9, I believe some parts of the ATA driver can be simplified, and perhaps even more so the SMAP driver. The IRQ relay module will work automatically with DEV9 IRQs too, and so on.
@AKuHAK, the first link seems broken, no?
Interrupts are now working with the ATA driver, via a new IRQ_IOP_SPD_ATA0
virtual IRQ relay, as explained in https://github.comhttps://github.com/frno7/iopmod/issues/5#issuecomment-878216166. Data transfer performance isn’t any better for it, at the moment, but it’s nevertheless a prerequisite for DMA that is expected to be a major improvement.
Now reading via DMA works, as a proof of concept, with a useful 31 MB/s:
# dd if=/dev/sda of=/dev/null bs=4096 count=4096
4096+0 records in
4096+0 records out
16777216 bytes (16.0MB) copied, 0.513587 seconds, 31.2MB/s
That’s a 17× improvement over polling. Verifying file checksums on a mounted file system etc. works too. Before implementing writing via DMA, I’m going to refactor the driver. Redesign it entirely, in fact, to make better use of resources. This harddisk driver will eventually need to work concurrently and share resources with the network driver described in #19, for example.
It is possible to place APA and GPT on the same hard disk, so you could e.g. boot Linux from the hard disk (possibly with a kernel bootstrapper as __mbr
(not to be confused with MBR partitioning scheme)) and use e.g. ext4 on that same hard disk. https://www.psx-place.com/threads/apa-jail.34847/
@uyjulian, I’m unfamiliar with APA, but the general steps for Linux 5.x are these:
root_pivot
command is performed to switch the root from the RAM disk to some other mounted file system, whatever that is;If the Linux kernel doesn’t understand APA, then this could of course, in principle, be implemented. I did some searching and found that someone has written a guide for APA partition support that claims that for Linux 2.x this allows your PS2 Linux and PS2 Game Data to co-exist on the same hard-drive. Linux 5.x may need some additional adaptions, though.
In short, yes, it should be possible with a bit of work.
This hybrid scheme will allow to not use APA at all. The only thing, in such a case primary GPT partition header should be in sector 2, not in sector 1, first 2 sectors will be filled with customized APA mbr, and partitions should start at specific offsets. This approach differs from link that you posted, cause there is firstly mounted APA partition, but with customized hybrid scheme you can skip APA step completely.
@AKuHAK, thanks for the clarification. I’ve created #39 for the APA.
modprobe pata_ps2
will now enable reading harddisks, including mounting compatible preexisting filesystems via mount -o ro /dev/sda1 /mnt
or suchlike. Something like dd if=/dev/sda of=/dev/null bs=4096 count=4096
can be used to test reading speeds.
Commits https://github.com/frno7/iopmod/commit/5ccf9d9c85ae65abeacaeff301c08a930d6aff93 and https://github.com/frno7/iopmod/commit/5ccf9d9c85ae65abeacaeff301c08a930d6aff93 have the corresponding DEV9 and ATA modules for the IOP. As usual, the IRX modules are compiled, installed and linked automatically on a properly updated and configured system.
Unfortunately, I’ve not yet been able to complete the refactoring, due to an apparent hardware timing issue. The DEV9 hardware appears to crash with the changes I was planning to do. Maybe something becomes slightly wrong with the interaction between the ATA and the DEV9 DMA registers? DEV9 triggers bus errors when attempting to access its registers. I won’t attempt to enable writing harddisks until this is solved.
A bounce buffer now handles all cases of unaligned transfers properly. Reading speed is currently about 20 MB/s with the particular hardware I have. At least 30 MB/s can be done with some tuning. Maybe even more than 40 MB/s with some effort.
One thing I noticed is that you are missing the delays when powering on dev9. If those delays are missing, they could be possibly causing a bus error.
Another possibility is the connection between the PS2 and the network adapter is not very good, which may also cause bus errors.
If the code is based on a Linux kernel that needs to be booted from kernelloader or RTE, it may be possible that dev9 was already powered on beforehand, so that code path was not tested very much.
One thing I noticed is that you are missing the delays when powering on dev9. If those delays are missing, they could be possibly causing a bus error.
Hmm. Yeah, but the pata_ps2
driver on the ps2-main
branch actually does work with the current delayless DEV9 power on sequence. It’s stable without issues after hours of testing. :-)
Another possibility is the connection between the PS2 and the network adapter is not very good, which may also cause bus errors.
I suspect it’s an ATA module (iopmod:module/ata.c
) software problem with the IOP, at least until I learn more details. The driver is fine when DEV9 DMA is serviced in its ata_sif_cmd_sg
SIF command interrupt. However, I should like it to use a separate IOP thread, by waking it up with thsemap_isignal_sema(dev->sg_sema_id)
, rather than calling ata_sif_cmd_sg_transfer
directly during the interrupt:
#if 0
/* Waking up the thread to do DEV9 DMA crashes the DEV9 hardware. Timing problem? */
thsemap_isignal_sema(dev->sg_sema_id);
#else
/* Servicing DEV9 DMA directly in the SIF command interrupt is fine. */
ata_sif_cmd_sg_transfer(dev);
#endif
Servicing DEV9 DMA requests in a separate IOP thread changes the timing, which I suspect is the issue with the IOP thread:
for (;;) {
thsemap_wait_sema(dev->sg_sema_id);
ata_sif_cmd_sg_transfer(dev);
}
If the code is based on a Linux kernel that needs to be booted from kernelloader or RTE, it may be possible that dev9 was already powered on beforehand, so that code path was not tested very much.
Yeah, the IOP DEV9 module iopmod:module/dev9.c
isn’t exactly tidy, mostly because it’s a pile of magic numbers for undocumented DEV9 registers. dev9_power_off
isn’t even implemented properly. Hopefully, all these registers and their bit fields can be given names and meanings, eventually.
For booting off HDD, you actually don't need to worry about APA. You can just stick with MBR or GPT while performing minimal modifications to sector 0.
In order to do hdd boot, only sector 0 (same as MBR) and the sector size/offset specified in sector 0 (can create a GPT partition to store this data) needs to be modified (which specifies the location and size of the program to be loaded).
In sector 0, it only cares about bytes 304-312. That is where the offset and size (4 byte integer each, in sectors) are stored
Normally, in MBR, that would be inside the bootstrap code part 2 location
https://en.wikipedia.org/wiki/Master_boot_record#Sector_layout
SoftDev2 is one such program that can be specified in that sector size/offset: https://github.com/parrado/SoftDev2
So it should be possible to make a disk that is bootable in both PC(BIOS+UEFI) and PS2. (But there isn't much of a use case for that.)
I assume some people will use the fdisk
command that comes with the precompiled Busybox, see https://github.com/frno7/gentoo-mipsr5900el/issues/5#issuecomment-1117279625, so everything about partitioning and formatting disks can be done on the PlayStation 2. That’s how I prepared its hard disk, anyway. Is this boot tied to having wLaunchELF?
wLaunchELF is not required. The Linux chainloader or the Linux kernel itself can be stored on the specified sector size/offset.
However, the following restrictions reply:
-O binary
argument of objcopy
), wrapped in a KELF container. SoftDev2 has a sample of this Ah, is standard Sony BIOS able to boot programs from hard disks?
Ah, is standard Sony BIOS able to boot programs from hard disks?
Only from SCPH-3xxxxx to SCPH-5xxxxx.
Ah, is standard Sony BIOS able to boot programs from hard disks?
Only from SCPH-3xxxxx to SCPH-5xxxxx.
Oh, this is very nice and something I think we should most definitely make good use of. :-)
Enable harddisk devices via the PlayStation 2 expansion bay. A starting point is drivers/ata/pata_ps2.c. DMA via the subsystem interface (SIF) is needed for reasonable performance. Plan:
See also #11 and #17.