search

starfive-tech / VisionFive2

445 stars 79 forks source link

Claimed PCIe support doesn't actually work #38

Closed jrtc27 closed 1 year ago

jrtc27 commented 1 year ago 
StarFive # version
U-Boot 2021.10 (Mar 24 2023 - 01:42:56 +0800), Build: jenkins-github_visionfive2-13

riscv64-buildroot-linux-gnu-gcc.br_real (Buildroot VF2_v2.11.5) 10.3.0
GNU ld (GNU Binutils) 2.36.1
StarFive # pci enum
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
StarFive # pci
Scanning PCI devices on bus 0
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x1556     0x1111     Bridge device           0x04

That just shows the host bridge for one of the PLDA controllers, with no sign of the other one and no sign of my NVMe drive.

rmsyn commented 1 year ago

Have you tried running the Debian or Armbian build? At least under Armbian, it is picking up NVMe.

Not sure what they are doing different from the vanilla buildroot image.

MichaelZhuxx commented 1 year ago 
U-Boot 2021.10 (Mar 24 2023 - 01:42:56 +0800), Build: jenkins-github_visionfive2-13

CPU:   rv64imacu
Model: StarFive VisionFive V2
DRAM:  8 GiB
MMC:   sdio0@16010000: 0, sdio1@16020000: 1
Loading Environment from SPIFlash... SF: Detected gd25lq128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB
*** Warning - bad CRC, using default environment

StarFive EEPROM format v2

--------EEPROM INFO--------
Vendor : StarFive Technology Co., Ltd.
Product full SN: VF7110B1-2250-D008E000-00001456
data version: 0x2
PCB revision: 0xb2
BOM revision: A
Ethernet MAC0 address: 6c:cf:39:00:27:31
Ethernet MAC1 address: 6c:cf:39:00:27:32
--------EEPROM INFO--------

In:    serial
Out:   serial
Err:   serial
Model: StarFive VisionFive V2
Net:   eth0: ethernet@16030000, eth1: ethernet@16040000
bootmode sd device 1
Failed to load 'uEnv.txt'
Can't set block device
Hit any key to stop autoboot:  0 
StarFive # 
StarFive # 
StarFive # 
StarFive # pci enum
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
StarFive # pci help
pci - list and access PCI Configuration Space

Usage:
pci [bus] [long]
    - short or long list of PCI devices on bus 'bus'
pci enum
    - Enumerate PCI buses
pci header b.d.f
    - show header of PCI device 'bus.device.function'
pci bar b.d.f
    - show BARs base and size for device b.d.f'
pci regions
    - show PCI regions
pci display[.b, .w, .l] b.d.f [address] [# of objects]
    - display PCI configuration space (CFG)
pci next[.b, .w, .l] b.d.f address
    - modify, read and keep CFG address
pci modify[.b, .w, .l] b.d.f address
    -  modify, auto increment CFG address
pci write[.b, .w, .l] b.d.f address value
    - write to CFG address
StarFive # pci 1 short
Scanning PCI devices on bus 1
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
01.00.00   0x1e4b     0x1202     Mass storage controller 0x08
StarFive # pci 1 long 
Scanning PCI devices on bus 1

Found PCI device 01.00.00:
  vendor ID =                   0x1e4b
  device ID =                   0x1202
  command register ID =         0x0006
  status register =             0x0010
  revision ID =                 0x01
  class code =                  0x01 (Mass storage controller)
  sub class code =              0x08
  programming interface =       0x02
  cache line =                  0x08
  latency time =                0x00
  header type =                 0x00
  BIST =                        0x00
  base address 0 =              0x38000004
  base address 1 =              0x00000000
  base address 2 =              0x00000000
  base address 3 =              0x00000000
  base address 4 =              0x00000000
  base address 5 =              0x00000000
  cardBus CIS pointer =         0x00000000
  sub system vendor ID =        0x1e4b
  sub system ID =               0x1202
  expansion ROM base address =  0x00000000
  interrupt line =              0xff
  interrupt pin =               0x01
  min Grant =                   0x00
  max Latency =                 0x00
StarFive # nvme help
nvme - NVM Express sub-system

Usage:
nvme scan - scan NVMe devices
nvme detail - show details of current NVMe device
nvme info - show all available NVMe devices
nvme device [dev] - show or set current NVMe device
nvme part [dev] - print partition table of one or all NVMe devices
nvme read addr blk# cnt - read `cnt' blocks starting at block
     `blk#' to memory address `addr'
nvme write addr blk# cnt - write `cnt' blocks starting at block
     `blk#' from memory address `addr'
StarFive # nvme scan
StarFive # nvme info
Device 0: Vendor: 0x1e4b Rev: SN10092  Prod: 0005053001292       
            Type: Hard Disk
            Capacity: 122104.3 MB = 119.2 GB (250069680 x 512)
StarFive # nvme detail
Blk device 0: Optional Admin Command Support:
        Namespace Management/Attachment: no
        Firmware Commit/Image download: yes
        Format NVM: yes
        Security Send/Receive: yes
Blk device 0: Optional NVM Command Support:
        Reservation: yes
        Save/Select field in the Set/Get features: yes
        Write Zeroes: yes
        Dataset Management: yes
        Write Uncorrectable: yes
Blk device 0: Format NVM Attributes:
        Support Cryptographic Erase: No
        Support erase a particular namespace: Yes
        Support format a particular namespace: Yes
Blk device 0: LBA Format Support:
Blk device 0: End-to-End DataProtect Capabilities:
        As last eight bytes: No
        As first eight bytes: No
        Support Type3: No
        Support Type2: No
        Support Type1: No
Blk device 0: Metadata capabilities:
        As part of a separate buffer: No
        As part of an extended data LBA: No
StarFive # nvme device 0

Device 0: Vendor: 0x1e4b Rev: SN10092  Prod: 0005053001292       
            Type: Hard Disk
            Capacity: 122104.3 MB = 119.2 GB (250069680 x 512)
... is now current device
StarFive # nvme part 0  

Partition Map for NVMe device 0  --   Partition Type: DOS

Part    Start Sector    Num Sectors     UUID            Type
  1     2048            250067632       ffbac99a-01     83
StarFive #
jrtc27 commented 1 year ago

Have you tried running the Debian or Armbian build? At least under Armbian, it is picking up NVMe.

Not sure what they are doing different from the vanilla buildroot image.

What does the OS have to do with whether the firmware works?

rmsyn commented 1 year ago

What does the OS have to do with whether the firmware works?

You're right, I skimmed, and thought you were running from Linux. My mistake.

Looks like the answer above more directly addresses your problem in U-Boot.

jrtc27 commented 1 year ago

U-Boot 2021.10 (Mar 24 2023 - 01:42:56 +0800), Build: jenkins-github_visionfive2-13

CPU: rv64imacu Model: StarFive VisionFive V2 DRAM: 8 GiB MMC: sdio0@16010000: 0, sdio1@16020000: 1 Loading Environment from SPIFlash... SF: Detected gd25lq128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB *** Warning - bad CRC, using default environment

StarFive EEPROM format v2

--------EEPROM INFO-------- Vendor : StarFive Technology Co., Ltd. Product full SN: VF7110B1-2250-D008E000-00001456 data version: 0x2 PCB revision: 0xb2 BOM revision: A Ethernet MAC0 address: 6c:cf:39:00:27:31 Ethernet MAC1 address: 6c:cf:39:00:27:32 --------EEPROM INFO--------

In: serial Out: serial Err: serial Model: StarFive VisionFive V2 Net: eth0: ethernet@16030000, eth1: ethernet@16040000 bootmode sd device 1 Failed to load 'uEnv.txt' Can't set block device Hit any key to stop autoboot: 0 StarFive # StarFive # StarFive # StarFive # pci enum starfive_pcie pcie@2C000000: Starfive PCIe bus probed. PCI: Failed autoconfig bar 10 StarFive # pci help pci - list and access PCI Configuration Space

Usage: pci [bus] [long]

  • short or long list of PCI devices on bus 'bus' pci enum
  • Enumerate PCI buses pci header b.d.f
  • show header of PCI device 'bus.device.function' pci bar b.d.f
  • show BARs base and size for device b.d.f' pci regions
  • show PCI regions pci display[.b, .w, .l] b.d.f [address] [# of objects]
  • display PCI configuration space (CFG) pci next[.b, .w, .l] b.d.f address
  • modify, read and keep CFG address pci modify[.b, .w, .l] b.d.f address
  • modify, auto increment CFG address pci write[.b, .w, .l] b.d.f address value
  • write to CFG address StarFive # pci 1 short Scanning PCI devices on bus 1 BusDevFun VendorId DeviceId Device Class Sub-Class

    01.00.00 0x1e4b 0x1202 Mass storage controller 0x08 StarFive # pci 1 long Scanning PCI devices on bus 1

Found PCI device 01.00.00: vendor ID = 0x1e4b device ID = 0x1202 command register ID = 0x0006 status register = 0x0010 revision ID = 0x01 class code = 0x01 (Mass storage controller) sub class code = 0x08 programming interface = 0x02 cache line = 0x08 latency time = 0x00 header type = 0x00 BIST = 0x00 base address 0 = 0x38000004 base address 1 = 0x00000000 base address 2 = 0x00000000 base address 3 = 0x00000000 base address 4 = 0x00000000 base address 5 = 0x00000000 cardBus CIS pointer = 0x00000000 sub system vendor ID = 0x1e4b sub system ID = 0x1202 expansion ROM base address = 0x00000000 interrupt line = 0xff interrupt pin = 0x01 min Grant = 0x00 max Latency = 0x00 StarFive # nvme help nvme - NVM Express sub-system

Usage: nvme scan - scan NVMe devices nvme detail - show details of current NVMe device nvme info - show all available NVMe devices nvme device [dev] - show or set current NVMe device nvme part [dev] - print partition table of one or all NVMe devices nvme read addr blk# cnt - read cnt' blocks starting at block blk#' to memory address addr' nvme write addr blk# cnt - writecnt' blocks starting at block blk#' from memory addressaddr' StarFive # nvme scan StarFive # nvme info Device 0: Vendor: 0x1e4b Rev: SN10092 Prod: 0005053001292
Type: Hard Disk Capacity: 122104.3 MB = 119.2 GB (250069680 x 512) StarFive # nvme detail Blk device 0: Optional Admin Command Support: Namespace Management/Attachment: no Firmware Commit/Image download: yes Format NVM: yes Security Send/Receive: yes Blk device 0: Optional NVM Command Support: Reservation: yes Save/Select field in the Set/Get features: yes Write Zeroes: yes Dataset Management: yes Write Uncorrectable: yes Blk device 0: Format NVM Attributes: Support Cryptographic Erase: No Support erase a particular namespace: Yes Support format a particular namespace: Yes Blk device 0: LBA Format Support: Blk device 0: End-to-End DataProtect Capabilities: As last eight bytes: No As first eight bytes: No Support Type3: No Support Type2: No Support Type1: No Blk device 0: Metadata capabilities: As part of a separate buffer: No As part of an extended data LBA: No StarFive # nvme device 0

Device 0: Vendor: 0x1e4b Rev: SN10092 Prod: 0005053001292
Type: Hard Disk Capacity: 122104.3 MB = 119.2 GB (250069680 x 512) ... is now current device StarFive # nvme part 0

Partition Map for NVMe device 0 -- Partition Type: DOS

Part Start Sector Num Sectors UUID Type 1 2048 250067632 ffbac99a-01 83 StarFive #

Hm. Same board but different outcomes. What SSD is that? That vendor+device is for a Chinese brand of controller I've never heard of before. I'm just trying to use a boring, ubiquitous Samsung 970 Plus that, in general, is the kind of device that just works everywhere and always seems to have deals on (and yes, I'm aware that the antiquated physical link is going to be a bottleneck here given it's a paltry Gen 2x1).

MichaelZhuxx commented 1 year ago

@jrtc27 oh, the nvme ssd is connected by pcie bus 1

In:    serial
Out:   serial
Err:   serial
Model: StarFive VisionFive V2
Net:   eth0: ethernet@16030000, eth1: ethernet@16040000
bootmode sd device 1
Card did not respond to voltage select! : -110
Hit any key to stop autoboot:  0
StarFive # pci enum
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
StarFive # pci
Scanning PCI devices on bus 0
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x1556     0x1111     Bridge device           0x04
StarFive # pci 1 short
Scanning PCI devices on bus 1
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
01.00.00   0x15b7     0x5019     Mass storage controller 0x08
jrtc27 commented 1 year ago

@jrtc27 oh, the nvme ssd is connected by pcie bus 1

In:    serial
Out:   serial
Err:   serial
Model: StarFive VisionFive V2
Net:   eth0: ethernet@16030000, eth1: ethernet@16040000
bootmode sd device 1
Card did not respond to voltage select! : -110
Hit any key to stop autoboot:  0
StarFive # pci enum
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
StarFive # pci
Scanning PCI devices on bus 0
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x1556     0x1111     Bridge device           0x04
StarFive # pci 1 short
Scanning PCI devices on bus 1
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
01.00.00   0x15b7     0x5019     Mass storage controller 0x08
In:    serial
Out:   serial
Err:   serial
Model: StarFive VisionFive V2
Net:   eth0: ethernet@16030000, eth1: ethernet@16040000
switch to partitions #0, OK
mmc1 is current device
found device 1
bootmode flash device 1
** Invalid partition 3 **
Couldn't find partition mmc 1:3
Can't set block device
** Invalid partition 3 **
Couldn't find partition mmc 1:3
Can't set block device
Hit any key to stop autoboot:  0 
StarFive # pci enum
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
StarFive # pci
Scanning PCI devices on bus 0
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x1556     0x1111     Bridge device           0x04
StarFive # pci 1 short
Scanning PCI devices on bus 1
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
StarFive #

I had tried nvme scan before and it didn't find anything, so unsurprisingly bus 1 is missing the NVMe drive.

MichaelZhuxx commented 1 year ago

@jrtc27 oh, the nvme ssd is connected by pcie bus 1

In:    serial
Out:   serial
Err:   serial
Model: StarFive VisionFive V2
Net:   eth0: ethernet@16030000, eth1: ethernet@16040000
bootmode sd device 1
Card did not respond to voltage select! : -110
Hit any key to stop autoboot:  0
StarFive # pci enum
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
StarFive # pci
Scanning PCI devices on bus 0
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x1556     0x1111     Bridge device           0x04
StarFive # pci 1 short
Scanning PCI devices on bus 1
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
01.00.00   0x15b7     0x5019     Mass storage controller 0x08
In:    serial
Out:   serial
Err:   serial
Model: StarFive VisionFive V2
Net:   eth0: ethernet@16030000, eth1: ethernet@16040000
switch to partitions #0, OK
mmc1 is current device
found device 1
bootmode flash device 1
** Invalid partition 3 **
Couldn't find partition mmc 1:3
Can't set block device
** Invalid partition 3 **
Couldn't find partition mmc 1:3
Can't set block device
Hit any key to stop autoboot:  0 
StarFive # pci enum
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
StarFive # pci
Scanning PCI devices on bus 0
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x1556     0x1111     Bridge device           0x04
StarFive # pci 1 short
Scanning PCI devices on bus 1
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
StarFive #

I had tried nvme scan before and it didn't find anything, so unsurprisingly bus 1 is missing the NVMe drive.

does it work in linux? do you have try a more powerful DC adapter, like 5V@3A or 12V@1.5A?

jrtc27 commented 1 year ago

Hm, after taking the drive out to verify it wasn't dead in an NVMe enclosure and putting it back things are now working. Not sure how it's possible to have the drive screwed in without making good contact but I guess somehow reseating it fixed things... odd. (It was definitely making contact on at least the power pins before though given how hot it would get despite not being discovered)