Host doesn't succeed in enumerating devices

[rdolbeau]

I'm trying to add a USB host to my board using a @Dolu1990 pmod (https://github.com/Dolu1990/pmod_usb_host_x4 , thanks :-) ).

Using a 48 MHz input clock, the design did not pass timing (~2.5 ns NS on many endpoints). Switching to a 60 MHz all but solved the problem (I still have a couple of endpoints with <0.05ns negative slack), so that's what is used below. Not sure if that's legit but lowering the clock triggered some assert so I couldn't do that... raising it what the other option, no idea why it worked :-).

Only a single USB connector is connected (ports_count=1), the closest to the board on the Pmod (J5), so to {pmod}:3 (for dp, top line closest to ground pins, 7 in KiCad) and {pmod}:7 (for dn, bottom line closest to ground pins, 8 in KiCad).

Linux sees the USB Host just fine after adding it to the DTB:

root@buildroot:~# lsusb                                                                   
Bus 001 Device 001: ID 1d6b:0001
root@buildroot:~# cat /proc/interrupts                                                    
           CPU0       CPU1       CPU2       CPU3                                          
  2:        681          0          0          0  SiFive PLIC   2  eth0                   
  3:       2014          0          0          0  SiFive PLIC   3  litex-mmc              
  4:       2419          0          0          0  SiFive PLIC  16  ohci_hcd:usb1          
  5:     282842     282832     282832     282829  RISC-V INTC   5  riscv-timer            
(...)

However neither cold-plugin nor hot-plugin will work (here hot for a keyboard, also tried a mouse and a self-powered USB hub):

[   22.709622] usb 1-1: new low-speed USB device number 2 using ohci-platform             
[   24.585531] usb 1-1: device descriptor read/64, error -71                              
[   26.313545] usb 1-1: device descriptor read/64, error -71                              
[   26.577582] usb 1-1: new low-speed USB device number 3 using ohci-platform             
[   28.457590] usb 1-1: device descriptor read/64, error -71                              
[   30.185591] usb 1-1: device descriptor read/64, error -71                              
[   30.294008] usb usb1-port1: attempt power cycle                                        
[   30.525611] usb 1-1: new low-speed USB device number 4 using ohci-platform             
[   32.846035] usb 1-1: device descriptor read/8, error 0                                 
[   34.573845] usb 1-1: device descriptor read/8, error 0                                 
[   34.837576] usb 1-1: new low-speed USB device number 5 using ohci-platform             
[   36.877820] usb 1-1: device descriptor read/8, error 0                                 
[   38.605885] usb 1-1: device descriptor read/8, error 0                                 
[   38.713986] usb usb1-port1: unable to enumerate USB device                             

I tried changing /sys/module/usbcore/parameters/old_scheme_first to Y but that merely changes the order of the error (the read/8 goes first, read/64 second).

I also tried with the 48 MHz clock despite the timing failures, and got the same errors.

The Pmod connector has the pins 1 and 2 of J3 connected to the VIN (5v) / GND of the board; neither of the other pins are connected (they are pulled up in the Pmod I assumed that was enough). The (optical) mouse lights up as expected.

Any suggestion what I might have done wrong ?

[Dolu1990]

Hi,

Using a 48 MHz input clock, the design did not pass timing (~2.5 ns NS on many endpoints)

my guess is that you need to specify to the xilinx synthesis tool that the 48 MHZ clock is asyncronus with the main clock. This will relax timings quite a lot.

For instance on Saxon i have : set_clock_groups -asynchronous -group clocking_pll_CLKOUT0 -group clocking_pll_CLKOUT1

Only a single USB connector is connected (ports_count=1), the closest to the board on the Pmod (J5), so to {pmod}:3 (for dp, top line closest to ground pins, 7 in KiCad) and {pmod}:7 (for dn, bottom line closest to ground pins, 8 in KiCad).

Seems weird, i mean, with the actual config of this repository, the pmod you have should work out of the box on JA, using the lower usb port of the dual stacked connector.

I'm not sure to understand what you changed to : https://github.com/enjoy-digital/litex_vexriscv_smp_usb_host/blob/master/digilent_arty.py#L141 ?

Also be aware of the mirroring made on the kikad stuff, the pmod connector, even if placed on the bottom of the kikad is solder on the top to mirror stuff.

I tried changing /sys/module/usbcore/parameters/old_scheme_first to Y but that merely changes the order of the error (the read/8 goes first, read/64 second).

Normaly, it should have worked out of the box.

The Pmod connector has the pins 1 and 2 of J3 connected to the VIN (5v) / GND of the board;

Normaly, you can keep everything floating, excepted 5V that need to be connected to the 5V of the board.

[Dolu1990]

Hoo also, can you check if you get some interrupts out of cat /dev/interrupts for the usb ?

[rdolbeau]

@Dolu1990 I'm not using a Digilent Arty (I don't have one), but porting to my Qmtech Wukong, it's also an Artix 7 100T (SG -2). I had to do some copy/paste from this repo to my Litex to get everything connected...

In litex/soc/integration/soc.py, I created a add_usb_host():

    # Add USB Host
    def add_usb_host(self, name="usb_host", pads=None, ports_count=1, phy_frequency=48000000):
        from usb_host import USBHost
        self.submodules.usb_host = USBHost(self.platform, pads, ports_count, phy_frequency)
        usb_host_region_size = 0x100000
        usb_host_region = SoCRegion(origin=self.mem_map.get(name, None), size=usb_host_region_size, cached=False)
        self.bus.add_slave("usb_host_ctrl", self.usb_host.wb_ctrl, region=usb_host_region)
        self.dma_bus.add_master("usb_host_dma", master=self.usb_host.wb_dma)

while the target (litex_boards/targets/qmtech_wukong.py) sets things up:

       if usb:
            self.add_usb_host(pads = platform.request("usb_pmod"), ports_count=1, phy_frequency=60000000)
            self.comb += self.cpu.interrupt[16].eq(self.usb_host.interrupt)

The "usb_pmod" is in the platform (litex_boards/platforms/qmtech_wukong.py), I double-checked the pins (and they worked for my PS2 host with a PS2 pmod).

#Dolu1990's USB Pmod on J10, https://github.com/Dolu1990/pmod_usb_host_x4
def usb_pmod_io(pmod):
    return [
        ("usb_pmod", 0,
#            Subsignal("dp", Pins(f"{pmod}:0 {pmod}:1 {pmod}:2 {pmod}:3")),
#            Subsignal("dm", Pins(f"{pmod}:4 {pmod}:5 {pmod}:6 {pmod}:7")),
            Subsignal("dp", Pins(f"{pmod}:2")),
            Subsignal("dm", Pins(f"{pmod}:6")),
            IOStandard("LVCMOS33"),
        )
]
_usb_pmod_io = usb_pmod_io("j10") # Dol1990's USB Pmod on J10

I'm currently re-synthesizing to check with a different port, just in case. I'll try the bottom front one (0&4) next - at first I though I could use port_counts=4 but at this time the Litex wrapper doesn't connect more than one port.

There's definitely some interrupts going on; booting with nothing plugged in I have 0 interrupts; after plugging a keyboard and waiting for the enumeration to fail I see 624 interrupts. After unplugging the keyboard, 625. Linux also can tell the difference between 'new high-speed' (the hub) and 'new low-speed' (keyboard, mouse) when it fails.

As for the pmod, you definitely soldered the connector on the right (up) side :-) [EDIT] Mmm, could I have swapped D+ and D- because of it ? The D+ are further away from the edge so should always be on the top line of the Pmod connector no ? That's why I tried the J4 and J5 connector first - they are in the middle of the Pmod connector so even mirrored they can't be confused with PWR/GND, only with each other... and I guess I got that right as when I plug something on a different connector nothing happens.

For the sake of completeness, the DTS defines the USB as:

            usb0: mac@80000000 {
                compatible = "generic-ohci";
                reg = <0x80000000 0x1000>;
                interrupts = <16>;
                status = "okay";
            };

0x80000000 is the address for the usb_host_ctrl as outputed by litex in csr.csv (memory_region,usb_host_ctrl,0x80000000,1048576,io). It's handed to me by SoCRegion() instead of being hardwired.

I'm also using a dedicated MMCM to create the cd_usb at 60 MHz (or 48 MHz).

[rdolbeau]

Could I need SLEW=FAST ? It's not in the Digilent Arty code so I didn't put it.

[Dolu1990]

Could I need SLEW=FAST ? It's not in the Digilent Arty code so I didn't put it.

I don't think it is required.

[EDIT] Mmm, could I have swapped D+ and D- because of it ? The D+ are further away from the edge so should always be on the top line of the Pmod connector no ? That's why I tried the J4 and J5 connector first - they are in the middle of the Pmod connector so even mirrored they can't be confused with PWR/GND, only with each other... and I guess I got that right as when I plug something on a different connector nothing happens.

If the OHCI detect the device speed properly, that mean that pins aren't inverted, as inverted pins would change the speed of the device. Unless there is some floating pins.

I'm also using a dedicated MMCM to create the cd_usb at 60 MHz (or 48 MHz).

Be carefull here, the USB OHCI netlist is crafted to work at a fixed frequancy for the phy side. If you look at the netlist name, there is the frequancy inside.

The D+ are further away from the edge

Which edge ^^ ?

port_counts=4

I would say in a first time stay at portt_counts=1, as this is the only one tested on litex for now.

I guess it's time to scope some USB frame :D, that's the only way to keep sanity XD Do you have some ways of capturing long sequances with a logic analyser ?

[rdolbeau]

Litex regenerate the verilog, so it is using (properly) UsbOhciWishbone_Dw32_Pc1_Pf60000000.v.

For the D+/D- position, the edge with the pmod connector - D- is closest (so 'bottom' row, when the pmod is viewed with the USB connectors are on the upside) and D+ furthest from that edge (so the 'up' row).

I'm now only truing with ports_count =1.

I have a logic analyzer (Kingst LA2016, somewhat basic), but not a scope. I was hoping this was a 'dumb mistake' with known symptoms, but I guess not so yes I'll have to try examining what's going on in the signals. Fortunately it seems pulseview understands USB.

[Dolu1990]

I can also take a look at the signal wave, basicaly idealy, a capture starting when device is unpluged, and ending after 1 few seconds the device was plugged. Starting with low speed device first, as it will require less sampling in the wave ^^

[Dolu1990]

About the connections, if both speed of keyboard and hub are detected properly, i guess it is good on that side.

[Dolu1990]

@rdolbeau One thing is also that the OHCI DMA access should be coherent with the CPU accesses. So, the DMA wishbone bus should be connected to the memory system via the VexRiscv cluster DMA port (not directly to litedram) I'm not sure how this is set ^^

[rdolbeau]

@Dolu1990 The coherent DMA was enabled by default in the repo-specific wrapper for VexRiscv, but I'm running SMP so the coherent DMA is on by default as well. I don't see anything beyond that in the code.

... was the USB host core tested with SMP or only with UP ?

[Dolu1990]

... was the USB host core tested with SMP or only with UP ?

On litex, it was only tested with 1 cores with coherency. On SaxonSoc, it was tested with two cores.

[ 24.585531] usb 1-1: device descriptor read/64, error -71

mean #define EPROTO 71 / Protocol error /

My guess is that we may see things on the scope

[rdolbeau]

This particular issue was related to the DMA width; the example in this repo uses 32 bits, which I replicated, but my SoC uses 64 bits (forced by the use of the optional double-precision FPU in VexRiscv).

Forcing the DMA to 64 bits in Litex solves that problem for me.

[enjoy-digital]

Thanks @rdolbeau for the feedback, I'll add assertions for this when integrating the USB in LiteX to the to make sure the SoC is is not built silently with the wron't DMA's data-width.

[rdolbeau]

@enjoy-digital It should be possible to simply specify the soc.dma_bus width in the usb_host?

Anyway at this stage using the 64-bits DMA causes another issue where most devices are still not enumerated (but in a different way), even though e.g. a hub works fine (and then devices connected to that hub have the same issue). @Dolu1990 has been able to reproduce the problem.

I've also managed to get all 4 ports from the Pmod to be connected to the usb_host (and all recognize a hub), but the code is a bit ugly.

[enjoy-digital]

@rdolbeau: OK thanks. I started moving things to LiteX, so we could integrate changes/fixes there when the issue will be sorted out.

[Dolu1990]

@enjoy-digital

Found something about the 32 bits OHCI pluged on the 64 bits memory system.

It seems the main issue there is that the bridge used in litex to translate bus width do not handle the wishbone SEL and instead assume all bits set.

Bellow in yellow => OHCI DMA output in red => VexRIscv cluster DMA input wishbone sel isn't propagated

When running simulation, the sel signal get out of the OHCI, but endup not being used very far (until simsoc_dma_bus_dma_bus_sel) and do not goes in simsoc_cpu_dma_bus_XXX which is instead generated that way :

always @(*) begin
    simsoc_cpu_dma_bus_sel = 8'd0;
    case (simsoc_dma_bus_dma_bus_adr[0])
        1'd0: begin
            simsoc_cpu_dma_bus_sel[3:0] = 4'd15;
        end
        1'd1: begin
            simsoc_cpu_dma_bus_sel[7:4] = 4'd15;
        end
    endcase
end

while the data are handeler well :

always @(*) begin
    simsoc_cpu_dma_bus_dat_w = 64'd0;
    case (simsoc_dma_bus_dma_bus_adr[0])
        1'd0: begin
            simsoc_cpu_dma_bus_dat_w[31:0] = simsoc_dma_bus_dma_bus_dat_w;
        end
        1'd1: begin
            simsoc_cpu_dma_bus_dat_w[63:32] = simsoc_dma_bus_dma_bus_dat_w;
        end
    endcase
end
always @(*) begin
    simsoc_dma_bus_dma_bus_dat_r = 32'd0;
    case (simsoc_dma_bus_dma_bus_adr[0])
        1'd0: begin
            simsoc_dma_bus_dma_bus_dat_r = simsoc_cpu_dma_bus_dat_r[31:0];
        end
        1'd1: begin
            simsoc_dma_bus_dma_bus_dat_r = simsoc_cpu_dma_bus_dat_r[63:32];
        end
    endcase
end

@enjoy-digital Do you know where that width adapter is designed ? It is likely a simple fix :)

[Dolu1990]

@rdolbeau Fixed 32bits->64bits and 64bits->64bits with https://github.com/enjoy-digital/litex/commit/a064e9d0486821f09aa477834edffdabc62e0db4

Now everything should work fine :)

[rdolbeau]

Yes indeed now it works fine; enabled all 4 ports as available on the Pmod and:

lsusb -tvvv
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ohci-platform/4p, 12M
    ID 1d6b:0001  
    /sys/bus/usb/devices/usb1  /dev/bus/usb/001/001
    |__ Port 1: Dev 3, If 0, Class=, Driver=usbhid, 1.5M
        ID 04d9:1702  
        /sys/bus/usb/devices/1-1  /dev/bus/usb/001/003
    |__ Port 1: Dev 3, If 1, Class=, Driver=usbhid, 1.5M
        ID 04d9:1702  
        /sys/bus/usb/devices/1-1  /dev/bus/usb/001/003
    |__ Port 2: Dev 4, If 0, Class=, Driver=usbhid, 1.5M
        ID 1bcf:0007  
        /sys/bus/usb/devices/1-2  /dev/bus/usb/001/004
    |__ Port 4: Dev 2, If 0, Class=, Driver=hub/4p, 12M
        ID 1a40:0101  
        /sys/bus/usb/devices/1-4  /dev/bus/usb/001/002
        |__ Port 4: Dev 5, If 0, Class=, Driver=usbhid, 1.5M
            ID 04f3:0210  
            /sys/bus/usb/devices/1-4.4  /dev/bus/usb/001/005

keyboard (port 1), mouse (port 2), hub (port 4) with a second mouse. And they work in X11 too.

... not sure why my build of usbutils/libusb doesn't display as much information as on other platform.

Thanks :-)

[Dolu1990]

@rdolbeau

with a second mouse

Rawrrrrrrrrrrr XD

not sure why my build of usbutils/libusb doesn't display as much information as on other platform.

I have the same issue ^^ Very likely the lsusb binary need to be compiled with specific flags to be as verbose than the ones in regular distributions

For me, it show even less information XD

[rdolbeau]

For me, it show even less information XD

There's a lsusb in buildroot, but it's minimalistic (one of the many guise of busybox). I'm using the one from usbutils, which requires libusb and udev (I used eudev as I don't want systemd). I added the usb.ids file, but still no luck.

edit: newer versions of usbutils use HWDB, i needed to create the binary version with udevadm hwdb --update:

(root)buildroot:~# lsusb -tvvv
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ohci-platform/4p, 12M
    ID 1d6b:0001 Linux Foundation 1.1 root hub
    /sys/bus/usb/devices/usb1  /dev/bus/usb/001/001
    |__ Port 1: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
        ID 04d9:1702 Holtek Semiconductor, Inc. Keyboard LKS02
        /sys/bus/usb/devices/1-1  /dev/bus/usb/001/003
    |__ Port 1: Dev 3, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
        ID 04d9:1702 Holtek Semiconductor, Inc. Keyboard LKS02
        /sys/bus/usb/devices/1-1  /dev/bus/usb/001/003

edit 2: @Dolu1990 buildroot has a package for ubsutils, presumably that would create a properly installed lsusb with more features than busibox's.