1000001101000 / Debian_on_Buffalo

Tools for Installing/Running Debian on Buffalo ARM based Linkstation/Terastation/Kurobox/Cloudstor devices.
325 stars 40 forks source link

LS441DE #8

Open jcmoore1977 opened 5 years ago

jcmoore1977 commented 5 years ago

Still having network issues. Sorry It has been a while. I had to replace my hard drive PCB. It took a crash. I have used the updated files you put on here, now all the network does is blink then shuts off. I can not even start installer. The zip file I received from you worked for a moment, then went back to shutting off. Any suggestions. I have no stock firmware on drive it is blank and formatted with the files in /

1000001101000 commented 5 years ago

I've made a number of improvements to the ls441 dtb since we last worked on it. I learned that the ls400 devices use one of the pins from the ethernet chip (normally meant for led3) to control whether the device shuts down or restarts. This might have explained the problems you were seeing (maybe).

Now that i've accounted for that and a few other things i'm hoping we'll have better luck.

I'm a little unclear about how far you're getting right now, it sounds like it's failing to boot at all.

which installer are you currently trying? what are the leds on the front doing when you try? how big is the partition you're trying to boot from?

jcmoore1977 commented 5 years ago

Let me see if I can make a video and share it with you. Give me about 20 mins or so.

jcmoore1977 commented 5 years ago

Here is a photo of the files I use. The one highlighted and the initrd at the top.

https://1drv.ms/u/s!AsfCHdLfE-658Q9Z1SxgfL_fQmc-

jcmoore1977 commented 5 years ago

Here is the video of what the system does upon power on. I went back and forth from front to back where the network is plugged in. At the end you can see all lights solid and net work is off.

https://1drv.ms/v/s!AsfCHdLfE-658RBbtqLihW6wK2QZ

jcmoore1977 commented 5 years ago

Partition in about 200MB on boot

jcmoore1977 commented 5 years ago

I get this error when I can finally get into installer shell and type exit

Jan 1 00:00:30 sshd[1041]: Server listening on :: port 22. Jan 1 00:00:30 apt-install: Queueing package openssh-server for later installation Jan 1 00:00:37 kernel: [ 37.785266] mvneta d0074000.ethernet eth0: Link is Down Jan 1 00:00:39 kernel: [ 39.862199] mvneta d0074000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 1 00:00:59 kernel: [ 59.625323] mvneta d0074000.ethernet eth0: Link is Down Jan 1 00:01:00 kernel: [ 60.662152] mvneta d0074000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 1 00:01:28 sshd[1056]: Accepted password for installer from 192.168.0.13 port 65123 ssh2 Jan 1 00:01:28 sshd[1057]: lastlog_openseek: Couldn't stat /var/log/lastlog: No such file or directory Jan 1 00:01:28 sshd[1057]: lastlog_openseek: Couldn't stat /var/log/lastlog: No such file or directory Jan 1 00:01:28 debconf: Setting debconf/language to en Jan 1 00:02:21 kernel: [ 141.785336] mvneta d0074000.ethernet eth0: Link is Down Jan 1 00:02:23 kernel: [ 143.862244] mvneta d0074000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 1 00:02:26 kernel: [ 146.985336] mvneta d0074000.ethernet eth0: Link is Down Jan 1 00:02:29 kernel: [ 149.062189] mvneta d0074000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 1 00:02:49 kernel: [ 169.865259] mvneta d0074000.ethernet eth0: Link is Down Jan 1 00:02:51 kernel: [ 171.942247] mvneta d0074000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 1 00:03:00 kernel: [ 180.265201] mvneta d0074000.ethernet eth0: Link is Down Jan 1 00:03:01 kernel: [ 181.302203] mvneta d0074000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 1 00:03:14 kernel: [ 194.825204] mvneta d0074000.ethernet eth0: Link is Down Jan 1 00:03:15 kernel: [ 195.862269] mvneta d0074000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 1 00:03:18 kernel: [ 198.985257] mvneta d0074000.ethernet eth0: Link is Down Jan 1 00:03:20 kernel: [ 200.022213] mvneta d0074000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 1 00:03:18 kernel: [ 198.985257] mvneta d0074000.ethernet eth0: Link is Down

1000001101000 commented 5 years ago

alright, so we're using the current stretch installer (that's what i'd recommend). it looks like it's booting, the light changing to solid indicates it booted something.

the ethernet led turns off at the point i'd expect it to turn solid, I've got it configured as follows: LED0: On - Link, Blink - Activity, Off - No Link

I'm going to try with mine and confirm what mine does (it's been a while since I looked at it). I want to be sure the current files are working for me and the led behavior is how I remember it before we get too far.

did you ever end up getting a usb nic?

1000001101000 commented 5 years ago

oh, nice. that's helpful. looks like the link keeps going up and down for some reason.

back when we first worked on this that was probably triggering the device to shutdown too.... at least we're past that issue.

jcmoore1977 commented 5 years ago

Yes I have order one. It will be here before the end of the month. When HDD board went out my concern was getting everything off drive first.

1000001101000 commented 5 years ago

understandible.

have you tried using a different ethernet cable/switch/etc?

jcmoore1977 commented 5 years ago

Yes, I have recently replace all the network devices cables and switches in my entire house. The stock firmware still works perfectly.

1000001101000 commented 5 years ago

I figured you had but I thought I'd check. I checked the logs on a couple of my devices and didn't see any disconnections like yours. The fact that the stock firmware seems stable makes me hopeful there is some kind of solution though the fact that my ~10 devices don't do this nor do the devices of anyone else I've interacted with make me think there is something physically different about your device.

I'm going to do some research and see if there is a setting to make the driver and/or ethernet chip less sensitive to disconnects or something of that nature. I'll also take a look at the buffalo kernel source code again (I remember they had the code for the whole led2 as restart indicator thing, I don't recall what else they customized).

There are a few things you could try/look at:

  1. You could try the buster installer. It's possible the newer mvneta driver handles this better
  2. you could take a look at the logs when running the stock firware and confirm whether you see those disconnects or not
  3. you could try pinging the device continously for a while and see if packets are getting dropped and what kind of latency you get.

one other thing to note: I recently changed the installer configuration to auto detect the network adapter rather than just assume it will be eth0 (I've added support for the ts3000 series which have 2 ethernet ports). This should help when you try the usb3 adapter but also means it might not detected the on-board ethernet if it thinks the link is down when the auto detect runs.

jcmoore1977 commented 5 years ago

yeah buster doesn't work at all. I have been reading up on this even though it is for my current kernel 3.3.4 stock.

I see some things are the same and some are different. Your thoughts.

/ SPDX-License-Identifier: GPL-2.0 /*

/*

include "armada-370.dtsi"

include <dt-bindings/gpio/gpio.h>

include <dt-bindings/input/input.h>

/ { chosen { stdout-path = "serial0:115200n8"; };

memory@0 {
    device_type = "memory";
    reg = <0x00000000 0x20000000>; /* 512 MB */
};

soc {
    ranges = <MBUS_ID(0xf0, 0x01) 0 0xf1000000 0x100000
          MBUS_ID(0x01, 0xe0) 0 0xfff00000 0x100000>;

    internal-regs {
        coherency-fabric@20200 {
            broken-idle;
        };

        serial@12000 {
            status = "okay";
        };

        ethernet@74000 {
            status = "okay";
            pinctrl-0 = <&ge1_rgmii_pins>;
            pinctrl-names = "default";
            phy = <&phy0>;
            phy-mode = "rgmii-id";
        };

        usb@50000 {
            status = "okay";
        };
    };
};

regulators {
    compatible = "simple-bus";
    #address-cells = <1>;
    #size-cells = <0>;

    regulator@0 {
        compatible = "regulator-fixed";
        reg = <0>;
        regulator-name = "USB Power";
        regulator-min-microvolt = <5000000>;
        regulator-max-microvolt = <5000000>;
        regulator-always-on;
        regulator-boot-on;
        gpio = <&gpio1 27 GPIO_ACTIVE_LOW>;
    };
    regulator@1 {
        compatible = "regulator-fixed";
        reg = <1>;
        regulator-name = "SATA0 power";
        regulator-min-microvolt = <5000000>;
        regulator-max-microvolt = <5000000>;
        enable-active-high;
        regulator-always-on;
        regulator-boot-on;
        gpio = <&gpio1 18 GPIO_ACTIVE_HIGH>;
    };
};

gpio-keys {
    compatible = "gpio-keys";
    #address-cells = <1>;
    #size-cells = <0>;

    power {
        label = "Power button";
        linux,code = <KEY_POWER>;
        gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>;
        debounce-interval = <100>;
    };
    reset {
        label = "Reset Button";
        linux,code = <KEY_RESTART>;
        gpios = <&gpio1 23 GPIO_ACTIVE_LOW>;
        debounce-interval = <100>;
    };
    button {
        label = "USB VBUS error";
        linux,code = <KEY_UNKNOWN>;
        gpios = <&gpio1 21 GPIO_ACTIVE_LOW>;
        debounce-interval = <100>;
    };
};

gpio-leds {
    compatible = "gpio-leds";

    red-sata0 {
        label = "cumulus:red:sata0";
        gpios = <&gpio1 26 GPIO_ACTIVE_HIGH>;
        default-state = "off";
    };
};

gpio_poweroff {
    compatible = "gpio-poweroff";
    gpios = <&gpio1 25 GPIO_ACTIVE_HIGH>;
};

};

&pciec { status = "okay";

/* USB 3.0 Bridge ASM1042A */
pcie@1,0 {
    status = "okay";
};

};

&mdio { pinctrl-0 = <&mdio_pins>; pinctrl-names = "default";

phy0: ethernet-phy@0 {
    reg = <0>;
};

};

&pinctrl { pinctrl-0 = <&sata_led_pin>; pinctrl-names = "default";

sata_led_pin: sata-led-pin {
    marvell,pins = "mpp60";
    marvell,function = "sata0";
};
gpio_led_pin: gpio-led-pin {
    marvell,pins = "mpp60";
    marvell,function = "gpio";
};

};

&spi0 { status = "okay"; pinctrl-0 = <&spi0_pins2>; pinctrl-names = "default";

spi-flash@0 {
    #address-cells = <1>;
    #size-cells = <1>;
    /* MX25L8006E */
    compatible = "mxicy,mx25l8005", "jedec,spi-nor";
    reg = <0>; /* Chip select 0 */
    spi-max-frequency = <50000000>;

    partition@0 {
        label = "u-boot";
        reg = <0x0 0x100000>;
    };
};
1000001101000 commented 5 years ago
  1. What do you mean when you say buster doesn’t work at all? I’ve run buster on mine for a while without any particular issues.

  2. That’s a device tree for another armada370 device you can look at them all in the kernel source under arch/arm/boot/dts. You can find information about the various device bindings in /Documentation

You could certainly try adjusting the ethernet settings by modifying the device tree though I dont know of anything specific to try (i’d offer it up if I did).

The datasheet for the ethernet chip is available too, if you find a setting you want to change you can set it using phytool and make it permanent via adding it to the device tree.

Unfortunately I can’t help much with that line of reasearch, i’ve got 15 different devices (i just counted) running that ethernet chip and none are having this problem.

1000001101000 commented 5 years ago

I’ve got a link to the datasheet for the ethernet chip and all the reset of the hardware info I put together here: https://buffalonas.miraheze.org/wiki/Linkstation_LS441D

If you want an example of manually setting registers on the chip check out the ls400 restart script in the tools section.

You can see how to set that same register in the device tree under the ethernet binding

jcmoore1977 commented 5 years ago

Buster will boot device, but the network shuts off immediately. Can’t even login to it. It’s just very strange how stock version runs perfectly. 2 days now and no network shut down. I wish I could use the original buffalo uimage and then boot the stretch initrd file

1000001101000 commented 5 years ago

yeah. I want to know why your device behaves like this when none of the others seem to and I also want to know what's different about the stock firmware that doesn't trigger it.

I'd really like to see your dmesg output from the stock firmware to see if it's happening but the software handles it better or if their kernel just somehow avoids the issue.

you could probably get phytool running on the stock firmware and use it to dump all the register values which we could then compare to the debian values for a comparison.

you could also try running Jessie and see if the 3.16 kernel works any better for you.

you could also try running jessie with the buffalo kernel, I did with my ls421de for a while when I was first experimenting with this stuff. you can see an example of how to create an initrd for that type of setup in this project: https://github.com/rogers0/OpenLinkstation

1000001101000 commented 5 years ago

one other thing i'd be curious about is whether your device might have a different revision of main board or something like that. If you feel like taking your apart you can compare to some fairly high resolution pictures of mine at: https://buffalonas.miraheze.org/wiki/Linkstation_LS441D

jcmoore1977 commented 5 years ago

Okay I will do the things you suggest later this afternoon. I wil definitely take apart so we can see the board

1000001101000 commented 5 years ago

nice! I think verifying whether the board is different will be good, though I don't really know what kind of differences to look for, definitely make sure to get a good picture of the lettering on the Ethernet chip.

The biggest question I have is whether you'll see those constant up/down messages in the logs of the stock firmware or not. If so then buffalo's firmware must have a way of handling them more gracefully and we can try to something similar.

If not then it's probably something more subtle about how the device is initialized or how that older driver worked. If so you should be able to dump a bunch of info and register values using ethtool and phytool (though getting those working under the stick firmware could be challenging).

jcmoore1977 commented 5 years ago

Plus my usb network stick will be here by this weekend. So we can dig in for sure then.

jcmoore1977 commented 5 years ago

Here are this photos of my MB. https://1drv.ms/f/s!AsfCHdLfE-658RooP1DS_BNKluou

1000001101000 commented 5 years ago

Hmm one is a -ca and one is a -cb. I have no idea if that means anything but it’s interesting. Definitely the same ethernet chip.

jcmoore1977 commented 5 years ago

Yes all the components looked the same. My adapter should arrive tomorrow, so I will be digging into this, this weekend. I will post some dmesg on stock and if I can get Debian installer to stay active post those as well

jcmoore1977 commented 5 years ago

Okay before I plug in the debian installer here is the complete dmesg from stock firmware. dmesg-stock.txt

I also ran the dmesg command 10 mins later and it is the same thing as the above post. As you can see no network issues.

jcmoore1977 commented 5 years ago

So far no issues with network, using usb nic. I have been up and running for 10 mins now.

jcmoore1977 commented 5 years ago

Also is there a reason why your its files are different for stretch vs buster?

1000001101000 commented 5 years ago

nice! I suppose the fact that that is stable lets us further narrow down the issue to the built in nic rather than something that coincidentally caused the link to go up and down.

your dmesg output seems to confirm that the problem isn't occuring in the stock firmware. That raises more questions than it answers but it's good to know. something is different about your device than all the others we've seen and whatever that difference is the stock firmware is fine with... somehow.

There are a few directions I can think of looking but they get progressively more complicated and I'm not sure what our best bet is.

I think using ethtool to look at all the network settings both under debian and stock might show if they're setting some sort of option differently.

if that doesn't give us any hints I could write a script using phytool to dump all the register values in the ethernet chip and we could compare them.

1000001101000 commented 5 years ago

There have been a number of changes to the device-tree specification and to the different bindings which require the device-trees to be updated to match the kernel they go with.

This makes keeping up with the testing branch a pain, I'm not sure if i'll keep posting testing images like I have been with Buster or not. Hopefully there will be less changes to account for this time around.

jcmoore1977 commented 5 years ago

Okay, I have installed the stretch software using the usb nic. I want to let it run for a bit to see what happens with this installed. Then I will use to built-in network. I want to test a theory I have.

jcmoore1977 commented 5 years ago

debian with usb nic plugged in. message.txt

1000001101000 commented 5 years ago

theories are good. mine are getting harder and harder to test.

I should have told you to get a usb3 gigabit adapter. The one you got will be good for testing (and good for testing other non-usb3 devices) but won't be a good replace ent for the built in one..... if it came to that.

jcmoore1977 commented 5 years ago

Yes, I do have another NAS by a completely different company. I use it to hold all my stuff on. This is just something to mess around with in my spare time.

But that theory did not work. I tried to use the system with built in nic. But all lights go off all I can to is use the usb one.

jcmoore1977 commented 5 years ago

this is something that sticks out in system log..

Mar 23 11:39:03 debian systemd-udevd[195]: Could not generate persistent MAC address for eth0: No such file or directory.

This is for the built in nic card

1000001101000 commented 5 years ago

I see that in my logs too, I think it's just because it's not getting the mac address from the chip or the boot loader.

jcmoore1977 commented 5 years ago

Okay, the built in network comes on when system is booting up kernel, but upon finishing boot it turns off.

1000001101000 commented 5 years ago

What does your interfaces file look like?

jcmoore1977 commented 5 years ago

networkctl WARNING: systemd-networkd is not running, output will be incomplete.

IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback n/a unmanaged 2 eth0 ether n/a unmanaged 3 enx00e04c534458 ether n/a unmanaged

and from dmesg...

[ 2.564836] mvneta d0074000.ethernet eth0: Using random mac address 92:71:46:7b:b8:44 [ 76.236581] usb 2-1: Product: USB 2.0 10/100M Ethernet Adaptor [ 76.397538] dm9601 2-1:1.0 eth1: register 'dm9601' at usb-d0050000.usb-1, Davicom DM96xx USB 10/100 Ethernet, 00:e0:4c:53:44:58

I am wondering if that random mac address it is creating is make the connection to be lost.. as you can see when I plug in the usb one it does not set random address.

1000001101000 commented 5 years ago

No, that’s “normal” for these devices.

The device gets a random mac early in the boot process because it can’t get it from the adapter and hasn’t loaded /etc/network/interfaces yet.

Your usb adapter has the mac address stored in the ethernet chip which the system can look up and use right away.

Let me zip up some logs from a device and post them. I’m thinking anything suspicious you find in your logs wouldn’t be in mine which should help

jcmoore1977 commented 5 years ago

Okay trying something else. I have the led for network staying on. I am adjusting the clock settings in the dts file to see if these will help.

jcmoore1977 commented 5 years ago

okay that did not work 2.573125] OF: /soc/internal-regs/ethernet@74000: could not find phandle [ 2.601463] mvneta: probe of d0074000.ethernet failed with error -2

but the light is on..

1000001101000 commented 5 years ago

What exactly did you change?

jcmoore1977 commented 5 years ago

ethernet@70000 { reg = <0x70000 0x4000>; interrupts = <0x8>; clocks = <0x7 0x4>; status = "disabled"; compatible = "marvell,armada-370-neta"; };

        mdio {
            #address-cells = <0x1>;
            #size-cells = <0x0>;
            compatible = "marvell,orion-mdio";
            reg = <0x72004 0x4>;
            clocks = <0x7 0x4>;

            ethernet-phy@0 {
                reg = <0x0>;
                marvell,reg-init = <0x3 0x10 0xf000 0x881>;
                linux,phandle = <0x8>;
                phandle = <0x8>;
            };
        };

        ethernet@74000 {
            reg = <0x74000 0x4000>;
            interrupts = <0xa>;
            clocks = <0x7 0x3>;
            status = "okay";
            compatible = "marvell,armada-370-neta";
            phy = <0x8>;
            phy-mode = "rgmii-id";

to this

ethernet@70000 { reg = <0x70000 0x4000>; interrupts = <0x8>; clocks = <&gateclk 4>; status = "disabled"; compatible = "marvell,armada-370-neta"; };

        mdio {
            #address-cells = <0x1>;
            #size-cells = <0x0>;
            compatible = "marvell,orion-mdio";
            reg = <0x72004 0x4>;
            clocks = <&gateclk 4>;

            ethernet-phy@0 {
                reg = <0x0>;
                marvell,reg-init = <0x3 0x10 0xf000 0x881>;
                linux,phandle = <0x8>;
                phandle = <0x8>;
            };
        };

        ethernet@74000 {
            reg = <0x74000 0x4000>;
            interrupts = <0xa>;
            clocks = <&gateclk 3>;
            status = "okay";
            compatible = "marvell,armada-370-neta";
            phy = <0x8>;
            phy-mode = "rgmii-id";
        };

Just look at the clocks info. basically put <&gateclk 4>; or <&gateclk 3>;

1000001101000 commented 5 years ago

On the bright side your dtb worked

jcmoore1977 commented 5 years ago

Yeah, I changed it to 4 on the ethernet@74000, now it’s in a boot loop. I need to plug hard drive up to my Linux box and put the backup of uImage, and initrd so I can boot up and go back to drawing board. So it has something to do with the line that starts with ethernet@74000.

I may go back into your archives and get the old dts to compared

1000001101000 commented 5 years ago

There’s another dtb you can compare to here: https://sites.google.com/site/shihsung/88fxxxx-soc

Plus you can look at any of the other mainline armada370/xp ones

jcmoore1977 commented 5 years ago

Yeah I was on that site last night doing some research. I am going to try some things after work today.

jcmoore1977 commented 5 years ago

Okay I found the buffalo kernel here.

http://opensource.buffalo.jp/ls400-110.html

Now to look and see how they bind some of their devices.

1000001101000 commented 5 years ago

I went ahead and wrote the script I was talking about to dump the control register values for the ethernet chip. I ran it against my ls421de and uploaded the output. you can find those under: Tools/physcan.sh Tools/ls421de_stretch_phy.txt

You can see what the values mean from the datasheet found here: https://www.marvell.com/documents/eoxwrbluvwybgxvagkkf/

I'm thinking we should try running the script on the stock firmware (which might require tweaking) and compare the values. If we find a relevant difference we can test ii by changing the corresponding value using phytool and if we find something that solves your problem we could change it permanently in the device tree.

That would only work if the difference in behavior between the stock firmware and Debian is related to how the chip is initialized/configured rather than some kind of logic within the driver/kernel.

jcmoore1977 commented 5 years ago

I ran you above script. I get this error

error: phy_write (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524) error: phy_read (-524)

and when running lshw I get this error

-network:0 DISABLED description: Ethernet interface physical id: 1 logical name: eth0 serial: fe:ac:35:ac:36:db capabilities: ethernet physical configuration: broadcast=yes driver=mvneta driverversion=1.0 link=no multicast=yes -network:1 description: Ethernet interface physical id: 2 logical name: enx00e04c534458 serial: 00:e0:4c:53:44:58 size: 100Mbit/s capacity: 100Mbit/s capabilities: ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd autonegotiation configuration: autonegotiation=on broadcast=yes driver=dm9601 driverversion=22-Aug-2005 duplex=full firmware=Davicom DM96xx USB 10/100 Ether ip=192.168.0.38 link=yes multicast=yes port=MII speed=100Mbit/s

of course ethernet interface 1 is my usb nic.