oofnikj / docker-openwrt

OpenWrt running in Docker
GNU General Public License v2.0
299 stars 63 forks source link

PI4 Permenantly inaccessible #12

Closed hllhll closed 3 years ago

hllhll commented 4 years ago

Hi,

As I understand, older version used macvlan by default? I See some changes regarding that put in in e49358c0cfd47fbeb56b126a76437fb129113a47 The default configuration is now bridge mode; When I was on an older version everything seemed fine, but on newer versions when running make run causes the pi to be permanently inaccessible (even after power cycle). I don't have a proper pi backup, nor console access, so I need to start with blank sd each experiment :(

Can the change from macvlan to bridge cause such issues? Should I switch to macvlan?

oofnikj commented 4 years ago

I switched back to bridge mode because of some problems described in #11, but it's a configurable flag now, so switching back to macvlan requires just setting LAN_DRIVER=macvlan. Be warned though, I switched back to bridge because of unresolved issues. If you had a previously working configuration with macvlan this may be all you need to do.

If you ran make install and you're running docker-openwrt as a systemd service, it's possible to mount your RPi SD card in a Linux environment and disable the systemd service from outside the running OS, see for example https://gist.github.com/htruong/7df502fb60268eeee5bca21ef3e436eb

And then running systemctl disable openwrt.service to disable the service from starting on boot.

But for the device to be inaccessible without make install even after reboot, that's a bit strange, since nothing is modified in the boot configuration when just running make run. I have not experienced this.

How are you connecting to the Pi, over Wifi or Ethernet?

hllhll commented 4 years ago

I Actually have yet to verify #11 on my setup so I'm not sure regarding that...

You'r saying that now by default make install installs a the script as a service that would run on boot? I Wasn't aware of that. Isn't this what might cause the issue?

Thanks for the mount tip, I Hope I won't need it (trying again now) :P

oofnikj commented 4 years ago

This was always the case with make install. For testing, it's not recommend to run make install, only make run.

If the README isn't clear about this please let me know and I will try to explain it better.

hllhll commented 4 years ago

I might have typed install instead of build/run; Sorry for that.

oofnikj commented 4 years ago

If you've managed to solve this issue @hllhll, please mark it as closed. Thanks.

hllhll commented 3 years ago

No, I Wasn't running make install only make run. I'v worked for a while with macvlan and when I eventually encountered the problem that LAN wasn't accessible from Wifi I've switched to bridge. This time openwrt configuration was ok so it was still accessible, however, when ssh session timed out, both died permanently ... Any idea on what change might cause persistency?

Possible hint: rpi stops responding for pings, ssh etc, however, it still respond to ARP request with the statically-allocated IP address.

hllhll commented 3 years ago

I'm zooming in on the problem, In my setup I have my router serving DHCP, while the pi is a member of the network with statically-allocated ip address on eth0 via dhcpcd.conf. A Few things I did't know:

  1. docker network create --driver bridge creates an actual bridge iface on the host system.
  2. This bridge is Not removed on reboot
  3. Since my configuration as above I set LAN_SUBNET to be the same as the statically configured network eth0 Which I think caused the bridge interface and eth0 to be in conflict.
hllhll commented 3 years ago

Changing LAN_SUBNET and LAN_ADDRESS so that the preconfigured-rpi-static ip address would different from it solves the permanent issue, but still, after adding the eth0 to the bridge, everything becomes blocked again. sudo brctl addif xxxxx eth0 (where xxxx is the preexisting-docker created LAN bridge)

All of the resources online regarding rpi-bridge are for Wifi, can't find any proper instructions to simply-bridge 2 interfaces ad-hoc; Thus I don't know if there any specific rpi guidelines for bridging interfaces. as far as I know just create the bridge, set ip and add the interfaces.

Any ideas would be appreciated :)

BTW, After restart, the both the original static ip, and LAN_ADDRESS responds to pings although the pi is not my GW nor the if is added to the bridge (w00t!?)

oofnikj commented 3 years ago

There isn't anything specific to the RPi re: bridging and networking, but I would recommend switching to iproute2 instead of brctl which is deprecated. The equivalent command using ip to add an interface to a bridge would be

# ip link set eth0 master br0

You're correct re: the docker network create ... command. The bridge persists on reboot until the network is deleted with docker network rm ....

Not sure I'm following your network topology here. What are you trying to achieve? Do you want your RPi to be chained behind your main router (double-NAT)?

If that is the case, you have your router serving DHCP on some subnet (let's say 10.0.0.0/24). You would then configure a non-overlapping subnet for your LAN, for example 10.0.1.0/24. When coming online, the virtual WAN interface (most likely eth1) should then receive an IP on 10.0.0.0/24 from your router. Then any devices connected either over Wi-Fi or attached to the virtual LAN bridge should receive an IP from OpenWrt in the 10.0.1.0/24 subnet.

hllhll commented 3 years ago

I Think I've got this, for whomever is intrested:

My setup is that the Pi, and the openwrt router should be inside the lan (whereas there's another edge router). I.E. I want all the fun extra features of open-wrt on a good hardware with the consistency wifi coverage speed etc, of my hardware-router for generic home devices. I've disable the bridging of the wifi and eth0 in order to configure firewall between them. Anyway my goal is to have the Pi as member of the nerwork + also have openwrt router as member of the network. and have networks behind it (Sort of a DMZ-router-ish)

One issue was fairly simple:

2nd issue is due to not carefully reading EVERY word in the config file

# Set LAN_ADDR to something other than the first available address
# in the subnet - Docker will claim this address for the host.

For macvlan it wasn't meaningful to know that (other then, don't use .1); But for bridged interfaces one should know that if docker defines the bride as .1 and the script adds eth0 to that bridge, then rpi would ONLY be accessible on .1.

The 3rd issue I had while debugging non-docker (just a plain) linux bridges is VERY peculiar: