search

PiSCSI / piscsi

PiSCSI allows a Raspberry Pi to function as emulated SCSI devices (hard disk, CD-ROM, and others) for vintage SCSI-based computers and devices. This is a fork of the RaSCSI project by GIMONS.
https://piscsi.org
BSD 3-Clause "New" or "Revised" License
537 stars 82 forks source link

Feature Request: Wifi bridge without NAT using parprouted #1387

Open fdanapfel opened 11 months ago

fdanapfel commented 11 months ago

Info

Describe the issue

The current setup described at https://github.com/PiSCSI/piscsi/wiki/Dayna-Port-SCSI-Link for the Wireless Raspberry Pi setup using NAT does not allow to to use DHCP on a system to which the DaynaPort adapter is attached, and it also makes it difficult to connect to the system from a computer other than the Pi on which piscsi is running.

Using the setup described in the section "Option 1 - Same Subnet" at https://www.willhaley.com/blog/raspberry-pi-wifi-ethernet-bridge/ would allow the computer connected to the PiSCSI to communicate directly with other systems on the WLAN and also enable it to use DHCP to get an IP address.

As far as I can see the setup described here does not have the issues of the solution described in https://github.com/PiSCSI/piscsi/issues/917 since it uses parprouted (https://www.hazard.maks.net/parprouted/).

I've now tested this setup on my PiSCSI and verified that it works. Here are the steps to get it to work manually.

First, install parprouted and dhcp-helper: sudo apt install parprouted dhcp-helper

Attach the DaynaPort either via the web-interface or by running

scsictl -i 6 -c attach -t scdp -f wlan0

Then "down" the piscsi0 interface and remove the bridge that is automatically created by piscsi when the DaynaPort was attached:

sudo /sbin/ip link set dev piscsi0 down
sudo /sbin/ip link delete piscsi_bridge

On bullseye the piscsi_bridge can also be removed with the following commands:

sudo /sbin/ip link set dev piscsi_bridge down
sudo brctl delbr piscsi_bridge

Make sure that IP forwarding is enabled: sudo sysctl -w net.ipv4.ip_forward=1 To make this change permanent: sudo sed -i'' s/#net.ipv4.ip_forward=1/net.ipv4.ip_forward=1/ /etc/sysctl.conf To verify that IPv4 forwarding is enabled you can run the following commands:

sysctl net.ipv4.ip_forward

This should return net.ipv4.ip_forward = 1 showing that IPv4 forwarding is enabled in the Linux kernel

On bullseye update /etc/dhcpcd.conf to disable dhcpcd controlling the piscsi0 interface:

sudo grep '^denyinterfaces eth0$' /etc/dhcpcd.conf || printf "denyinterfaces piscsi0\n" >> /etc/dhcpcd.conf

Since bookworm uses NetworkManger instead of dhcpcd for managing the network interfaces, to prevent NetworkManager from trying to manage the piscsi0 interface it is necessary to create an additional configuration file:

$ cat /etc/NetworkManager/conf.d/99-unmanaged-devices.conf 
[keyfile]
unmanaged-devices=interface-name:piscsi0
$ sudo systemctl reload NetworkManager

(see https://access.redhat.com/documentation/de-de/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/configuring-networkmanager-to-ignore-certain-devices_configuring-and-managing-networking for more information)

Configure dhcp-helper to use the right network interface (only necessary if using WLAN instead of the wired connection):

sudo systemctl stop dhcp-helper
sudo cat > /etc/default/dhcp-helper <<EOF
DHCPHELPER_OPTS="-b wlan0"
EOF

Start dhcp-helper and make sure it is enabled on boot:

sudo systemctl start dhcp-helper
sudo systemctl enable dhcp-helper

Verify that dhcp-helper is running with systemctl status dhcp-helper

Enable avahi reflector if it's not already enabled:

sudo sed -i'' 's/#enable-reflector=no/enable-reflector=yes/' /etc/avahi/avahi-daemon.conf
sudo systemctl restart avahi-daemon

Put wlan0 in promiscuos mode: sudo /sbin/ip link set wlan0 promisc on

Clone the dhcp-allocated IP from wlan0 to piscsi0 so dhcp-helper will relay for the correct subnet:

sudo /bin/bash -c '/sbin/ip addr add $(/sbin/ip -4 -br addr show wlan0 | /bin/grep -Po "\\d+\\.\\d+\\.\\d+\\.\\d+")/32 dev piscsi0'
sudo /sbin/ip link set dev piscsi0 up

If you now run ip a it should show output similar to the following:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: wlan0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether e4:5f:01:78:10:a2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.178.23/24 brd 192.168.178.255 scope global dynamic noprefixroute wlan0
       valid_lft 844129sec preferred_lft 844129sec
    inet6 2003:c9:d701:1200:470a:d09a:e629:147e/64 scope global dynamic noprefixroute 
       valid_lft 6903sec preferred_lft 3303sec
    inet6 fe80::8d9b:2326:42e9:e441/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
5: piscsi0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    link/ether 0e:f4:a6:9c:e6:d8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.178.23/32 scope global piscsi0
       valid_lft forever preferred_lft forever
    inet6 fe80::cf4:a6ff:fe9c:e6d8/64 scope link 
       valid_lft forever preferred_lft forever

The important things here are that both the piscsi0 and the wlan0 interface show the UP flag, that the PROMISC flag is also set on wlan0 and that both interfaces have the same IP address.

If that is the case you can start parprouted:

sudo /usr/sbin/parprouted piscsi0 wlan0

Running ps -ef| grep parprouted should show that parprouted is running and its connecting the piscsi0 interface to the interface used for forwarding the IP traffic:

root        3043       1  0 13:43 ?        00:00:01 /usr/sbin/parprouted piscsi0 wlan0

To find out if there is any issue with parprouted you can check its logs using journalctl|grep parprouted

If I now boot the Atari TT with EasyMiNT with the SCSILink driver enabled it wil automatically get an IP address via DHCP and I'm also able to ping other computers on the network directly.

fdanapfel commented 10 months ago

Did some further research over the weekend, and it looks like it should also be possible to use this setup for the wired Ethernet configuration. All that is needed is to replace wlan0 with eth0 in the instructions I've mentioned above.

Maybe somebody with a PiSCSI attached to a Raspberry Pi with Ethernet port can verify this.

If it also works for the wired Ethernet setup then it might be a good idea to alsp replace the Ethernet bridge setup with this and have a unified setup for both Ethernet and WiFi for the DaynaPort adapter.

rdmark commented 10 months ago

@fdanapfel I agree that if your solution is stable and portable with no hidden drawbacks, we should apply it to wired as well as wireless setups.

If I can find my RPi3B+ (has an Ethernet port) I will test this out during the coming weekend.

One remark: dhcpcd isn’t installed on Bookworm by default. Also, network interfaces are managed by nm by default in Bookwork. Do we have to change the nm configuration as well?

fdanapfel commented 10 months ago

@rdmark Sorry, I haven't had a chance to take a look at Bookworm so far, therefore I can't say what changes will be required to make this setup work on Bookworm.

The only changes in this setup regarding dhcpcd is to make sure the "ip-forwarding" option is set on DHCP requests and to disable mangement of the piscsi0 interface by dhcpcd. So it would be necessary to find out how this is done with the DHCP client that replaced dhcpcd in bookworm, but I'm not 100% sure if these two changes are actually really needed.

If the only way to manage network interfaces in Bookworm is by using nm, then yes the nm configuration would most likely have to be changed as well. But if it is still possible to use the "ip" command then I would try to use the commands I mentioned above first to see if they still work on Bookworm.

rdmark commented 10 months ago

Makes sense, thanks. Getting the bridge working (at all) on Bookworm is priority no.1 for me right now, so I'll try on the weekend to see if there are further complications.

Btw, I think it's fair to say that we should remove or rehash the piscsi code that creates piscsi_bridge dynamically, if we were to make this approach the default. It seems to only obstruct your procedure.

fdanapfel commented 10 months ago

Btw, I think it's fair to say that we should remove or rehash the piscsi code that creates piscsi_bridge dynamically, if we were to make this approach the default. It seems to only obstruct your procedure.

Yes, if the setup I describe in this issue is adopted as the default for giving network access to devices using the emulated DaynaPort device then the piscsi_pridge is no longer needed and the piscsi code to create it dynamically should be removed.

rdmark commented 9 months ago

@fdanapfel I finally got around to playing around with your steps. Sorry for the hold-up!

The good news is that I seem to be able to get an IP address from my DHCP router. The bad news is that no TCP/IP traffic is coming through. Saw once instance of this error in the syslogs:

Jan 15 23:46:06 rasp32 dhcpcd[669]: wlan0: authentication failed from fe80::7277:81ff:fe9c:d862: No such file or directory

This is on a Performa 6116CD running 8.1. I'll try other Macs and OS variants and see if it makes a difference!

fdanapfel commented 9 months ago

@rdmark Thanks for the feedback and giving this a try.

If the Mac is able to get an IP address via DHCP then the network connection should be working. Not sure why you are not getting any other TCP/IP traffic coming trough.

The error message seems to be related to authentication for the WiFi connection of the Rasberry Pi, but since DHCP seems to be working it should not be related to the issue you are seeing.

Which version of Raspberry Pi OS are you using on the PiSCSI, bullseye or bookworm?

If I have time until the weekend I'll try to get my RaSCSI hooked up to one of my old Macs to see if the network connection works with my existing PiSCSI setup on bullseye. If that works I'll can try to figure out what needs to be changed to make it work on bookworm as well.

fdanapfel commented 9 months ago

@rdmark I was now able to test my proposed setup with the RaSCSI (running bullseye) connected to a Macintosh SE 1/40 running System 7.5 with MacTCP and had no issues with TCP/IP communication. Ping, DNS lookup, etc all work (tested with MacTCP Ping and MacTCP Watcher).

fdanapfel commented 9 months ago

@rdmark I was now also able to test this setup on bookworm (using the scsi2pi binaries provided by @uweseimet at https://github.com/uweseimet/scsi2pi) and can confirm that it works there as well.

While testing the setup I noticed one important piece of information missing from my original instructions: to enable IPv4 forwarding on the Raspberry Pi immediately without rebooting the following command must be run before setting up the piscs0 interface and starting parprouted: $ sudo sysctl -w net.ipv4.ip_forward=1

Also I found out that it is not necessary to use brctl to remove the piscsi_bridge, this can actually be done directly with ip (which should work on both bullseye and bookworm): $ sudo /sbin/ip link delete piscsi_bridge (it is not necessary to "down" the piscsi_bridge first)

Since bookworm uses Network Manger instead of dhcpcd for managing the network interfaces, to prevent Network Manager from trying to manage the piscsi0 interface it is necessary to create an additional configuration file:

$ cat /etc/NetworkManager/conf.d/99-unmanaged-devices.conf 
[keyfile]
unmanaged-devices=interface-name:piscsi0
$ sudo systemctl reload NetworkManager

(see https://access.redhat.com/documentation/de-de/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/configuring-networkmanager-to-ignore-certain-devices_configuring-and-managing-networking for more information)

rdmark commented 9 months ago

Thanks for all the detailed instructions! I didn't get around to looking at this today, but I think I can take some time tomorrow.

rdmark commented 9 months ago

@fdanapfel May I ask you to update the instructions in the original description, to make it easier to follow along your steps and avoid making mistakes?

I tried again this evening to get my Power Mac working with this setup but still no luck.

Do you have other network interfaces in your setup? I have a wired interface as well as the virtual docker interface (that I took down) that might be interfering perhaps?

My next step is to try to get back to a working "traditional" piscsi bridge to make sure that nothing else is malfunctioning...

FWIW I'm testing on Bullseye right now since that's what my main piscsi setup is using. The Bookworm instructions will come in handy later on for sure, though.

fdanapfel commented 9 months ago

@rdmark OK, I already though about updating the original instructions myself. I'll probably also add some of the information I mention below on how to check if everything is set up correctly.

Does the Power Mac you are using for testing this setup have a built-in network adapter? The reason I ask is that I remember reading somewhere that the Daynaport driver for the Mac does not work correctly if another network adapter is installed as well. If you have another Mac without a built-in network adapter it might be worth trying to use it for testing this setup instead of the Power Mac.

Since my RaSCSI board is connected to a Raspberry Pi Zero 2 W I only have the WLAN interface in my setup. But it should not matter if there is also wired Ethernet interface, as long as all the other network interfaces are not configured for the same IP network as the one that is used by the virtual daynaport adapter. To make sure no other network is interferring you could try to disable them, so that the only network interfaces is the one that is used for the daynaport adapter.

To verify that your setup is correct you can run the following commands: sysctl net.ipv4.ip_forward This should return net.ipv4.ip_forward = 1 showing that IPv4 forwarding is enabled in the Linux kernel

ip a This should give an output similar to the following, showing that the piscsi0 interface is up and has the same IPv4 address as the one that is used for forwarding the traffic (192.168.178.23 in my example)

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: wlan0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether e4:5f:01:78:10:a2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.178.23/24 brd 192.168.178.255 scope global dynamic noprefixroute wlan0
       valid_lft 844129sec preferred_lft 844129sec
    inet6 2003:c9:d701:1200:470a:d09a:e629:147e/64 scope global dynamic noprefixroute 
       valid_lft 6903sec preferred_lft 3303sec
    inet6 fe80::8d9b:2326:42e9:e441/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
5: piscsi0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    link/ether 0e:f4:a6:9c:e6:d8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.178.23/32 scope global piscsi0
       valid_lft forever preferred_lft forever
    inet6 fe80::cf4:a6ff:fe9c:e6d8/64 scope link 
       valid_lft forever preferred_lft forever

It is also important to make sure that the "real" network interface is in "promiscous" mode (the "PROMISC" flag must show up in the output for the inteface).

systemctl status dhcp-helper.service should show that the dhcp-helper process is running and bound to the network interface used for forwarding the IP traffic from the daynaport adapter (in my case -b wlan0)

ps -ef| grep parprouted should show that parprouted is running and its connecting the piscsi0 interface to the interface used for forwarding the IP traffic: root 3043 1 0 13:43 ? 00:00:01 /usr/sbin/parprouted piscsi0 wlan0

To find out if there is any issue with parprouted you can check its logs using journalctl|grep parprouted

Hope this helps.

fdanapfel commented 9 months ago

@rdmark I've now updated the instructions in the initial description with some of my findings and also added the information on how to check if everything is set up correctly. Hope this helps.

rdmark commented 9 months ago

That’s perfect, thanks! This week I’m working offsite, again, without access to my vintage Mac testbed (obviously) so I’ll try again this coming weekend.

And yes, this Power Mac has a built-in Ethernet interface. However, this hasn’t been a problem for me in the past with piscsi DaynaPort. “Built in” and “alternate” Ethernet were coexisting harmoniously and both could be used. That was with a different Power Mac though.

rdmark commented 8 months ago

Progress!

Got my SE/30 set up, running System 7.1.1, OT1.3. After some tinkering I got the network interface to initialize and get assigned correct IP / gateway by the DHCP server. I can ping the interface from another computer. However, pinging another IP from the Mac doesn't work. 100% packet loss.

But it's very close to working now!

rdmark commented 8 months ago

@fdanapfel I started from scratch with a fresh release image on a RPi Zero W. This time the end result was a success on the same SE/30! It got assigned the same IP as before, but this time routing worked. Not entirely sure why it worked on the Zero W, but not the RPi 3B+.

Next up, to try the 3B+ again but with a fresh release image... I want to make sure the parprouted solution works equally well on wired network, on a device with more than one interface... Ideally we should have one common solution for all setups.

rdmark commented 8 months ago

Bridging eth0 on the RPi3+ while wlan0 is enabled worked fine too, on the RPi3B+ and a fresh OS. There must have been some residual crap going on before.

So the next step would be to figure out how to make this configuration persistent and automatic (between reboots & reattaching of the dp device). I think updating the C++ code to remove the bridge creation, and instead do this every time the dp device is attached should do the trick: 1) promisc on, 2) copy the IP, 3) start parprouted

Any volunteers to attempt to hack the C++ code? :)

fdanapfel commented 8 months ago

@rdmark Woohoo! Great to hear that you managed to get the setup working now. And even better that you could verify that it also works for both wlan0 and eth0.

As far as I can see there are two parts needed to make this configuration persistent and automatic: one is the "static" part that needs to be carried out during the initial configuration, which would involve installing dhcp-helper and parprouted, enabling net.ipv4.ip_forward, configuring dhcp-helper to use the correct interface, enabling the avahi reflector (which might not really be neded, since avahi seems to be for Bonjour/Zeroconf/mDNS) and setting up the config file for NetWorkManager to not manage piscsi0. And then there would be the "dynamic" part that needs to be done every time the daynaport is attached consisting of the 3 steps you mentioned. And of course when the daynaport is detached those 3 changes should be reverted in the opposite order (stop parprouted, then remove piscs0 interface, then disable promiscuous mode)

Regarding the required changes in the C++ code @uweseimet might be able to help. As you can see at https://github.com/uweseimet/scsi2pi/issues/45#issuecomment-1912214768 he mentioned he might look into changing the code in scsi2pi for manging the bridge based on the outcome of this issue.

rdmark commented 8 months ago

@fdanapfel WIP branch here: https://github.com/PiSCSI/piscsi/pull/1432

So far, I've only gotten as far as disabling the creation of piscsi_bridge. Next up is to copy the IP address of the "chosen" interface to piscsi0. It should be done somewhere in CTapDriver::Init I think.

MarcTremblay1981 commented 7 months ago

@fdanapfel WOW... This is incredible! I'm now trying this on a RaSCSI Reloaded attached to a Pi Zero W v1.1 running PiSCSI v22.12.1 and it works! My old Mac has its own IP and traffic is routed to and from it!

fdanapfel commented 7 months ago

@MarcTremblay1981 Cool! Glad to hear that my solution works for others as well. Hopefully this will help to get it officially included in future versions of PiSCSI.

marciot commented 7 months ago

@fdanapfel: Thank you for this thread! I am trying to join GlobalTalk using PiSCSI and I feel like this will get me closer to that goal, as it gets rid of the NATing that was causing me problems.

I do have a couple comments:

1) The directions use a /32 address. In my case, I had a /24 so I changed:

sudo /bin/bash -c '/sbin/ip addr add $(/sbin/ip -4 -br addr show wlan0 | /bin/grep -Po "\\d+\\.\\d+\\.\\d+\\.\\d+")/32 dev piscsi0

To:

sudo /bin/bash -c '/sbin/ip addr add $(/sbin/ip -4 -br addr show wlan0 | /bin/grep -Po "\\d+\\.\\d+\\.\\d+\\.\\d+")/24 dev piscsi0

Is this necessary? I don't know, but it was one of the things I did at first when trying to get it to work. It may not be necessary.

2) On classic Macintosh computers, MacTCP does not appear to work with modern DHCP servers, so those computers still need a static configuration when using this setup. This may trip new users up and break disk images that are preconfigured with 10.10.20.2, so it may not be the best default config for PiSCSI, but perhaps it could be an additional transparent bridge option during setup.

That said, if you do set a static address, it seems to work fine for both outgoing and incoming connections, which is something the previous setup did not allow, so thank you for that!

fdanapfel commented 7 months ago

@fdanapfel: Thank you for this thread! I am trying to join GlobalTalk using PiSCSI and I feel like this will get me closer to that goal, as it gets rid of the NATing that was causing me problems.

I do have a couple comments:

  1. The directions use a /32 address. In my case, I had a /24 so I changed:

sudo /bin/bash -c '/sbin/ip addr add $(/sbin/ip -4 -br addr show wlan0 | /bin/grep -Po "\\d+\\.\\d+\\.\\d+\\.\\d+")/32 dev piscsi0

To:

sudo /bin/bash -c '/sbin/ip addr add $(/sbin/ip -4 -br addr show wlan0 | /bin/grep -Po "\\d+\\.\\d+\\.\\d+\\.\\d+")/24 dev piscsi0

Is this necessary? I don't know, but it was one of the things I did at first when trying to get it to work. It may not be necessary.

According to the parprouted documentation (https://www.hazard.maks.net/parprouted/) it is mecessary to have an IP address assigned on all interfaces that wil be used by parproute, but it doesn't have to be the same IP: You should have an IP address assigned on all of your interfaces that you do bridging on. There is no need for this address to be from the same networks as the bridged networks. Any dummy address will do. I just suggested to do it this way since I saw it done like that in other documentation (I guess the reason others did it like that is to make it easier to spot that the two interfaces are connected together via parprouted).

That said, if you do set a static address, it seems to work fine for both outgoing and incoming connections, which is something the previous setup did not allow, so thank you for that!

Great to hear that it is working for you.