radio24 / TorBox

TorBox is an easy to use, anonymizing router based on Raspberry Pi, which also runs on Debian and Ubuntu based systems.
https://www.torbox.ch
GNU Affero General Public License v3.0
257 stars 27 forks source link

Pi Zero 2 W takes 30-40 minutes to establish connection and SSH (Torbox_054v) #293

Open PapiJalopy opened 3 months ago

PapiJalopy commented 3 months ago

I installed branch "torbox_054v" via script install on a Pi Zero 2 W. SSH access is very difficult to maintain and constantly disconnects and even connecting takes about 20 minutes after the device has booted. About 5 minutes after im able to connect the connection drops and im no longer able to connect anymore.

I would like to add that internet access does not work either. I can login with hdmi as a display and Tor connection can be established but internet connection is not being passed to the interface and ssh is not working. Im thinking this isnt an issue with ssh but something to do with the connection itself.

EDIT: After waiting about 30-40 minutes ssh works and internet connection works. But i feel like thats a very long time to wait and should be investigated. This is an issue for me on all branches and 2 latest releases as well.

zotil commented 2 months ago

TorBox Mini is an alpha version that has not been officially released yet. You can follow the steps published HERE.

I recently got a brand-new Pi Zero 2 W, and following the steps in the link worked well for me without any issues.

Which OS are you using to access TorBox Mini?

PapiJalopy commented 2 months ago

TorBox Mini is an alpha version that has not been officially released yet. You can follow the steps published HERE.

I recently got a brand-new Pi Zero 2 W, and following the steps in the link worked well for me without any issues.

Which OS are you using to access TorBox Mini?

@zotil Torbox mini works fine. I'm referring to normal torbox install.

Im used a waveshare ethernet "hat" and wifi to provide the network and it takes a long time to provide a stable ssh and network connection.

Just to clarify i installed using the normal install script not the mini script. Previous releases have the same issue so its not specific to just v054

zotil commented 2 months ago

Thanks for the clarification.

TorBox is officially recommended for Raspberry Pi models 3 and 4, and we're testing it on model 5. The Raspberry Pi Zero 2 W isn't fully supported in the main branch, which might be causing the issues. For this model, TorBox Mini is the better option as it's specifically designed for the Zero 2 W.

I'm getting an Ethernet cap soon, so I'll test it on my end and see if I encounter the same issue as you.

PapiJalopy commented 2 months ago

@zotil You don't really need an ethernet hat to test now if you wanted to. You can just flash any of the releases or build from script and connect to the Torbox Wi-Fi network and try to ssh in.

zotil commented 2 months ago

@PapiJalopy I actually tested it with both the main branch and torbox_v054, and I didn't experience the same issues. However, the problem might be related to the Ethernet cap, so I'll test it again once mine arrives to see if I can replicate the issue.

PapiJalopy commented 2 months ago

That's very interesting. Let me know how it goes. I admit I haven't tried with the hat removed so Ill test that myself and let you know if I still experience issues.

PapiJalopy commented 2 months ago

@zotil I quickly just flashed the latest release build without any hat attached to the Pi Zero 2 W and I'm still experiencing the same issue. I'm using Arch Linux but I've tried to ssh on a windows machine just to make sure it wasn't an OS issue and the problem persists.

PapiJalopy commented 2 months ago

Not sure exactly what you did differently but its doesn't seem to be Ethernet hat related.

zotil commented 2 months ago

Not sure exactly what you did differently but its doesn't seem to be Ethernet hat related.

I ran the run_install.sh script on a freshly flashed SD card with Pi OS Lite (it did take a while).

PapiJalopy commented 2 months ago

I would like to note, I was having the same issue with the v054 branch. It was installed using the same script as well. Connection still takes about 20 minutes. On a minor occasion I might be able to SSH in about 5-10 minutes, but only briefly before it drops connection requiring to have to wait another 10 or so minutes before it allows me to SSH again. From there it's a hit or miss if I get a stable connection or if connection is again dropped. Though after about 30 minutes it seems that everything is stable and there is no more issue. The main concern is the long wait period before being able to use. I was imagining a smaller Torbox with Ethernet capability that can be carried around easier rather than an a full sized raspberry pi 5.

On a side note I have been using a Raspberry Pi 5 with master branch for about a year or so now, occasionally updating, and only recently updating to v054 branch, without any issues.

zotil commented 2 months ago

Hello @PapiJalopy,

I just finished testing with an Ethernet/USB expansion board.

After installing TorBox and configuring it to route internet through the Ethernet port, I waited up to 2 hours, but the SSH port (22) never opened. While waiting, I investigated the issue and found that the eth0 input ports are blocked by the firewall. This is likely a security measure, especially in potentially hostile environments.

If your network is secure, you can edit /etc/iptables.ipv4.nat to allow SSH traffic by adding the following rule:

-A INPUT -i eth0 -p tcp --dport 22 -s 192.168.100.100 -j ACCEPT

Replace 192.168.100.100 with your local network IP address.

Alternatively, you could change eth1 to eth0 in line 14 of /etc/iptables.ipv4.nat, but this will expose all ports on eth0 to the network, which isn't recommended unless you're sure it's safe.

It would be interesting to know how your SSH port eventually opened. Did you make any specific configurations?

Thank you for your feedback.

PapiJalopy commented 2 months ago

@zotil

That is quite interesting to hear. No special configurations were made. Installs that we attempted were last 2 releases, master branch, and v054 branch via script install with similar if not same results. SSH does eventually come through so its interesting to hear that it didn't eventually open for you. This has me speculating if the issues we experienced are related or not, taking into account that I was eventually able to SSH in and you didn't have the same "success".

Nevertheless I'm glad to hear of a possible fix/workaround. Having port 22 visible to the network is a bit of a concern, not the end of the world, but if a threat actor has access to SSH that would allow more of an attack area. I'm imagining one of the use cases for this device would be to use it on a insecure network (such as public networks at restaurants and airports) to allow a secure connection to the internet.

Edit: I would also like to add, that in my testing, I did it the other way around. Ethernet was input and WIFI was output. This is possibly a reason why we had different results.

zotil commented 2 months ago

Edit: I would also like to add, that in my testing, I did it the other way around. Ethernet was input and WIFI was output. This is possibly a reason why we had different results.

Thanks for the details! It’s interesting that you were eventually able to SSH in while I wasn’t, which suggests that our issues might be related but not identical.

After considering your setup, I realized that the difference in our testing methods could be key. I tested with Ethernet as the output, whereas you used Ethernet as the input and Wi-Fi as the output. In your case, TorBox might be reconfiguring wlan0 to function as a hotspot, which could cause the Wi-Fi internet connection to drop after a while. Additionally, since TorBox treats the Ethernet port as a potentially hostile input, it’s likely filtering the traffic, which is why the IP from Ethernet is being blocked.

To better understand the situation, could you clarify how you’re connecting the Raspberry Pi to the internet? Are you using the onboard Wi-Fi or Ethernet? Also, how are you connecting to the device as a client—via Wi-Fi or Ethernet?

PapiJalopy commented 2 months ago

In my configuration, Ethernet is being used as the input, and onboard WIFI was being used as the, lets call it a "hotspot". Attempting to SSH was being done through the WIFI "hotspot". Though I had also connected monitor to the onboard HDMI and was able to access menu through there before SSH and network were open and the connection was fine inside the PI itself.

I was able to connect the network to TOR and local network inside the console/terminal was functioning just fine before SSH was finally open and I could connect through the "hotspot".
There was no connection drop out's, the WIFI network stayed active for the duration of the test the only issue was SSH was not stable and would only open about 20 minutes after device was booted.

WIFI network had no connection issues other then the network would only work when SSH was stable.

radio24 commented 1 month ago

If you want to install TorBox v.0.5.4 (Test) on a Pi Zero 2 W, you have to use the script called _run_install_mini.sh_ or use our test image (see here).

As zotil mentioned, Pi Zero 2 W is in a test phase. Using a Pi Zero 2 W in connection with a Waveshare Ethernet "hat" additionally complicates the determination of the problem. As gozillah experienced here, the assignment of an IP address could be a problem in such a kind combination. Important: TorBox is configured as a DHCP client, which means that the router has to give TorBox all necessary network information (see here).

Because you wrote that you had connection problems already using a "normal" Raspberry Pi, let's focus on solving that problem first. If I understand you correctly, your configuration looks like this: Client (with SSH) -> WLAN0 -> ETH0 -> Internet

My questions are the following:

To look further into the issue, we need the following information/files from a TorBox installation with that particular problem:

You can send these information/files via email, but you have clearly to refer to this thread.

PapiJalopy commented 1 month ago

@radio24 Let me clarify once again.

In my testing, I installed "v0.5.3 release image", "master" branch, as well as "torbox_v054" branch via script install on a Pi Zero 2 W, using the the normal install script, not the mini script. The issue I am having has nothing to do with the "Torbox Mini", I am not attempting to install the Torbox mini at all. I have stated that in previous comments..

I was testing the concept of a small form factor torbox using a Pi Zero 2W and a Waveshare Ethernet Hat.

The device will function the same as Pi 3, 4, or 5 but have a smaller foot-print, allowing you to carry it around more comfortably.

The issue that i was experiencing was when booting the device, SSH access does not open until about 20 minutes after the device has booted. After waiting for 20 minutes the device is able to connect to ssh and network connection begins to work.

In my configuration, Ethernet is being used as the network input, (for example, ethernet cable from my router is plugged into the Waveshare Hat) and I am using my computer to connect to the Wifi network that the Raspberry Pi created. .

I had also connected monitor to the onboard HDMI and was able to access menu through there before SSH and network were open and the connection was fine inside the PI itself, allowing me to select the option to connect to tor without issue.

After waiting the stated 20 or so minutes, SSH finally allows connection and at the same time it network connection begins working as well. This is the only issue that I am having, It feels like a DHCP issue taking a long time to assign a IP address but I really don't know, I dont know what else could cause the long wait before stable connection is established.

The WIFI network that is created by the PI Zero 2W does not disappear or disconnect either, it stays active the entire time.

The issue is not present using a Raspberry Pi 4 or 5.

SSH is being run through linux terminial.

PapiJalopy commented 1 month ago

Because you wrote that you had connection problems already using a "normal" Raspberry Pi, let's focus on solving that problem first.

@radio24 Could you show me where I wrote this?

PapiJalopy commented 1 month ago

@radio24 You can quickly test this yourself. Flash your v053 release image to a SD card, boot it with a Pi Zero 2W, connect to the Torbox WIFI network and try to ssh in.

You do not even need the Waveshare hat the issue will still be present with or without the Ethernet hat.

Edit: Reason i say v053 is for convenience, so you dont have to install from script. The issue will be the same on v054 and master

radio24 commented 1 month ago

Because you wrote that you had connection problems already using a "normal" Raspberry Pi, let's focus on solving that problem first.

@radio24 Could you show me where I wrote this?

Ok, then I misunderstood.

radio24 commented 1 month ago

@radio24 You can quickly test this yourself. Flash your v053 release image to a SD card, boot it with a Pi Zero 2W, connect to the Torbox WIFI network and try to ssh in.

You do not even need the Waveshare hat the issue will still be present with or without the Ethernet hat.

Edit: Reason i say v053 is for convenience, so you don't have to install from script. The issue will be the same on v054 and master

Of course, I will find the same or a similar problem. When I started with TorBox mini, I did precisely that - flashing the v053-release on a SDCard and starting it up in a Pi Zero 2 W. And because it didn't work, I changed the configuration in the v054, which is not finished, yet.

Some of the configuration has to be done during the installation. That's why we have a separate installation script. The goal is to integrate these changes into the main installation script, but even then, the option --torbox-mini has to be used (the option is already implemented in here , but we have still to test it).

PapiJalopy commented 1 month ago

Interesting. From my testing v053 release image, master branch main script, and v054 main script, are all functional on a Pi Zero 2W.

The only issue is the long wait time for SSH connection. After waiting 20ish minutes for SSH to connect everything works without any other issues.

I tried to investigate it myself but had no luck.

Maybe we are misunderstanding each other. Because in this scenario I am not trying to run TorBox mini on the Pi Zero 2W, I am running normal TorBox on the Pi Zero 2W, and it works. The problem I am trying to fix is the long SSH issue.


On a side note. I would like to share with you a TorBox mini I made with a screen and functional buttons -

https://github.com/user-attachments/assets/f0733406-2e42-4b09-bee9-45bb70cc0927

radio24 commented 1 month ago

Wow, nice.

Some other questions...

PapiJalopy commented 1 month ago

192.168.42.1

I recall trying at one point, it shared the same issue.

radio24 commented 1 month ago

Ok, that seems to be right...

If only ssh were lagging, it would probably be something with the sshd configuration. A quick Google search brings up solutions like uncommenting UseDNS no and setting IPQoS 0x00 in /etc/ssh/sshd_config and then restarting sshd.

However, if also the connection to webssh is lagging, it may be beneficial after powering up the Pi Zero 2W to check if a program or script is running in the background, producing a heavy load for the Pi Zero 2W and probably with the timeout setting is therefore responsible for the lagging.

PapiJalopy commented 1 month ago

If only ssh were lagging, it would probably be something with the sshd configuration. A quick Google search brings up solutions like uncommenting UseDNS no and setting IPQoS 0x00 in /etc/ssh/sshd_config and then restarting sshd.

I have tried those solutions and it didn't fix the issue. I googled it as well and spent the better part of 3 days trying various fixes I found online with no success.

It could be possible something is causing high resource usage but it wouldn't make sense for it to last over 20 minutes.

If you read back on more of my earlier comments, I stated that after 20 minute wait time, SSH would allow me to connect but roughly 1 or 2 minutes later it would drop the connection again, then another 2-5 minutes later it would restore connection and after that it would be stable from there on out.

Basically it would be an unstable connection for little after allowing to connect. and then it stabilizes completely after that.

It's might be easier to understand the issue if tried to reproduce the issue just so you can have a visual understanding of it.

radio24 commented 1 month ago

It's might be easier to understand the issue if tried to reproduce the issue just so you can have a visual understanding of it.

That's a no-go because I currently don't have the equipment to reproduce the issue. However, I have an idea what the cause of the issue could be by looking into the differences between the normal TorBox configuration and the one for TorBox mini.

Could you please download the following file: https://raw.githubusercontent.com/radio24/TorBox/torbox_v054/etc/rc.local.mini.special-test, rename it to rc.local and replace the one in /etc. Important the file needs the ownership as root:root and has to be executable.

In code on the Raspi:

cd
sudo cp /etc/rc.local /etc/rc.local.bak
wget https://raw.githubusercontent.com/radio24/TorBox/torbox_v054/etc/rc.local.mini.special-test
sudo cp rc.local.mini.special-test /etc/rc.local
sudo chmod a+x /etc/rc.local

Then, please restart your Pi Zero 2W and check if something has changed.

PapiJalopy commented 1 month ago

The modified rc.local caused the WIFI network that the Pi Zero 2W created to break and not accept a connection, prompting me to input a password for the network, though it had not been changed. I logged in via HDMI and attempted to change the password, but that did not work, the network still no longer allowed me to connect, even after trying to change the password.

I do not understand why you are modifying the rc.local tailored for the TorBox mini when I have told you many times that I am not using the mini script or the mini image.

Once again, I am using the main/normal/default TorBox installation/image/script, on the PI Zero 2W.

I am not using the mini version of TorBox.

I'm not sure if we are both on the same page here but it seems that you are treating this issue as being related to the TorBox mini version of project, when it is not.

I wanted to clarify again so we can both be on the same page, because currently it feels like we are not.

radio24 commented 1 month ago

Don't panic.

If you used the command lines above, you can easily revert the changes with the following command: sudo cp /etc/rc.local.bak /etc/rc.local

Even if it was not a success, it may give me clues about where the problem could be. And yes, I'm very well aware that you use the normal TorBox configuration and not TorBox mini. However, you must be mindful that you are not using the normal Raspberry Pi 3B, 4 or 5, but the Pi Zero 2W. So, I aim to find out what in the normal TorBox configuration doesn't ideally work with the PI Zero 2W. Because I'm currently unable to test it on my own, I hope you are willing to do some tests. However, if this is too much to ask or freaks you out, just let me know.

After reverting to your old rc.local, I would like to ask to do an ifconfig and the top 5 processes showing up with top when SSH is still blocked. Please repeat that when SSH is working and post the results.

PapiJalopy commented 1 month ago

I do not care about the installation. It was a fresh install just to test.

PapiJalopy commented 1 month ago

I have no problem testing, I very much would like to get this sorted. I just wanted to make sure we are both on the same page. I seen that you referenced the rc.local file using "mini" and that had me confused so i wanted to clarify.

IMG_20240919_031301_330 IMG_20240919_031246_706

radio24 commented 1 month ago

Ok, that looks promising. The network should work properly, and there seems to be no background process interfering with sshd.

I guess in the standard rc.local for the TorBox on the Raspberry Pi 3B, 4, 5 is something included that doesn't work well with the Pi Zero 2W. So I took the rc.local.mini, which I know works with the Pi Zero 2W and implemented some of the things from the standard rc.local necessary for your configuration. It didn't work out the first time because I missed some of the things related to hostapd.

In the meantime, based on your test, I changed the rc.local.mini.special-test accordingly and would be thankful, if you could repeat the test with this new file. Thanks. You can see the changes also here.

PapiJalopy commented 1 month ago

Ill try it now. here is the top with ssh working

SSH working

IMG_20240919_032112_995

PapiJalopy commented 1 month ago

The WIFI is fine now but the Issue still persists. I would like to mention that the time between SSH working or not is not consistent. Some times it takes up to 30 minutes for SSH to open, some times it could be 10 minutes, but there is always a possibility of it disconnection again even after the connection is established, but after a few more minutes of waiting it will allow connection again.

What i am trying to say is it is inconsistent and i felt that it should be mentioned.

PapiJalopy commented 1 month ago

It is very late/early here in the USA and i need to get to bed. I will be able to continue testing more after i get some rest.

PapiJalopy commented 1 month ago

i tried to restart one more time and connect. if i try to connect as soon as the device has booted it allows me in for about 1 minute and then drops the connection. would like to mention that because that is new,

PapiJalopy commented 1 month ago

The connection time has improved significantly actually. It allowed me in for 1 minute, dropped the connection for 1 minute and now it has allowed me to connect again.

PapiJalopy commented 1 month ago

Seems that the issue is partly resolved. Other than dropping the connection for a brief 1 minute after booting and connecting the issue seems to be fixed.

radio24 commented 1 month ago

You can avoid the 1 minute drop after booting your Pi Zero 2W with the following changes in the rc.local.mini.special-test, which should now be used as /etc/rc.local.

  1. Remove the following line at the beginning of the file: sudo systemctl stop hostapd

  2. Remove the following block:

    # If the configured interface is available, we will start hostapd
    INTERFACE=$(grep "^interface=" /etc/hostapd/hostapd.conf | sed "s/.*=//g")
    if ip link | grep "$INTERFACE" ; then sudo systemctl start hostapd; fi

These two blocks are used because diverse hardware with diverse plugged-in gadgets are used. We have to be sure that hostapd is really loaded if the correct interface is present. With that entry, we could avoid lock-out situations. However, in your setting, it is unlikely that it is needed.

PapiJalopy commented 1 month ago

@radio24 Removing/commenting out "sudo systemctl stop hostapd" as well as the mentioned block causes a failure to connect tothe wireless network created by the Pi, same as the issue previously regarding it notifying me the password is incorrect.

radio24 commented 1 month ago

Ok,that's odd because hostapd should be started by the system before rc.loca'.

I guess, there are two possible solutions to that problem:

  1. Instead of removing sudo systemctl stop hostapd, switch stop with start and remove block (2).
  2. Remove sudo systemctl stop hostapd but keep block (2).

In my opinion, the first solution is the better.

PapiJalopy commented 1 month ago

I tried the second option you recommended already and it had almost the same issue but rather than asking to input password it said cannot connect. i will try the 1 option you recommended

for option 1 do i still keep the block uncommented or do i comment it out?

PapiJalopy commented 1 month ago

I tried both options. For option 1, tried with the block commented and uncommented and still the same issue. Networkerror

I mean honestly waiting a minute isn't that bad. If you would like you continue trying to investigate the issue ill continue to test, but isn't too inconvenient waiting a minute.

PapiJalopy commented 1 month ago

I was playing around in the menu just now and I tried to disable web ssh in the menu and it caused normal ssh to not work anymore. Disabling it seems to have some kind of effect on the normal ssh for some reason.

Thought I would mention that.