search

Ysurac / openmptcprouter

OpenMPTCProuter is an open source solution to aggregate multiple internet connections using Multipath TCP (MPTCP) on OpenWrt
https://www.openmptcprouter.com/
GNU General Public License v3.0
1.92k stars 274 forks source link

OpenMPTCProuter v0.58 Release Candidate 5 #1619

Open Ysurac opened 3 years ago

Ysurac commented 3 years ago

Changes:

Server script:

Previous Beta **V0.58 Beta 1:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58beta1/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` _Know issues:_ * Some translation problem like "MacVLAN" in wizard instead of "VLAN" * Patch for 16 WANs support not added * V2Ray issue **V0.58 Beta 2:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58beta2/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` _Know issues:_ * Wrong display of not good firewall zone in status page when MPTCP over VPN enabled **V0.58 Beta 3:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58beta3/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` _Know issues:_ * Tracker still running for omrvpn interface even when VPN is set as none * OpenVPN TCP issues **V0.58 Beta 4:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58beta4/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` _Know issues:_ * Tracker set interface down if gateway doesn't answer to ping * OMR-ByPass issue with domains **V0.58 Beta 5:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58beta5/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` **V0.58 Beta 6:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58beta6/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` **V0.58 Beta 7:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58beta7/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` **V0.58 RC1:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58rc1/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` _Know issues:_ * Some translation issues * ModemManager protocol not working * Wrong theme after update * MPTCP over VPN issues **V0.58 RC2:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58rc2/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` **V0.58 RC3:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58rc3/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` _Know issues:_ * Error in omr-tracker script on loop detection, RC4 is compiling. **V0.58 RC4:** _Images_: https://download.openmptcprouter.com/refs/tags/v0.58rc4/5.4/ _VPS script_: ```wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh``` _Know issues:_ * Packages list not updated after upgrade (fixed in develop branch)

V0.58 RC5: Images: https://download.openmptcprouter.com/refs/tags/v0.58rc5/5.4/ VPS script: wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh

JeffWatsonGH commented 3 years ago

Is the option "Support 16 WANs on x86_64 instead of 8" already included? With which commit was this added?

Ysurac commented 3 years ago

I forgot to add it. This will be added in next beta.

JeffWatsonGH commented 3 years ago

Alright. Thanks for the quick reply! :)

geeuk commented 3 years ago

Hi, Your Images for RPI4 does not seem to work for some reason? The pi does not boot and loops, I tried both ext4 and squashfs.. I was tempted to try the wireguard method. Thanks

Ysurac commented 3 years ago

I tested on my RPI4 this image: https://download.openmptcprouter.com/refs/tags/v0.58beta2/5.4/rpi4/targets/bcm27xx/bcm2711/openmptcprouter-v0.58beta2-r0%2B16336-b36068d35d-bcm27xx-bcm2711-rpi-4-ext4-factory.img.gz And it's working.

geeuk commented 3 years ago

Hi, Apologize as the led light sequence must of changed as it was flashing differently, I assumed it was not working.

glutfloc commented 3 years ago

Hi Ysurac,

I'm currently tying to install the VPS part from script (wget -O - http://www.openmptcprouter.com/server-test/debian10-x86_64.sh | sh) and it seems that the repository is not good : " Err:8 https://repo.openmptcprouter.com buster/main amd64 Packages File has unexpected size (8500 != 8498). Mirror sync in progress? [IP: 89.44.9.58 443] Hashes of expected file:

Regards,

Yohann

Ysurac commented 3 years ago

It's better now ? I purged CDN cache.

glutfloc commented 3 years ago

Yes ! I was able to configure one of my VPS. Thank you. By the way I noticed that it was also the case for the current stable release of the script (https://www.openmptcprouter.com/server/debian10-x86_64.sh). I haven't checked since this morning but I guess this is also fixed for this one. Thank you. Yohann

Ysurac commented 3 years ago

Yes, it was related to the update of the Debian repo, but I forgot to purge CDN cache so this give some cache inconsistency. It's now fixed for stable and snapshot.

wicadmin commented 3 years ago

Is ModemManager still broken on the RC? What is broken about it?

Kalimeiro commented 3 years ago

Is ModemManager still broken on the RC? What is broken about it?

Wait RC2

Speedy37 commented 3 years ago

Great work! I'm on RC1 (RPi3 B+). So far so good :)

Is it me, or bbr2 congestion algorithm is very good? At least that's the feeling after 2 days. I've 1 slow adsl link (5mbits) and 2 "good" 4G (10-80Mbits) and the latency is much more stable while the bandwidth is higher.

UDP is still not quite there (Glorytun TCP), haven't tried anything else on this release and I disabled HTTP/3 for now due to that. Any advice, thing you want me to try.

Is it normal that I have to restart a TCP connection to get the benefits of adding a 3rd temporary 4G for a temporary boost, I had the feeling that MLTCP allowed for dynamic add/remove of ressources? The TCP connection stays up, but gets no speed boost.

Ysurac commented 3 years ago

Yes BBR2 seems quite good even if still in alpha/ For UDP, you can try v2ray proxy that support TCP and UDP but use more CPU/memory. It's not normal to have to restart a TCP connection, you can try another Multipath TCP scheduler in Network->MPTCP.

Ysurac commented 3 years ago

@Speedy37 I'm not able to reproduce the TCP connection problem at least without using MacVLAN, I will test with MacVLAN used for WANs. How do you make your test ?

Speedy37 commented 3 years ago

Default settings, ipv4 only wan1,wan2,wan3 are MacVLAN on eth0. usb0, Normal (the one I add/remove).

I can see it on the MPTCP fullmesh almost instantly.

Bumpalert commented 3 years ago

Hey @Ysurac .. The beta version is almost perfect. I got only one issue, when I enabled wireguard on both master and usb interface no gateway found, first interface becomes no gateway. As I expect all 4G networks to block MPTCP, can I keep all with wireguard or master need to be left without wireguard for making the connection to the VPS? The speed addition tested successfully now with 2 wired internet gateways without wireguard. One wired and 1 4g network wrapped over wireguard successfully.

image

Bumpalert commented 3 years ago

both some issue with wireguard also on usb2_tether(2nd 4g). but i am still happy that aggregation works without any special settings.

image

techworldthink commented 3 years ago

is v0.58 Release source from master branch?

Ysurac commented 3 years ago

@Bumpalert I try to fix that @techworldthink for Release Candidate yes, it's from master branch. There is a v0.58rc1 tag.

techworldthink commented 3 years ago

I tried to build v0.58 for my rpi3 .but got an error!

make -r world: build failed. Please re-run make with -j1 V=s or V=sc for a higher verbosity level to see what's going on make: *** [/home/ubuntu/openmptcprouter/rpi3/source/include/toplevel.mk:230: world] Error 1

How to solve this ?

Bumpalert commented 3 years ago

I got another look and feel bug. When you have 2 vps and disable the one with IP address and enable one with domain name, the omr-iperf shows that it connected to the disabled IP address one and status shows direct output. But public ip of connected devices is actually the enabled vps which has domain name.

image image

I was planning to add noip sub domain names to the dynamic ip on the vps(with automatic address update from vps). Let me know whether it has any performance concerns.

Ysurac commented 3 years ago

@Bumpalert

Bumpalert commented 3 years ago

factory reset is a problem. It says yes, but settings are not changed on web interface. Is there any thing I missed?

root@OpenMPTCProuter:~# root@OpenMPTCProuter:~# root@OpenMPTCProuter:~# root@OpenMPTCProuter:~# firstboot y This will erase all settings and remove any installed packages. Are you sure? [N/y]

root@OpenMPTCProuter:~# root@OpenMPTCProuter:~# firstboot yes This will erase all settings and remove any installed packages. Are you sure? [N/y]

root@OpenMPTCProuter:~# root@OpenMPTCProuter:~# reboot

Ysurac commented 3 years ago

@Bumpalert this can only work on squash FS. You can also update with same image and not keep settings.

Bumpalert commented 3 years ago

confirmed domain name in the vps server address has some issue. Same connection when ip address used working fine will fail if dynamicdns is used eventhough public ip is resolved.

image image

any working way for using pi 4b wifi as ap? last time I had to reflash only because I tried wifi on pi. https://github.com/Ysurac/openmptcprouter/issues/1232 https://github.com/Ysurac/openmptcprouter/wiki/Configure-Wifi

Speedy37 commented 3 years ago

I'm upgrading right now, but in RC1 I had very high CPU usage on my RPI:

image

image

Ysurac commented 3 years ago

I don't have this issue on 0.58rc1 and it's strange that you have high CPU usage on "uci" commands.

Speedy37 commented 3 years ago

Yeah lots of very short lived uci requests, I think omr-tracker seems to loop like crazy. Could this be caused by a disconnected usb0 interface? Omr-tracker seems to be configure with 0s delay on fail "Wait after a failed test (s)".

The new sysupgrade page currently suggest openmptcprouter-v0.57.3-r0+15225-bfc433efd4-x86-64-generic-ext4-combined.img.gz, which won't work on an RPi3 :).

I was unable to use "Flash new firmware image", the interface just reload and does nothing (same thing happened when I tried to switch theme, I add to use ssh and uci). Probably something messed up with my configuration.

Just did an upgrade to rc2 (reinstall from nothing) to a faster SD card (20MB/s vs 5MB/s). It was running great (0.2 load), until I added usb0 back. omr-tracker just non stop looping (load is at 1.0, uci requests are not visible in htop anymore).

Ysurac commented 3 years ago

Do you have errors in System logs about omr-tracker ?

Speedy37 commented 3 years ago

Nothing special.

dooley1981 commented 3 years ago

Hi just installed the new latest firmware for the first time. I'm getting "Multipath seems to be blocked on the connection". I didn't get this on the none beta firmware. This is a new usb stick running on my laptop 64-bits. 20210510_204524~2

Ysurac commented 3 years ago

@Speedy37 I fixed sysupgrade (it was an issue on my server). For uci cpu load I have no idea, there is a delay between each run of omr-tracker and it's not the problem here as it's really UCI. What is the RPI model ?

@dooley1981 Server part is updated and rebooted ? Since 0.58 beta 7 there is a new way to detect if multipath is enabled or not and this use the server API.

Speedy37 commented 3 years ago

RPI 3B+, 64bits image. The new SD card or reinstall seems to have fixed that uci issue. Still, load is quite high for simple ping/http requests. Would you probably accept a PR with a rewrite of omr-tracker in python to lighten the load?

Speedy37 commented 3 years ago

A quick look at the current code tells me there isn't much to improve. I should probably track the root cause of that load first, it might simply be IO stalls.

Edit: I will create an issue dedicated to this to not spam this thread.

Ysurac commented 3 years ago

@Speedy37 Python script need Python interpreter, it's why there is nothing in python on the router, to keep the image light.

flimo44 commented 3 years ago

Hello @Ysurac , i update RC2 but https://packages.openmptcprouter.com/v0.58rc2/.... not found. Impossible to install packages : http://packages.openmptcprouter.com/v0.58rc2/x86_64/packages/.

Else after each update need to upload my certificat. It would not be possible as for the parameters to restore them automatically after each update and the same for software i need to add ( agent pour VM syno... ) ? Last point, following the change from RC1 to RC2 problem with IPV6...

I use this version : https://download.openmptcprouter.com/develop/5.4/x86_64/targets/x86/64/openmptcprouter-v0.58rc2-r0+16336-b36068d35d-x86-64-generic-ext4-combined.img.gz

Merci.

Ysurac commented 3 years ago

@flimo44 I fixed the install package problem. There is no change about IPv6 between RC1 and RC2 For software I made some changes and it should resintall packages, but I never tested (and while package url was not working it's normal). For certificate, you need to add the file in System->Flash firmware, tab configuration. But default file (/etc/uhttpd.key and .crt) should be saved by default.

flimo44 commented 3 years ago

@Ysurac , Utl acess https it's OK in browser but in openMTCTPRouteur not OK : Downloading http://packages.openmptcprouter.com/v0.58rc2/x86_64/luci/Packages.gz *** Failed to download the package list from http://packages.openmptcprouter.com/v0.58rc2/x86_64/luci/Packages.gz

Problem link htpp and not https ?

For certificate thank you for backup solution.

Bumpalert commented 3 years ago

@Speedy37 I fixed sysupgrade (it was an issue on my server). For uci cpu load I have no idea, there is a delay between each run of omr-tracker and it's not the problem here as it's really UCI. What is the RPI model ?

@dooley1981 Server part is updated and rebooted ? Since 0.58 beta 7 there is a new way to detect if multipath is enabled or not and this use the server API.

@Ysurac How to update server part? Just run vps script again? or vps script path is changed?

Bumpalert commented 3 years ago

@Speedy37 I fixed sysupgrade (it was an issue on my server). For uci cpu load I have no idea, there is a delay between each run of omr-tracker and it's not the problem here as it's really UCI. What is the RPI model ? @dooley1981 Server part is updated and rebooted ? Since 0.58 beta 7 there is a new way to detect if multipath is enabled or not and this use the server API.

@Ysurac How to update server part? Just run vps script again? or vps script path is changed? Got vps update script

Malaga82 commented 3 years ago

Great to see wireguard as MPTCP over VPN, but for me it's not bypassing ISP block at all, only OpenVPN does: am i the only one? Other than that, in wreguard interface i need to manually paste server PK, or the VPN will never comes really up. For OpenVPN, when enabled from Setting Wizard, it must manually started, it stays stopped otherwise. Console of omr is filled with "__mptcp_init4_subsockets: token 0x5e5d82a6 bind() to ip_vpn index 75 failed, error -99" Will see the impact on CPU/Memory. Thanks

flimo44 commented 3 years ago

Hello @Ysurac,

I went back to the beta6 version which is very stable at home (the beta7 also remembered). Since RC1 version too many problems. ( I'm on VM on my Nas Synology for info )

From RC1: -IPV6 no longer works. No ip V6 or when it seems OK, impossible to go out in IPV6. (ping KO)

-This error in continuous system log: Sat May 15 00:05:20 2021 daemon.warn odhcpd [5968]: A default route is present but there is no public prefix on lan thus we don't announce a default route ! Sat May 15 00:05:20 2021 daemon.err odhcpd [5968]: Failed to send to ff02 :: 1% lan @ eth0 (Permission denied)

(Merci et vraiment bravo pour le super travail que tu fais sur OpenMPTCProuter.)

Ysurac commented 3 years ago

@Malaga82 I will check MPTCP over VPN again @flimo44 I don't have any IPv6 problems, I think you have a problem in your configuration. Check that "Always announce default router" is checked in Network->Interfaces->Lan->DHCP server->IPv6 Settings. When you say public IP is your freebox, you mean from OpenMPTCProuter or from internet ? It's an IPv4 or IPv6 ?

flimo44 commented 3 years ago

Hello, @Ysurac I don't have any IPv6 problems, I think you have a problem in your configuration. ->Possible but same configuration is OK in beta6

Check that "Always announce default router" is checked in Network->Interfaces->Lan->DHCP server->IPv6 Settings. -> Yes it's checked

When you say public IP is your freebox, you mean from OpenMPTCProuter or from internet ? It's an IPv4 or IPv6 ? -> when I test my public Ip on sites like http://ip.lafibre.info/ or http://www.mon-ip.com/ from my PC or mobile, it is my IP v4 freebox wan and not that of my VPS that is seen . In beta6 (ipv6 OK) it is indeed that of the VPS. : No problem

Ysurac commented 3 years ago

@flimo44 the only change about IPv6 after beta6 is the new "DNS64" checkbox, but this doesn't change anything when not used. How is the OpenMPTCProuter status page ? All is green ? If it's ok, check gateway you use on PC or mobile.

flimo44 commented 3 years ago

@flimo44 the only change about IPv6 after beta6 is the new "DNS64" checkbox, but this doesn't change anything when not used. How is the OpenMPTCProuter status page ? All is green ? If it's ok, check gateway you use on PC or mobile.

Yes for beta6 Yes for RC1 or RC2 : just no ipV6 and this error in continuous system log: Sat May 15 00:05:20 2021 daemon.warn odhcpd [5968]: A default route is present but there is no public prefix on lan thus we don't announce a default route ! Sat May 15 00:05:20 2021 daemon.err odhcpd [5968]: Failed to send to ff02 :: 1% lan @ eth0 (Permission denied)

Gateway is OpenMPTCProuter 192.168.100.1

Ex : je viens de lancer un tracert sur mon mobile : Hop 1 : OpenMPTCProuteur Hop 2 : 10.255.255.1 Hop 3 : 51.xx.xx.1 ( gateway VPS OVH ) Hop 4 : 192.168.143.254 .....

So all is well on the route. Except that if I go to any site that gives its Public IP, it is that of my freebox V6 that appears (19 times / 20). From time to time but very rarely that of my VPS. Really weird. I'm going to dig because it really intrigues me.

Ysurac commented 3 years ago

@Malaga82 I found the issue: Now you can use MPTCP over VPN on all interfaces and it's aggregated, but if only one interface use it, it's not aggregated. I will fix that. @flimo44 For TCP traffic the proxy is used (by default shadowsocks), the only way to not use it (and not use the VPN when proxy is down) is to bypass with omr-bypass.

Malaga82 commented 3 years ago

@Ysurac sorry if i was not detailed enough. I have 3 WAN, only 1 sometimes get MPTCP blocked ( it's a virtual ISP, it can get 2 different ASN, one of two block MPCTP ), and with OpenVPN it bypasses the block, not with WireGuard. I'm testing with 1 WAN over VPN, if you want i can test with 2 WAN over VPN. When WireGuard is enabled for WAN, it will say "Gateway DOWN", because PK is not setted up: image and WireGuard status shows only PK, Listen Port and FW Mark, not the connection status. Once edited WGWAN* with PK from VPS, with "wg show", it wll start "works": image

Regards.

flimo44 commented 3 years ago

Hello @Ysurac

I just installed the latest upgraded RC2 on beta6 and it looks not too bad. No more error in the system log, IPV6 OK. Regarding my packages they are not reinstalled automatically after the update. Only problem encountered, I can no longer reinstall the qemu-ga package because dependencies are missing.

qemu-ga

Best regards