OMR-bypass errors

[darthclide]

Expected Behavior

OMR bypass correctly understands what ports to bypass, and never decides to give up and start forwarding along another interface

Actual Behavior

My game will reconnect through another interface and the logs show the attached message

Steps to Reproduce the Problem

1. Play game
2. Wait a random amount of time

Specifications

• OpenMPTCProuter version: version 0.54.5
• OpenMPTCProuter VPS version: 0.1015
• OpenMPTCProuter platform: RPI4

[darthclide]

Actually, I don't even think your OMR bypass is working at all now. I just swapped over to a hard line to my DSL, and my ping in games immediately went from 100ms to 40ms. Any ideas on 1. How to confirm traffic is going over the correct interface and 2. Why it ignores the OMR bypass rules?

[darthclide]

Extra info that doesn't make any sense:

When I turn on multipath TCP for my DSL while I am streaming to Twitch, my ping in game goes all over from 60ms to 150ms. But when it is turned off, my game settles in around 100ms. How could my ping possibly get better if my stream is trying to use my DSL at the same time as the game?

The fact it dipped down to 60ms shows that there must be some special combination of settings that will get me back there stably. I just need help finding them.

[Ysurac]

What is the settings you put in OMR-ByPass ? (if I play the "steps to reproduce the problem" I see nothing related to OMR-ByPass)

Twitch streaming use TCP or UDP ? And what game are you talking about ? Some use ICMP ping, some use TCP to calculate a latency.

[darthclide]

I just put in a bunch of ports and port ranges just like any other router. The log in my first post shows this.

Twitch uses mostly TCP since the ingest servers are all RTMP. I am talking about War Thunder. I am 90% sure they use standard ICMP because whenever people report bugs on the forums, they always ask them to do ping tests.

But that is beside the point. The fact that I saw my ping drop down to 60ms at random (when the DSL is being used by my stream AND game at the same time) means that this is where my ping should be settling. Not at 100ms (when I have DSL disabled in multipath). I just need to figure out what settings will get me there. And for starters, I think we need to make sure your OMR bypass is working correctly before I go trying to do some QOS on specific ports (which I assume is possible I hope?)

[Ysurac]

For port range, you need to use ":" and not "-". OMR-ByPass is working correctly.

TCP use shadowsocks, UDP/ICMP use the VPN. ICMP Ping is always low priority (and should always be) and I don't know how many connections you have and what type, so it's difficult to say anything about that.

SQM is used and not QOS, But you can install any package you want, it's based on OpenWrt.

[darthclide]

Ah... that may explain my issues then. I have never encountered a single router that uses a colon. My mistake.

I don't know how many connections you have and what type, so it's difficult to say anything about that.

I am unsure what you mean? The only traffic would be my game, twitch stream, and various background things that use less than 50kbps of internet.

But if I am hearing you correctly: You think that my reported ping is incorrect because it is routed over the VPN? Is there a setting I can change so ICMP doesn't use the VPN? (I would only use it for a short test)

But you can install any package you want, it's based on OpenWrt.

I am unsure what you mean by "install any package you want"? Unless you meant I can install QOS manually? I can't find anything about installing packages on the SQM page and this may be overkill for what I need. If you could just help me setup a basic rule that prioritizes traffic over certain ports it would be much appreciated. I assume any SQM rules would apply even when using OMR bypass for those ports?

[Ysurac]

I talk about connections, I know you have at least one ADSL, but how many other ? ICMP will always use the VPS, shadowsocks only work for TCP. But you can set another VPN.

If you really want, you can install QOS or anything you need in System->software menu. You can also try DSCP. And yes SQM apply in all cases.

[darthclide]

Oh, my other connections are: T-mobile, AT&T, and Verizon.

shadowsocks only work for TCP. But you can set another VPN

How would I do this? And I am a little lost on how this will help me? I mean, I still need your help setting up an openvpn config file from ExpressVPN, such that it is used only by specific devices in my network, but that isn't what we are solving right now I don't think?

I will go ahead and try DSCP + fix those port ranges and see how it goes.

[Ysurac]

ok

In System->OpenMPTCProuter and advanced setting checkbox you can choose a VPN, using another VPN can help for a better ping sometimes. Not related to expressvpn.

[darthclide]

I hope you are still around, may I ask how you do port ranges with DSCP? Is it also a colon ":"?

choose a VPN

Could I technically use ExpressVPN? Can you give me a guide on how to do this? The only section of this advanced settings that looks like I can choose something else is under Dead Simple VPN. But I have no idea how to set something on the OpenVPN page to work on the Dead Simple VPN.

[darthclide]

Considering I am seeing these kind of errors all the time:

I really do think it would be wise to try something else other than shadowsocks. It is always crapping out on me even when there is almost no traffic.

[darthclide]

I figured I would try rebooting, but now I get these errors that I think are related to it saying "VPN tunnel DOWN"

How do I fix this "VPN tunnel DOWN" and if you could help me try something other than shadowsocks it would be great. Or if you know how to make shadowsocks more stable I am willing to give that a try. Considering it keeps going up and down within the span of a minute, could explain those "ERR_CONNECTION_CLOSED" messages.

EDIT I disabled all vpns -> restarted router -> turned on glorytun alone and this resolved the "tunnel down" message. But this still doesn't help with shadowsocks constantly going down and resulting in this error coming and going at random:

Heck it took 5 tries to just upload that image right now...

[bnhf]

You need to update your VPS -- you're a number of revisions back. Check the OMR documentation for how it's done based on the version of Linux you're running.

[darthclide]

You need to update your VPS -- you're a number of revisions back. Check the OMR documentation for how it's done based on the version of Linux you're running.

Okay, but I actually ended up just disabling all Multipath WANs and slowly adding them back in one by one and this resolved the issue. Weird, because only a couple weeks ago I did an update on the server. Is there a changelog page somewhere so I can keep track of updates and what changes were made?

Also, it says to "download and run" the script, but I assumed the wget command did both at the same time? Or is there some other command I am supposed to use that is for updating purposes only?

[darthclide]

Another question: Is there any way to prioritize my Twitch stream over everything else? I tried using the domain part of the ingest server: live-ord02.twitch.tv from rtmp://live-ord02.twitch.tv/app/{stream_key} and putting into DSCP domains under CS7 (latency sensitive) and it has no effect on the stream. Also, apparently Halo uses more than the ports specified on the portforward.com because whenever I am in a match my 5500kbps stream is constantly jumping to 6300 and down to 4300. But as soon as I leave a match it stabilizes. Frankly I want either 1. My game to have good ping because OMR bypass correctly sends all data over the non-bonded DSL line or 2. My stream has the rock solid stability even when I am in a match.

[Ysurac]

Shadowsocks is the better way to get a fast TCP. And in most case the issue is not with Shadowsocks. For the VPN tunnel down, did you set an other VPN on tun0 like OpenVPN ? if it's not the case what is the result of ip a on the router via SSH. If what you mean is that you only have OpenVPN and DSVPN as VPN choice, Save the wizard again this will update the available VPN list.

[Ysurac]

You can try to stabilize connections to set some value in SQM upload/download in the wizard. This will limit speed of connection to the value you set here. You can also put a connection out of aggregation in Network->MPTCP so this connection can be only used via omr-bypass.

[darthclide]

Previously I had DSVPN enabled at the same time as glorytun and everything worked fine. But no matter how many times I hit apply on the wizard page my vpn tunnel was always down. I now only have glorytun and shadowsocks activated. This is the result of ip a

You can also put a connection out of aggregation in Network->MPTCP so this connection can be only used via omr-bypass.

Can you explain what you mean by this?

[darthclide]

Also, my issue isn't that my WANs are going all over the place for speed. My issue is that even though I have all ports bypassed to DSL, there is apparently some traffic still being routed over AT&T, T-mobile, or Verizon because as soon as I quit the game or sit in the lobby, my stream goes back to a straight horizontal line across.

Just a reminder, my DSL is currently turned off for bonding. It is only connected into the router for OMR bypass for multiplayer games. I really really don't want to plug in my DSL into my computer directly -> change metric to use it -> Not touch my browser or any other bandwidth sucking app.

[Ysurac]

DSVPN must'nt be enabled at the same time as glorytun (and why enabling it at the same time ?) The problem can be that twitch use most of bandwidth available, it's why I saying that you can disable MPTCP on the DSL line, this line will then be available only for your game and not used at all by twitch.

[darthclide]

Yes, you replied faster than my edit. It is already disabled. And yet Halo and Twitch are both using something on my T-mobile, AT&T, or Verizon because the game has a little lag + my stream has spikes only during matches.

[darthclide]

I just used CurrPorts to see the hostname of what I am connecting to in OBS. I put that into DSCP Domains and tried both super high priority and super low priority. In both cases, it only slightly helped in 1 direction. High priority Twitch = not a straight line, but still small bounces up and down, while my game was very very laggy. Low priority Twitch = game had fewer lag moments (not as good as gaming on pure DSL alone), but my stream was bouncing very wildly. Is there anyway you could help me setup QOS if you think it would help?

[darthclide]

Is it possible that I am overloading OMR-bypass with too many ports? Would be of any help to just use CurrPorts in the middle of a match, and only bypass those ports + set them to higher priority?

[darthclide]

Well I have no idea what is going on. I decided to restart my router tonight to see if that would make my changes activate, and for a little bit I thought it did. My ping was 40-50ms as expected, but I decided to tweak the priority for my Twitch port (1935) and then my ping was back to the incorrect 100ms. I decided "no problem, I will just swap back to my previous settings that worked". But lo and behold, I am on my 4th restart (with 2-3 minutes in between to let the router setup everything) and I can't attain the 40-50ms I had before. Any ideas on what is happening?

[Ysurac]

For OMR-ByPass, to bypass ports iptables is used, so there is no real problem to have many. For the ping, you can try a traceroute -I <whereyouwanttoping> under Linux or tracert <whereyouwanttoping> under windows, this use ICMP and can tell what is the route used for ICMP Ping.

[darthclide]

Hmmm, I just sadly don't know what the server actually is. I think it is hidden behind an AWS.

But while I think I have dealt with my ping issue in game, I still can't wrap my head around why my stream starts to jump around when I am in a match. CurPorts shows War Thunder using 7853, 443 and shows OBS using 1935 and 80. Multiplayer games should have 0 impact on my stream, but they do (Halo and Warframe do the same thing to my stream that War Thunder does). Any ideas?

I am afraid to bump the 1935 port above CS4 because then it interferes with OMR VPN and other traffic. What would you recommend to ensure my stream is untouched by other traffic?

[darthclide]

Okay then... While my new issue I just created still needs help, you can consider this one closed. I randomly forgot to switch my scene from intermission to the game, and I discovered that it is simply OBS developer's usual poor coding skills that is the culprit. Whenever there is a lot of activity on screen (such as an MMO with tanks, planes, and bombs) it makes the bitrate jump around even when CBR is used. Yeah... CONSTANT bit rate according to their logic means "eh, if it is within 1000kbps it will be good". I am now attempting to find a way to change the buffer size to 50 or something, but this is out of your hands. Thanks for helping me learn more about your router though, I am only getting more experience through all of this.

Edit For anyone coming across this thread, my solution was to switch back over from NVENC to X264, and put a custom buffer size of 50kbps (since for whatever dumb reason they don't give you a buffer size option for NVENC). I am betting you could make it a little higher if your internet is not as stable as mine.

Edit 2 I am using this for now: https://github.com/Xaymar/obs-StreamFX/releases/tag/0.8.0a3 I have no idea how long it will be supported for, but not only does it give custom buffer size for NVENC, it also gives you much finer controls over the NVENC encoder.

Edit 3 For those who care, after a few streams I noticed my video do this cycle of "fuzzy->crisp->fuzzy->crisp" every few seconds. I have sadly determined that the buffer size results in this. I am guessing there is probably some complicated way to get the NVENC encoder to work harder in spite of the small buffer size, but I sadly have not found it. Looks like I am stuck with a 5500kbps stream jumping to 6300 and down to 4300 whenever there is a fast-motion game on screen.