RTPuri2 idea

[rdyar]

Now that there is a backup rtpuri, I am trying out an idea.

The normal problem we have is a user is behind a corp firewall that blocks traffic on random ports like 8181 - which is what we use for our rtp server main port. When this happens a user cannot send the order, we walk them thru how to save it to their desktop and then use our simple uploader to get it to us. It works but is a pain to explain.

Assuming the problem is that all random/non-standard ports are blocked on some corp firewalls, why not use a more normal port? the problem with using a standard port like port 80 is that you can only have one thing listening on a port, and 80 is supposed to be for web traffic. So you cannot use that on your webserver, and I wouldn't want to use that on my internal local ROES server (no way we would allow public web traffic into our network). But what about other ports? my understanding of ports is that they are mostly for the receiving server - not so much the client, and there is nothing magical about port 80 being web traffic - it is just supposed to be web traffic, and to play well with others that is all it is used for. So the ROES client wants to connect to an RTP server on a specific port - lets say port 465, which is normally used for the encrypted connection to email with TLS. If you are using a server that doesn't have a mail server on it using that port, I think it is ok to tell the ROES client to use port 465, and then setup the RTP server to use that port.

By using port 465, maybe offending corp firewalls will let the traffic thru? I don't think they would block that port, and I don't think it would know it is not coming from an email client. If the client is using port 465 in another application, I think that is ok - just like you can use firefox and chrome at the same time - there is more to the connection than just the port number. It is more important I think that the server knows 465 is now RTP. I suppose a good firewall could see a huge file coming thru port 465 from an application it doesn't know and put a stop to it, but if that is the case it failed already on 8181 so what is the harm?

I am testing this now, my main rtpuri is still set to port 8181, but I set rtpuri2 to use port 465 (different IP address) on a simple web server I have setup on AWS. I am hoping that when a corp user tries to send an order, their firewall will block the first rtpuri (8181) but then when the ROES client tries rtpuri2 on port 465 it will work and go thru.

The problem with this is that you will not have any idea that it happened - the web server only has the RTP server on it, it receives the order and nothing happens - you have to have a way to be alerted that there is an order there so you can go get it and drop it into your incoming ROES folder. I already have Syncback Pro running on the web server for something else, so I have it also monitoring for roes orders on the web server, if it finds one it sends it to me and also sends me an email so I know to look for it.

[loktah]

Did you have any luck with testing your idea? We have the same issue for clients behind a firewall and I have never been able to provide a solution for them.

[rdyar]

Well, it doesn't not work. I have yet to get an order on the backup rtp server where I knew it failed on the primary one. I have received orders on the backup, but it seems like they were from orders where the connection failed, so part of the order came on the primary, then it failed, retried, and sent the entire order to the secondary. So I think there is some weird logic where it tried rtp2 when rtp1 was working but then stopped. This wasn't a big deal, but did lead to some confusion. It has happened a few times.

In theory I think it will work, it may just be a matter of working out what the best port to use is. I looked up 465 and it wasn't exactly what I thought it was - I was thinking of 587 - which may be more likely to be open. 465 is a fairly confused port apparently. I still have mine on 465.

Try it! rtp2 itself totally works, and I feel much better that it is there and having it off site is great. You just have to have something like SyncbackPro or maybe dropbox or something on the rtp2 server as you need something to tell you it happened. I suppose you could load the SAS on there too, just to send you an email or something, but that seems like too much.

Just to be clear - I would not use port 465 or 587 on a machine in our labs network for fear of drawing attention to the open port. On my little web server I am ok with the risk.

[loktah]

Hey Ron, Thanks for the quick reply! I currently have ours using RTP1 with 8080 and RTP2 with 8081 however I don't see that solving the issue. I never thought about putting it on dropbox I never knew I could do that. We have a Sonicwall firewall that is highly strict with incoming and seems to prevent intrusions. Our Synology Servers we have within our office are what all of our ro files go to and that has its own firewall which I get block notices with ip addresses from Russia and China multiple times a day. We used to get hacked in the beginning and they would crash our entire network, VOIP phones and lock me out of our servers. But once we locked down every port except for the ports for the ROES Server and for out of office ftp staff we have not had an issue since. Yeah we have 465 locked and we use 587 with SSL encryption only for our outgoing email.

So if I'm understand you correctly you actually have the Roes Transport Software loaded onto a online server? If so what are you using if you don't mind me asking?

[rdyar]

if you are doing 8080 and 8081 then you aren't accomplishing much - and if they are on the same machine then you really aren't doing anything.

Our main rtp is 8181 and runs in the shop pretty well locked down.

rtp2 is running on an AWS T1 micro which is a tiny server in the amazon cloud that is all mine. It costs something like $10/month. I use it for a couple little things as well as running the rtp2 server. I don't use it for email, so setting rtp2 to use 465 or 587 should be ok.

By having it off site then we really get the rtp2 benefit - if our internet is down at the shop, then you can still send an order to us. By having it on 465 or 587 I am hoping to also get the random people who are blocking 8181.

The problem with it is that if an order goes there, we would never know - no emails would go out as we don't run the sas on there. That is where dropbox could come in - if DB was on the server then you could have the rtp2 server save to the dropbox, and then you could see it in dropbox at the shop - but you still wouldn't know to look for it. I have SyncbackPro which is a backup/monitoring app that watches the rtp2 folder on the server and if it sees something come in it sends me an email so I know to look for it.

So you having 465 locked down, that makes me feel like I want to switch to 587. Almost certain people would allow outgoing traffic on 587. I'll switch it and see if it works.

[loktah]

Hey Ron! Thank you for sharing the info, I just spent part of the day setting up a Windows Server on Amazon EC2 and I have successfully got it running. So now the RTP client is on the T1 Micro server, receives the order and then our server app syncs it with our local production server within a few seconds to push the file directly to the incoming folder that Roes Standalone Server is watching. I submitted a very large order and was very impressed with the speed!

This is a huge help to prevent any hangups on our firewall and intermittent network downtime locally. Again thanks for the info, it was a huge help! By the way I ended up using the 465 port which worked without issue behind our Sonicwall Firewall which blocks a ton of stuff.

[rdyar]

cool! the aws stuff is fun. I do the reserved instance (3 years) as I know I will want it no matter what, much cheaper than just the hourly rate. I've had really good luck with aws, basically zero outages over the last few years, though I do monitor it with uptimerobot.

[loktah]

Your right the price is much better running a reserved instance I did the same 3 year account. I will have to look at the uptimerobot to see about setting that up.

[rdyar]

I have updated my rtp2 to port 587, it works but I think it already got spammed for having 587 open, there were obvious email things in the rtp log. Not sure I care though but monitoring it a little more closely.

Joshua, what is 'our server app'? how are you transfering from the external server to the internal one?

One problem I have noticed running rtp1 internally and rtp2 externally is that orders start on rtp1 then stop and restart on rtp2. this can cause a bit of confusion. RTP2 gets the whole order, rtp1 ends up with a partial order.

[loktah]

I have installed a program made by our internal server company "Synology" that runs directly within the DSM. Its called Cloud Sync. This program is installed on the Amazon AWS server and monitors the incoming folder. It then takes the RO file and syncs it to our local server incoming folder.

[loktah]

Do you use a NAS server internally?

[loktah]

By the way I have not had the RTP1 and RTP2 partial file issue like you mention between the two and RTP1 stopping.

[rdyar]

ok, I see. Since you are keeping the 2 in sync you probably won't have the same problem. I am monitoring the server and then it ftps the ro file to me if it finds one, but that is going to a separate folder, no the incoming folder.

It sounds like your way is a 2 way sync, I am only going in one direction. hmm.