Recording fails on enterprise wifi

[mlaethem76]

I am trying to run load tests on a Shiny dashboard hosted on my company's RStudio Connect server. The dashboard pulls data from our organization's Vertica database each time it runs. However, when I try to use the record_session() function, some of the dashboard's UI modules load, but the main display dataframe is not pulled in from Vertica. In order to access RStudio Connect, I have to be logged into my company's wifi, but the same thing happens when trying to use pre-loaded R datasets (such as geyser). I have reached out to RStudio Connect support team, but they told us to open an issue here so that we could contact the package maintainers and authors.

[ajwtech]

Can you tell us a bit more about the call you are trying to make to the session recorded.

• Are you using a local Rstudio install or a hosted server?
• Are you trying to pass anything in the query of the url?
• does the connect server have anything in the error log for the running session

A reproducible example would be very helpful here. I would also try running record session with open_browser = FALSE and see if you can navigate to the page manually.

[mlaethem76]

• The function call I am trying to make looks like this: record_session(https://connect.xxx.massmutual.com/content/xxx/)

• No I am not trying to pass anything in the query of the url. Right now I'm just trying to get the app to display non-UI elements while in the recording session.

• There are no errors in the error log for the running session. For reference, I have attached the recording.log file that is generated when I use record_session(https://connect.xxx.massmutual.com/content/xxx/).

• I am not sure how it would be possible to create a reproducible example, because to access the application and the RStudio Connect server, one needs to be on the company's proprietary internet connection, which can only be accessed using company credentials. I tried to contact the support team for our RStudio Connect server, but they told me that the best option would be asking my question here.

• Also, I did try running the record_session function with open_browser = FALSE and navigating to the page manually, but the problem persisted

What I see when the application opens after running record_session(https://connect.xxx.massmutual.com/content/xxx/):

The recording.log file: recording.log

[ajwtech]

You said that you can recreate the problem when using geysers dataset right? That might allow someone to reproduce it. How do you connect to vertica from R? The reason I ask is that this package sets up a proxy server to handle recording the actions between your browser and the connect server. I am wondering if a server is not being called correctly. @alandipert might have better intuition on what might be causing this off hand.

[mlaethem76]

We connect to Vertica using the RJDBC package and the batch ID and password that allows us to access the schemas needed for each project. On RStudio Connect, we have this batch ID and password injected as environment variables. And yes we had the same issue with the geysers data. I uploaded the dummy geysers example app from shiny to our RStudio Connect server and got the same issue as above. I was trying to see if this was only an issue when the application pulled data from Vertica, but it appears to be happening with any app that we upload to our RStudio Connect server.

[ajwtech]

Ok I had a similar issue with Connect a week ago and put some fixes in place for it. There is currently a pull request open for those fixes but in the mean time could you try using my fork of this repo to see if it works for you https://github.com/ajwtech/shinyloadtest Please understand that this fork is just for your troubleshooting and that fork will not be maintained and also I do not work for Rstudio or represent them. Just trying to help out when I can.

[alandipert]

@mlaethem thank you for the report. I'm curious about the following things:

1. Do you notice any errors in your application's log in Connect after attempting a recording?
2. Do you see any errors in the JavaScript console in the browser?
3. Which packages are responsible for displaying the parts of the UI that are missing? It might be that these packages are (not yet) compatible with shinyloadtest.

Thanks for the additional information, hopefully we can get to the bottom of this.

[mlaethem76]

Hi Alan, firstly thank you so much for responding to this issue! I really appreciate it.

1. No, there are no errors in the application's log in RStudio Connect after I attempt the recording

2. There is an error in the JS console on the browser. I have attached an image of the error below:

   

3. This is interesting because we currently use an in-house R package for our DS work called mmlib. The data is pulled in using this package, which connects to our Vertica database through the RJDBC package. Just for thoroughness, the app the following packages in addition to mmlib: shiny, dplyr, lubridate, DT, shinyWidgest, shinyalert, plotly, jsonlite, and stringr.

[alandipert]

@mlaethem no problem, happy to help 😄

Hm, that JS console error is telling. It indicates to me that SockJS (a JavaScript library used by Shiny in Connect to communicate with the server) is selecting a transport other than WebSockets for communication.

Transports other than WebSocket are not currently supported.

Usually SockJS decides not to use WebSockets when they're not available for some reason. Frequently the reason is that Connect is behind a proxy or load balancer that does not understand WebSockets.

Do you know if this could be the case?

One thing you could do to check is go to your application normally and look at the Network tab of the developer console. If WebSockets are being used, you should see that in the log:

Another way to rule out the lack of WebSockets would be to visit your application normally and specify manually which transport to use.

If it turns out that WebSockets are not available to you, then unfortunately record_session won't work. We would love eventually to support another transport, but it would require a significant amount of effort.

Otherwise, if you see WebSockets are available and working, then the problem could be something else and I'll need to think harder 😄

[mlaethem76]

When I open the developer console while running the app, the only transport I see isxhr not websocket:

I saw that the link you posted about manually specifying which transport to use, but it appears that the page is referencing Shiny Server Pro, and I am using RStudio Connect. I'm not sure if this makes a difference, I just wanted to make sure we were on the same page!

[alandipert]

@mlaethem ah, yes the documentation is incidentally on the Shiny Server Pro docs, but the bit about changing the transport in the browser is common to both Connect and SSP.

Based on that screenshot... it definitely looks like Shiny is using XHR and not WebSocket, which explains why record_session isn't working ☹️

Other than a WebSocket-unaware proxy or load balancer, it's possible Connect is configured explicitly to disable WebSockets for some reason. Could that be the case? The relevant configuration option is DisabledProtocols listed at https://docs.rstudio.com/connect/1.5.4/admin/appendix-configuration.html

[mlaethem76]

It is definitely possibly that Connect is configured to disable WebSockets. Maintenance of Connect within data science has just shifted to @fzwaeustc so I think he would be able to shed some more light on this.

It's looking like this is definitely an issue on our end Alan; thank you so much for helping us to figure out what is going on! We really appreciate it.

[alandipert]

@mlaethem I shared your issue with the team this morning and @jcheng5 made a suggestion. He pointed out that in your first screenshot, there's a syntax error in Chrome for sockjs-0.3.min.js. If you see that error every time, it's almost certainly why nothing else works. If SockJS isn't able to load, then the browser won't be able to communicate with the server.

In Chrome when you click on the sockjs-0.3.min.js, does it look like the file was truncated or is otherwise malformed?

This would be a separate and more fundamental issue than lack of WebSocket support, and could indicate a bug in shinyloadtest.

Sorry for the red herring, and thanks in advance for your continued sleuthing!

[mlaethem76]

Of course! I'm happy to help with this in any way I can. I'd really like to introduce load testing into our formal development workflow, and I'm a big fan of both RStudio and Shiny!

We've done testing on a few different machines, and it looks like the sockjs-0.3.min.js error is appearing every time. Upon my cursory inspection, it looks like there is a missing ) at the end of the argument list, which is exactly what the error message in the console said. Here are the contents of that file from Chrome's developer tools:

var JSON;JSON||(JSON={}),function(){function str(a,b){var c,d,e,f,g=gap,h,i=b[a];i&&typeof i=="object"&&typeof i.toJSON=="function"&&(i=i.toJSON(a)),typeof rep=="function"&&(i=rep.call(b,a,i));switch(typeof i){case"string":return quote(i);case"number":return isFinite(i)?String(i):"null";case"boolean":case"null":return String(i);case"object":if(!i)return"null";gap+=indent,h=[];if(Object.prototype.toString.apply(i)==="[object Array]"){f=i.length;for(c=0;c<f;c+=1)h[c]=str(c,i)||"null";e=h.length===0?"[]":gap?"[\n"+gap+h.join(",\n"+gap)+"\n"+g+"]":"["+h.join(",")+"]",gap=g;return e}if(rep&&typeof rep=="object"){f=rep.length;for(c=0;c<f;c+=1)typeof rep[c]=="string"&&(d=rep[c],e=str(d,i),e&&h.push(quote(d)+(gap?": ":":")+e))}else for(d in i)Object.prototype.hasOwnProperty.call(i,d)&&(e=str(d,i),e&&h.push(quote(d)+(gap?": ":":")+e));e=h.length===0?"{}":gap?"{\n"+gap+h.join(",\n"+gap)+"\n"+g+"}":"{"+h.join(",")+"}",gap=g;return e}}function quote(a){escapable.lastIndex=0;return escapable.test(a)?'"'+a.replace(escapable,function(a){var b=meta[a];return typeof b=="string"?b:"\\u"+("0000"+a.charCodeAt(0).toString(16)).slice(-4)})+'"':'"'+a+'"'}function f(a){return a<10?"0"+a:a}"use strict",typeof Date.prototype.toJSON!="function"&&(Date.prototype.toJSON=function(a){return isFinite(this.valueOf())?this.getUTCFullYear()+"-"+f(this.getUTCMonth()+1)+"-"+f(this.getUTCDate())+"T"+f(this.getUTCHours())+":"+f(this.getUTCMinutes())+":"+f(this.getUTCSeconds())+"Z":null},String.prototype.toJSON=Number.prototype.toJSON=Boolean.prototype.toJSON=function(a){return this.valueOf()});var cx=/[\u0000\u00ad\u0600-\u0604\u070f\u17b4\u17b5\u200c-\u200f\u2028-\u202f\u2060-\u206f\ufeff\ufff0-\uffff]/g,escapable=/[\\\"\x00-\x1f\x7f-\x9f\u00ad\u0600-\u0604\u070f\u17b4\u17b5\u200c-\u200f\u2028-\u202f\u2060-\u206f\ufeff\ufff0-\uffff]/g,gap,indent,meta={"\b":"\\b","\t":"\\t","\n":"\\n","\f":"\\f","\r":"\\r",'"':'\\"',"\\":"\\\\"},rep;typeof JSON.stringify!="function"&&(JSON.stringify=function(a,b,c){var d;gap="",indent="";if(typeof c=="number")for(d=0;d<c;d+=1)indent+=" ";else typeof c=="string"&&(indent=c);rep=b;if(!b||typeof b=="function"||typeof b=="object"&&typeof b.length=="number")return str("",{"":a});throw new Error("JSON.stringify")}),typeof JSON.parse!="function"&&(JSON.parse=function(text,reviver){function walk(a,b){var c,d,e=a[b];if(e&&typeof e=="object")for(c in e)Object.prototype.hasOwnProperty.call(e,c)&&(d=walk(e,c),d!==undefined?e[c]=d:delete e[c]);return reviver.call(a,b,e)}var j;text=String(text),cx.lastIndex=0,cx.test(text)&&(text=text.replace(cx,function(a){return"\\u"+("0000"+a.charCodeAt(0).toString(16)).slice(-4)}));if(/^[\],:{}\s]*$/.test(text.replace(/\\(?:["\\\/bfnrt]|u[0-9a-fA-F]{4})/g,"@").replace(/"[^"\\\n\r]*"|true|false|null|-?\d+(?:\.\d*)?(?:[eE][+\-]?\d+)?/g,"]").replace(/(?:^|:|,)(?:\s*\[)+/g,""))){j=eval("("+text+")");return typeof reviver=="function"?walk({"":j},""):j}throw new SyntaxError("JSON.parse")})}()

SockJS=function(){var a=document,b=window,c={},d=function(){};d.prototype.addEventListener=function(a,b){this._listeners||(this._listeners={}),a in this._listeners||(this._listeners[a]=[]);var d=this._listeners[a];c.arrIndexOf(d,b)===-1&&d.push(b);return},d.prototype.removeEventListener=function(a,b){if(!(this._listeners&&a in this._listeners))return;var d=this._listeners[a],e=c.arrIndexOf(d,b);if(e!==-1){d.length>1?this._listeners[a]=d.slice(0,e).concat(d.slice(e+1)):delete this._listeners[a];return}return},d.prototype.dispatchEvent=function(a){var b=a.type,c=Array.prototype.slice.call(arguments,0);this["on"+b]&&this["on"+b].apply(this,c);if(this._listeners&&b in this._listeners)for(var d=0;d<this._listeners[b].length;d++)this._listeners[b][d].apply(this,c)};var e=function(a,b){this.type=a;if(typeof b!="undefined")for(var c in b){if(!b.hasOwnProperty(c))continue;this[c]=b[c]}};e.prototype.toString=function(){var a=[];for(var b in this){if(!this.hasOwnProperty(b))continue;var c=this[b];typeof c=="function"&&(c="[function]"),a.push(b+"="+c)}return"SimpleEvent("+a.join(", ")+")"};var f=function(a){var b=this;b._events=a||[],b._listeners={}};f.prototype.emit=function(a){var b=this;b._verifyType(a);if(b._nuked)return;var c=Array.prototype.slice.call(arguments,1);b["on"+a]&&b["on"+a].apply(b,c);if(a in b._listeners)for(var d=0;d<b._listeners[a].length;d++)b._listeners[a][d].apply(b,c)},f.prototype.on=function(a,b){var c=this;c._verifyType(a);if(c._nuked)return;a in c._listeners||(c._listeners[a]=[]),c._listeners[a].push(b)},f.prototype._verifyType=function(a){var b=this;c.arrIndexOf(b._events,a)===-1&&c.log("Event "+JSON.stringify(a)+" not listed "+JSON.stringify(b._events)+" in "+b)},f.prototype.nuke=function(){var a=this;a._nuked=!0;for(var b=0;b<a._events.length;b++)delete a[a._events[b]];a._listeners={}};var g="abcdefghijklmnopqrstuvwxyz0123456789_";c.random_string=function(a,b){b=b||g.length;var c,d=[];for(c=0;c<a;c++)d.push(g.substr(Math.floor(Math.random()*b),1));return d.join("")},c.random_number=function(a){return Math.floor(Math.random()*a)},c.random_number_string=function(a){var b=(""+(a-1)).length,d=Array(b+1).join("0");return(d+c.random_number(a)).slice(-b)},c.getOrigin=function(a){a+="/";var b=a.split("/").slice(0,3);return b.join("/")},c.isSameOriginUrl=function(a,c){return c||(c=b.location.href),a.split("/").slice(0,3).join("/")===c.split("/").slice(0,3).join("/")},c.getParentDomain=function(a){if(/^[0-9.]*$/.test(a))return a;if(/^\[/.test(a))return a;if(!/[.]/.test(a))return a;var b=a.split(".").slice(1);return b.join(".")},c.objectExtend=function(a,b){for(var c in b)b.hasOwnProperty(c)&&(a[c]=b[c]);return a};var h="_jp";c.polluteGlobalNamespace=function(){h in b||(b[h]={})},c.closeFrame=function(a,b){return"c"+JSON.stringify([a,b])},c.userSetCode=function(a){return a===1e3||a>=3e3&&a<=4999},c.countRTO=function(a){var b;return a>100?b=3*a:b=a+200,b},c.log=function(){b.console&&console.log&&console.log.apply&&console.log.apply(console,arguments)},c.bind=function(a,b){return a.bind?a.bind(b):function(){return a.apply(b,arguments)}},c.flatUrl=function(a){return a.indexOf("?")===-1&&a.indexOf("#")===-1},c.amendUrl=function(b){var d=a.location;if(!b)throw new Error("Wrong url for SockJS");if(!c.flatUrl(b))throw new Error("Only basic urls are supported in SockJS");return b.indexOf("//")===0&&(b=d.protocol+b),b.indexOf("/")===0&&(b=d.protocol+"//"+d.host+b),b=b.replace(/[/]+$/,""),b},c.arrIndexOf=function(a,b){for(var c=0;c<a.length;c++)if(a[c]===b)return c;return-1},c.arrSkip=function(a,b){var d=c.arrIndexOf(a,b);if(d===-1)return a.slice();var e=a.slice(0,d);return e.concat(a.slice(d+1))},c.isArray=Array.isArray||function(a){return{}.toString.call(a).indexOf("Array")>=0},c.delay=function(a,b){return typeof a=="function"&&(b=a,a=0),setTimeout(b,a)};var i=/[\\\"\x00-\x1f\x7f-\x9f\u00ad\u0600-\u0604\u070f\u17b4\u17b5\u200c-\u200f\u2028-\u202f\u2060-\u206f\ufeff\ufff0-\uffff]/g,j={"\0":"\\u0000","\x01":"\\u0001","\x02":"\\u0002","\x03":"\\u0003","\x04":"\\u0004","\x05":"\\u0005","\x06":"\\u0006","\x07":"\\u0007","\b":"\\b","\t":"\\t","\n":"\\n","\x0b":"\\u000b","\f":"\\f","\r":"\\r","\x0e":"\\u000e","\x0f":"\\u000f","\x10":"\\u0010","\x11":"\\u0011","\x12":"\\u0012","\x13":"\\u0013","\x14":"\\u0014","\x15":"\\u0015","\x16":"\\u0016","\x17":"\\u0017","\x18":"\\u0018","\x19":"\\u0019","\x1a":"\\u001a","\x1b":"\\u001b","\x1c":"\\u001c","\x1d":"\\u001d","\x1e":"\\u001e","\x1f":"\\u001f",'"':'\\"',"\\":"\\\\","\x7f":"\\u007f","\x80":"\\u0080","\x81":"\\u0081","\x82":"\\u0082","\x83":"\\u0083","\x84":"\\u0084","\x85":"\\u0085","\x86":"\\u0086","\x87":"\\u0087","\x88":"\\u0088","\x89":"\\u0089","\x8a":"\\u008a","\x8b":"\\u008b","\x8c":"\\u008c","\x8d":"\\u008d","\x8e":"\\u008e","\x8f":"\\u008f","\x90":"\\u0090","\x91":"\\u0091","\x92":"\\u0092","\x93":"\\u0093","\x94":"\\u0094","\x95":"\\u0095","\x96":"\\u0096","\x97":"\\u0097","\x98":"\\u0098","\x99":"\\u0099","\x9a":"\\u009a","\x9b":"\\u009b","\x9c":"\\u009c","\x9d":"\\u009d","\x9e":"\\u009e","\x9f":"\\u009f","\xad":"\\u00ad","\u0600":"\\u0600","\u0601":"\\u0601","\u0602":"\\u0602","\u0603":"\\u0603","\u0604":"\\u0604","\u070f":"\\u070f","\u17b4":"\\u17b4","\u17b5":"\\u17b5","\u200c":"\\u200c","\u200d":"\\u200d","\u200e":"\\u200e","\u200f":"\\u200f","\u2028":"\\u2028","\u2029":"\\u2029","\u202a":"\\u202a","\u202b":"\\u202b","\u202c":"\\u202c","\u202d":"\\u202d","\u202e":"\\u202e","\u202f":"\\u202f","\u2060":"\\u2060","\u2061":"\\u2061","\u2062":"\\u2062","\u2063":"\\u2063","\u2064":"\\u2064","\u2065":"\\u2065","\u2066":"\\u2066","\u2067":"\\u2067","\u2068":"\\u2068","\u2069":"\\u2069","\u206a":"\\u206a","\u206b":"\\u206b","\u206c":"\\u206c","\u206d":"\\u206d","\u206e":"\\u206e","\u206f":"\\u206f","\ufeff":"\\ufeff","\ufff0":"\\ufff0","\ufff1":"\\ufff1","\ufff2":"\\ufff2","\ufff3":"\\ufff3","\ufff4":"\\ufff4","\ufff5":"\\ufff5","\ufff6":"\\ufff6","\ufff7":"\\ufff7","\ufff8":"\\ufff8","\ufff9":"\\ufff9","\ufffa":"\\ufffa","\ufffb":"\\ufffb","\ufffc":"\\ufffc","\ufffd":"\\ufffd","\ufffe":"\\ufffe","\uffff":"\\uffff"},k=/[\x00-\x1f\ud800-\udfff\ufffe\uffff\u0300-\u0333\u033d-\u0346\u034a-\u034c\u0350-\u0352\u0357-\u0358\u035c-\u0362\u0374\u037e\u0387\u0591-\u05af\u05c4\u0610-\u0617\u0653-\u0654\u0657-\u065b\u065d-\u065e\u06df-\u06e2\u06eb-\u06ec\u0730\u0732-\u0733\u0735-\u0736\u073a\u073d\u073f-\u0741\u0743\u0745\u0747\u07eb-\u07f1\u0951\u0958-\u095f\u09dc-\u09dd\u09df\u0a33\u0a36\u0a59-\u0a5b\u0a5e\u0b5c-\u0b5d\u0e38-\u0e39\u0f43\u0f4d\u0f52\u0f57\u0f5c\u0f69\u0f72-\u0f76\u0f78\u0f80-\u0f83\u0f93\u0f9d\u0fa2\u0fa7\u0fac\u0fb9\u1939-\u193a\u1a17\u1b6b\u1cda-\u1cdb\u1dc0-\u1dcf\u1dfc\u1dfe\u1f71\u1f73\u1f75\u1f77\u1f79\u1f7b\u1f7d\u1fbb\u1fbe\u1fc9\u1fcb\u1fd3\u1fdb\u1fe3\u1feb\u1fee-\u1fef\u1ff9\u1ffb\u1ffd\u2000-\u2001\u20d0-\u20d1\u20d4-\u20d7\u20e7-\u20e9\u2126\u212a-\u212b\u2329-\u232a\u2adc\u302b-\u302c\uaab2-\uaab3\uf900-\ufa0d\ufa10\ufa12\ufa15-\ufa1e\ufa20\ufa22\ufa25-\ufa26\ufa2a-\ufa2d\ufa30-\ufa6d\ufa70-\ufad9\ufb1d\ufb1f\ufb2a-\ufb36\ufb38-\ufb3c\ufb3e\ufb40-\ufb41\ufb43-\ufb44\ufb46-\ufb4e\ufff0-\uffff]/g,l,m=JSON&&JSON.stringify||function(a){return i.lastIndex=0,i.test(a)&&(a=a.replace(i,function(a){return j[a]})),'"'+a+'"'},n=function(a){var b,c={},d=[];for(b=0;b<65536;b++)d.push(Str```

[alandipert]

@mlaethem hmm - is it possible some proxy in between you and your Connect instance is truncating JavaScript?

Another thing to try is a different browser, or your current browser with all extensions turned off (such as incognito mode)

[mlaethem76]

As a note to your comment above, Alan, we have tried different browsers and Chrome incognito mode with the same result. However, I have been doing some digging since last week and I think we've identified the problem, thanks to all of your suggestions! We currently have an AWS ELB load balancer in front of the RStudio Connect cluster. The ELB's are older and are not WebSockets aware. We are working to switch over to an ALB, but the timeline is still a bit up in the air. Once we get that set up though, I will try to use the package again and see if we have the same issue. I will definitely keep updating this thread!

[alandipert]

@mlaethem hi there, it's been awhile, just wanted to check in 😄

Have you had any success with load testing? Happy to continue to assist if you're still running into problems.