Alternate phantomjs provider?

[scottcain]

Hi,

Great work so far; I installed it in a staging server and it works right out of the box. Have you considered allowing alternate phantomjs providers? The folks at WormBase are concerned that on a busy day (say, right before a conference), the 500 free screenshots would be used up and then users would be greeted with an embarrassing (for us) message that "you used all all of your free screenshots" message. I see that there are a few web services-type projects listed in the phantomjs related projects page: http://phantomjs.org/related-projects.html Perhaps one of those could be configured to be a drop in replacement, and the screenshot plugin could just take a url via configuration for where to go.

Not that I'm necessarily suggesting that you do this now or ever, but if it's easy you could (in your copious free time). But if you don't, I may do it sometime in the next few months (with my copious free time :-)

Thanks!

[bhofmei]

I’m glad it works right out of the box. This plugin has been difficult to test and can be a bit temperamental.

As far as alternative services, I haven’t looked into it much recently. When I was originally getting the plugin up and running, I know I tried out quite a few options and PhantomJS Cloud was by far the easiest to use, test, and configure. A lot required have phantomJS installed on the server and/or had poor documentation. I had no plans to add alternate providers since PhantomJS cloud works so well.

If your only concern is the 500 free screenshot limit, that is a fairly easy fix. One thing to consider is each IP address gets 100 free screenshots a day using the demo API key. That’s what we do in our lab. Another option would be to require each WormBase user to get a PhantomJS cloud account so each user gets 500 free screenshots a day. Implementing that wouldn’t be too difficult but would be more annoying for your users.

A final option would be to output the configured URL for the webpage that a user could copy and paste into a different service. This wouldn’t be too difficult to implement either.

On Feb 8, 2017, at 11:02 AM, Scott Cain notifications@github.com wrote:

Hi,

Great work so far; I installed it in a staging server and it works right out of the box. Have you considered allowing alternate phantomjs providers? The folks at WormBase are concerned that on a busy day (say, right before a conference), the 500 free screenshots would be used up and then users would be greeted with an embarrassing (for us) message that "you used all all of your free screenshots" message. I see that there are a few web services-type projects listed in the phantomjs related projects page: http://phantomjs.org/related-projects.html http://phantomjs.org/related-projects.html Perhaps one of those could be configured to be a drop in replacement, and the screenshot plugin could just take a url via configuration for where to go.

Not that I'm necessarily suggesting that you do this now or ever, but if it's easy you could (in your copious free time). But if you don't, I may do it sometime in the next few months (with my copious free time :-)

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bhofmei/jbplugin-screenshot/issues/5, or mute the thread https://github.com/notifications/unsubscribe-auth/APWCOqe5qxQ1zj1Aq5n0ttYAyTN32Nkmks5raecGgaJpZM4L7Aa-.

[scottcain]

I finally have some time to look at allowing an alternate phantomjs provider and am diving into your code, particularly Util.js. The service I'm looking at providing will be an internal implementation of Manet (https://github.com/vbauer/manet) which seems to work pretty well when I run it interactively via a web form.

The URL that it requires is a little different, so I need to significantly alter how Util builds it. Can you give me some guidance for how to start? For example, if you could tell me (or show me) how to have it spit out the constructed JBrowse url to a dialog box (as you suggested above), I think that would get me 90% of the way to where I need to be.

Thanks much, Scott

[bhofmei]

Building the URL for the screenshot comes in 2 parts

1. local parameters to be remembered for JBrowse
2. request parameters for the phantomjs provider

In the dialog box, almost all of the "output configurations" have to do with phantomjs parameters. Zoom factor is the only one that needs to be part of both.

The final URL gets built in 2 steps. In the Util.js file, encode is for JBrowse parameters and encodePhantomJSSettings is for phantomJS parameters. Most likely, you will only need to update this function to be able to connect it to Monet. Encoding for JBrowse parameters basically gives you the current URL (which is what we want the screenshot of) and local parameters for JBrowse to implement when building the page (rather than using only configurations on the server).

Having it output the url in the dialog box requires a few steps to ScreenShotDialog.js

1. Add 'dijit/form/TextBox' and "dijit/registry" as TextBox and registry to the declare/function statement at the to of the file.
2. In the show function, under var paneFooter..., add

   var p = dom.create('div',{});
   var urltextbox = new TextBox({
   id: 'url-text-box',
   value: ''
   });
   urltextbox.placeAt(p);

   and add p to the content as

       this.set('content', [
           mainPaneTop,
           mainPaneBottom,
           paneFooter,
           p
       ] );

3. In _getPhantomJSUrl at the bottom, comment out these lines

   //var jsEncode = Util.encodePhantomJSSettings(jsParams);
   //return this.requestUrl + jsEncode;

   and add this line

   return currentUrl;

4. In _fillActionBar, comment out the line

   //window.open(url);

   and add the line

   registry.byId('url-text-box').set("value",url);

I didn't use very good names for these variables and it's not going to have pretty CSS style, but it will get the job done. Now when you click "Render" in the dialog box, there is a textbox that will fill with the URL. For whatever reason, it requires clicking "Render" twice the first time to output the URL. Subsequent changes to parameters will change in the URL when clicking "Render".

Hopefully this helps! Let me know if you have any other questions.

[scottcain]

Thanks so much for the clear description. I suspect getting this working is going to be even easier than I thought: it looks like I just need to intercept renderSettings and instead of creating stringified json, I need to create GET style tag-value pairs (?url=htt...&zoom=2&format=png ...etc...). I'm guessing there's even a dojo function that will do that for me; now I just have to find it :-)

[scottcain]

OK, definitely zeroing in on it: dojo.objectToQuery is my friend, and I can grab every piece of info that would get passed to phantomjscloud and save it in a GET style string, so I have:

pdfOptionsGET (format=letter&orientation=landscape) viewPortGET (width=1800&height=1200) renderDictGET (zoomFactor=2&quality=70)

and lastly, outDictGET which has the url and renderType. One question about the URL: in it I see "screenshot" followed by and equals sign and a lot of stuff (example below). I'm guessing there's stuff hidden in there, probably to do with modifying track defaults. Do I need to parse that out, or just pass it along to phantomjs and it uses it to create the appropriate image.

What the URI escaped URL looks like:

http%3A%2F%2Fstaging.wormbase.org%2Ftools%2Fgenome%2Fjbrowse-simple%2F%3Fdata%3Ddata%252Fc_elegans_PRJNA13758%2526loc%3DIII%253A13695021..13727224%2526tracks%3DCurated_Genes%252CTSS%2520(Chen)%2520Forward%252CBalancers%252CPolymorphisms%2526highlight%3D%2526screenshot%3Dp20o1r0n1u1b1m111s000000z2~0h2500i0q0d0~1h50i0x25q1~2h600i0q0d0~3h600i0q0d0

[bhofmei]

The stuff followed by “screenshot=“ is for modifying track defaults. As long as the screenshot plugin is activated, you don’t need to do anything with this.

If you want a bit more detail, it is part of the URL that is getting sent to phantomjs. When phantomjs requests the URL, there is logic in the screenshot plugin to decode the string of characters following “screenshot=“ and make those changes to the track configurations. Applying configurations occurs in “main.js” (~ line 1829) and the actual decoding is done by the decode function (~ line 30) in “Util.js”.

If you are interested, I can explain what everything in that string means if you are curious but it isn’t necessary for having the plugin actually work.

On Mar 8, 2017, at 12:39 PM, Scott Cain notifications@github.com wrote:

OK, definitely zeroing in on it: dojo.objectToQuery is my friend, and I can grab every piece of info that would get passed to phantomjscloud and save it in a GET style string, so I have:

pdfOptionsGET (format=letter&orientation=landscape) viewPortGET (width=1800&height=1200) renderDictGET (zoomFactor=2&quality=70)

and lastly, outDictGET which has the url and renderType. One question about the URL: in it I see "screenshot" followed by and equals sign and a lot of stuff (example below). I'm guessing there's stuff hidden in there, probably to do with modifying track defaults. Do I need to parse that out, or just pass it along to phantomjs and it uses it to create the appropriate image.

What the URI escaped URL looks like:

http%3A%2F%2Fstaging.wormbase.org%2Ftools%2Fgenome%2Fjbrowse-simple%2F%3Fdata%3Ddata%252Fc_elegans_PRJNA13758%2526loc%3DIII%253A13695021..13727224%2526tracks%3DCurated_Genes%252CTSS%2520(Chen)%2520Forward%252CBalancers%252CPolymorphisms%2526highlight%3D%2526screenshot%3Dp20o1r0n1u1b1m111s000000z2~0h2500i0q0d0~1h50i0x25q1~2h600i0q0d0~3h600i0q0d0

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bhofmei/jbplugin-screenshot/issues/5#issuecomment-285111811, or mute the thread https://github.com/notifications/unsubscribe-auth/APWCOquFAJqeQqIAESMgN1b1PNFRrcdAks5rjufKgaJpZM4L7Aa-.

[scottcain]

OK, really close now. With this commit: https://github.com/scottcain/jbplugin-screenshot/commit/d4f73d3c8700ab77c4ecafed31e564282de1b411 the plugin works with the dockerized version of manet, but I had to break the phantomjscloud functionality to do it. What I'd really like to do is get the optional "customURL" parameter added to params that gets passed into encodePhantomJSSettings so that I can make the changes conditional (and then can give you a pull request), but it's not clear to me how that gets populated. Can you clarify that for me?

[bhofmei]

I don’t think I understand what you are asking. Part of my confusion I think is just I don’t know how the syntax for manet parameters or how manet would work with this. Maybe if I explain how the encodePhantomJSSettings gets it’s information, you’ll have a better idea of where the conditional information needs to be.

For PhantomJS cloud, to get a screenshot you create a URL with this set up: https://phantomjscloud.com/api/browser/v2/apiKey/?request={url:”url”,renderType:”type”,renderSettings:{ect}}

The underlined portion of that is the “requestUrl” mentioned in “_getPhantomJSUrl” (~ line 522) from ScreenShotDialog.js. The apiKey is a plugin parameter. The request url is constructed in the “_getPhantomJSUrl” (~ line 1866) function of main.js and passed to ScreenShotDialog as the requestUrl parameter (~ line 1843). The non-underlined part is what is returned by “encodePhantomJSSettings” from Util.js which includes the url of JBrowse and rendering parameters for PhantomJS cloud.

I’m going to assume that manet accepts parameters in a very different format than the url shown above, so having a customURL to “encodePhantomJSSettings” probably won’t work. The better option probably, would be to leave “encodePhantomJSSettings” as is and write a new function “encodeManetSettings” in Util.js. Then, add the conditional in “_getPhantomJSUrl” to either call “enchodePhantomJSSettings” or “encodeManetSettings”. There would probably need to be an additional plugin parameter to determine which service to use (or hijack apiKey somehow) and pass that parameter to ScreenShotDialog. You’d probably need to make some minor changes to the ScreenShotDialog constructor as well.

I know that was a lot of information, hopefully it helps.

On Mar 9, 2017, at 11:43 AM, Scott Cain notifications@github.com wrote:

OK, really close now. With this commit: scottcain@d4f73d3 https://github.com/scottcain/jbplugin-screenshot/commit/d4f73d3c8700ab77c4ecafed31e564282de1b411 the plugin works with the dockerized version of manet, but I had to break the phantomjscloud functionality to do it. What I'd really like to do is get the optional "customURL" parameter added to params that gets passed into encodePhantomJSSettings so that I can make the changes conditional (and then can give you a pull request), but it's not clear to me how that gets populated. Can you clarify that for me?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bhofmei/jbplugin-screenshot/issues/5#issuecomment-285407193, or mute the thread https://github.com/notifications/unsubscribe-auth/APWCOvujB0VF4xGp0yXH_onUa167tTfBks5rkCw2gaJpZM4L7Aa-.

[scottcain]

Thanks for this--yes it helps!

I already have a "customURL" parameter that is added in the jbrowse plugin config (via this commit https://github.com/scottcain/jbplugin-screenshot/commit/ad28e2d52aada5f28528fac4ba591074587f6021) so I was hoping that I could modify the string returned by Util/encodePhantomJSSettings, especially since I haven't found yet where the url is actually submitted, but since I can change both pieces (the http part and the GET part) before they are assembled, I'd be good. I don't disagree (ugh, with the double negatives) that adding a new method like encodeManetSettings is probably a better idea, since putting in infrastructure in that way would make extending to other services even easier, but I don't have a good enough feeling for the code to make those changes yet. I'll look at ScreenShotDialog.js (the existence of which escaped me until now) and see if I can make this a little cleaner.

[scottcain]

OK, I have it working, though without creating a separate encodeManetSettings method--I just have a conditional in encodePhantomJSSettings to return a different string if the customURL option is specified in jbrowse.conf. I'll add a section to the README describing this option. When I'm done with that, do you want a pull request to bring these code changes into your code base, or would you rather this just remain in my fork for now? Here is the compare page between your master branch and mine: https://github.com/bhofmei/jbplugin-screenshot/compare/master...scottcain:master

The only issue I have remaining with the way this works is that manet doesn't respect the formatting options for PDFs, but I'm pretty sure that's a problem with manet and not this plugin, since I'm using the tag options it specifies in its documentation.

I may also add an option for changing the delay that phantomjs uses before drawing the image. I had to hardcode a value in but it makes sense to allow people to alter that if it's too long or too short. I don't plan on doing that in the near future though.

[bhofmei]

You can do a pull request. I might end up moving the code into it’s own function to keep things modular. But thanks for doing this and updating the README. It’s good to have multiple options for phantomjs. I was avoiding a server-side approach (which I think is what manet is) but this is definitely a better option for some people.

On Mar 9, 2017, at 3:02 PM, Scott Cain notifications@github.com wrote:

OK, I have it working, though without creating a separate encodeManetSettings method--I just have a conditional in encodePhantomJSSettings to return a different string if the customURL option is specified in jbrowse.conf. I'll add a section to the README describing this option. When I'm done with that, do you want a pull request to bring these code changes into your code base, or would you rather this just remain in my fork for now? Here is the compare page between your master branch and mine: master...scottcain:master https://github.com/bhofmei/jbplugin-screenshot/compare/master...scottcain:master The only issue I have remaining with the way this works is that manet doesn't respect the formatting options for PDFs, but I'm pretty sure that's a problem with manet and not this plugin, since I'm using the tag options it specifies in its documentation.

I may also add an option for changing the delay that phantomjs uses before drawing the image. I had to hardcode a value in but it makes sense to allow people to alter that if it's too long or too short. I don't plan on doing that in the near future though.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bhofmei/jbplugin-screenshot/issues/5#issuecomment-285464874, or mute the thread https://github.com/notifications/unsubscribe-auth/APWCOsXk9-XHnqmQvV16v751XObsu60Aks5rkFrygaJpZM4L7Aa-.