Issues with ITSI Screenshot

[rajadain]

We added the ability for the ITSI portal to request screenshots in #700 and #738. The following are known issues with the current implementation:

1. Currently the maps in the Compare view don't render:
   
2. Sliders reflect the correct numeric values in labels, but are always fixed to the mid-point in the screenshot.

3. Taking a screenshot causes the following error message in the iframe-phone library, seen in the developer console:

   cant handle type: htmlFragResponse

4. After selecting a Google basemap (Terrain or Hybrid), the CSS fails to render correctly henceforth in any screenshot until the app is reloaded:
   
5. Google Maps apparently does not allow screenshots without attribution, so in cases when they are set as basemaps we must switch to a different layer under the hood, take the screenshot, and then switch back.
6. It is unclear what the UI is within an activity in ITSI for taking screenshots. While the Lara Interactive sandbox has a "Screenshot" button, I couldn't find it within ITSI itself.

See comments on #738 for more discussion.

[scytacki]

Hi guys, if you want to break these into separate issues it would be easier to discuss them.

For _#_5 take a look here: https://github.com/concord-consortium/shutterbug.js#shutterbug-jquery-custom-events the saycheese and asyouwere events are intended for this. For example we've used them change from webgl rendering to non-webgl rendering. If you can switch the base layer in the background then you could do that on saychesse, and then switch it back on asyouwere.

_#_3 should be harmless

_#_2 will probably require some more discussion

[rajadain]

Thanks @scytacki. Tagging @mmcfarland

[rajadain]

A new issue seen on Staging: taking a screenshot when a user is logged in shows the login screen:

[rajadain]

Another issue is sometimes the screenshot is of the login screen because it is accessing the URL from an endpoint which hasn't logged in.

[scytacki]

Is this something I can help with? The iframe should be returning a html fragment. All scripts should have been stripped out of this fragment by shutterbug.js, so when this fragment is rendered on the server there shouldn't be any resources download by this fragment other than images and css.

If that is all working correctly, then it means the fragment itself is being generated with the login screen.

[rajadain]

Thanks Scott, I have to investigate further. We can't always reproduce this, but thanks for confirming that it is not causing new requests.

[scytacki]

Another helpful debugging tool is to look at the html that is sent to the sever. If you look at the URL for the snapshot image and you change the file extension to .html then you can download/view the html page that was used to generate the image. The server does some processing of this html, so you might find differences between this html and the resulting image, but it might give you a clue. If you want to know the details about the processing the server does, let me know.

[rajadain]

Ooh that's useful! Thanks for the tip.

[rajadain]

Hi Scott, I'm just starting on this now.

Could you explain why I'm seeing the map in the HTML here: https://ccshutterbugtest.s3.amazonaws.com/26f1b57f77b.html

But not in the PNG here: https://ccshutterbugtest.s3.amazonaws.com/26f1b57f77b.png

?

[scytacki]

I don't have a quick answer. I'll have to dig into it a bit. One possibility I can think of, is that mapbox is identifying the 'phantomjs' user agent that is making the request and it is blocking that request.

Another possibility is the 3d transform that is used by leaflet here:

<svg class="leaflet-zoom-animated" width="316" height="113" viewBox="-6 -2 316 113" 
  style="transform: translate3d(-6px, -2px, 0px);">
<g><path stroke-linejoin="round" stroke-linecap="round" fill-rule="evenodd" pointer-events="none" stroke="#fff" stroke-opacity="1" stroke-width="3.5" fill="#000" fill-opacity="0.5" d="M-6 111L-6 -2L310 -2L310 111zM133 19L83 44L132 89L221 43z"></path></g>
</svg>

Phantom.js doesn't handle 3d transforms. Is there an option in Leaflet to disable those transforms? You probably don't want to leave it disabled, but at least that way you can check if that fixes the problem.

Alternatively, does leaflet have an option to generate a static image from a map? If it does (and it works well), that might be the best approach. So on the 'say-cheese' event you'd replace all maps with their images and then return to maps afterwards.

It is useful to setup a system so you can send snapshot.concord.org some html and see what image you get back. This way you can fiddle with the html and see what makes it break and what makes it work. Here are few options to do that:

1. Go into the browsers developer tools and modify the dom in the browser and then use the snapshot button to send that modified dom to the server to see what the result it.
2. You can use send post the html text and width, height and css using postman for Chrome. This will handle some of the encoding for you.
3. You can use curl.

For postman and curl, the tricky/annoying part of is because of how iframes are handled. The iframe content is sent encoded as a data-uri src attribute of an iframe element. You can see this if you inspect the web request to snapshot.concord.org. Or if you look at the source of the link you sent: view-source:https://ccshutterbugtest.s3.amazonaws.com/26f1b57f77b.html. To completely duplicate the environment you'd need to encode your html content inside of the iframe src attribute and then post that to snapshot.concord.org. However you can also send the iframe's content directly without using the iframe tag.

If you do use curl when you send the html to shutterbug you'll need to encode it like a browser does when it sends a form. (postman will do this encoding for you) Here is a shell snippet that does the encoding with python and then adds the result as a curl post param using the --data @- option to curl: https://slack-files.com/T02V91KU0-F0A5YDDQC-d61701cbde

If you are so inclined it would be great to have a little python (or other script) that takes a html file, and the other parameters and posts it to snapshot.concord.org

[rajadain]

Thanks Scott for the detailed answer. I will look into disabling 3D transforms, and testing with the PhantomJS user agent (although that may be less likely).

We have talked about using some sort of image generation for the map in the past, and if we can't get this to work then that conversation may return to fore.

I've been using the Chrome dev tools to edit the DOM and then sending requests. Those are good suggestions about POSTman / curl, and I may have to use one if I am making multiple quick edits.

[rajadain]

I don't think it's the 3D transforms, since we use them in other areas which are rendered correctly, e.g. https://ccshutterbugtest.s3.amazonaws.com/31e2442245d.png sourced from https://ccshutterbugtest.s3.amazonaws.com/31e2442245d.html. Comparing the map element's HTML structure between this and the failing page shows no meaningful differences.

There is a strong likelihood this is a CSS issue. Moving on to investigate that now.

[rajadain]

I've made some progress, although it is difficult since there's almost no way of seeing what PhantomJS sees except by trial and error. By default the Leaflet map wants to be positioned relatively, but if we force override that with static the map renders: https://ccshutterbugtest.s3.amazonaws.com/21ea752f028.png

Although we run in to other layout issues. I'm now going to see why this doesn't happen in other Leaflet maps, which are also positioned relatively. It's probably because of some CSS rule being applied up the chain.

[scytacki]

I haven't tried them, but there are tools for seeing what phantom sees. I don't know if the setup cost will be worth it for you. But if you install phantom on your system, and then tell it render the html, you can launch a dev tools environment to inspect its dom. And there is some tool for getting a snapshot what phantom 'sees'. These are all phantom things not specific to shutterbug so they should have good documentation. :) However, you might still see differences because our phantom is running on linux and your phantom will probably be running on windows or os x. If you want the best possible environment you can use our docker config for snapshot.concord.org located here: https://github.com/concord-consortium/snapshot_app

[rajadain]

Thanks Scott. I'm continuing to use the Lara Interactive Sandbox and send requests there for snapshot testing, so environmentally we're okay.

I think I've identified the root cause: PhantomJS does not support vh and vw units, which we only use in the Compare view. By changing those to percentages, the map renders: https://ccshutterbugtest.s3.amazonaws.com/bad970b7a21.png

It can be seen that the original page does not have that big empty space between columns: https://ccshutterbugtest.s3.amazonaws.com/bad970b7a21.html. This is also because of that width being specified in vw and not pixels or percentages. This change was originally made in #673, and I will talk to @lewfish and see if we can switch to non-viewport based units.