great work; ready for an htmlwidget

[timelyportfolio]

I just spotted the slides on Twitter, and this is really great work. I'm happy to convert the vis to htmlwidget if you are up for it. I've been doing one a week at buildingwidgets, and this would be perfect.

I'm assuming it is ok, and I'll start on a forked version. The way your code is structured should make this very easy. Happy to abandon if I hear disapproval.

[kshirley]

Thanks! Creating an htmlwidget version would be great. As I understand it, htmlwidgets will make it more portable and also embeddable in R markdown and the Rstudio html view pane. Is that the main idea as you see it? I still think it's important for users to be able to view the vis as a standalone webpage in their local browser (mainly so that they have the whole screen to view it), so I hope that capability will persist.

Let me know if you have any Q's as you work on the forked version. There may be plenty of things buried in my code that aren't well-explained.

Your buildingwidgets site is cool - I'll def. be reading through it to see all the examples.

best, kenny

[timelyportfolio]

Ok, the first version is done at the forked htmlwidget branch. I tried to stay as close as possible to the original vis minus tweaks to make it work within the confines of an htmlwidget.

With an htmlwidget, you get lots of advantages, including better control over sizing by allowing user full control but also developer opinion. As of now, I did not attempt to exert any developer opinion, and rather chose to fit the svg within the bounds of the containing div for the htmlwidget.

Other advantages include viewing in RStudio Viewer and seamless inclusion into rmarkdown and Shiny. If you want a standalone page, htmlwidgets::saveWidget makes it easy. Some widget authors write an easy access function to perform this save step.

Let me know if you get a chance to check it out.

[timelyportfolio]

I added a little vignette in that branch to demo how easily they work in a rmarkdown document.

[kshirley]

Wow, you made quick work of that! Sounds awesome. I'll have a look tomorrow.

[kshirley]

This looks like a nice start, but I'm seeing some weird behavior when I use the widget:

1. The axis ticks and labels don't show up on the slider. These are svg elements created by the javascript and appended to the slider div. Can these be included using htmlwidgets?
2. The text input box to change k doesn't submit the new value when you click the "submit" button.
3. The tree seems to be centered vertically in the window that I set up (using the height and width arguments from summarytrees_widget()). And the whole thing resizes to fit the window. So when k is small, for example, the sizes of nodes increase. I'd prefer to keep the legend width and node widths fixed (users can change them using the arguments in prepare.vis()), and have the tree smoothly grow from (roughly) the upper left corner of the vis to as far as it needs to grow as k increases.

I know this third concern is considered a feature of htmlwidgets (the re-sizing policy, right?) Is there a way to fix the widths of elements in pixels, and just let the tree grow, rather than be re-sized within a box? For now in my original version I just set the main svg to be very large, like 3000 pixels wide and 5000 pixels high, so that it should accommodate practically any summary tree (for any data) up to a few hundred nodes.

If we can get the widget to look and behave like the page here, for example, that would be great!

http://www.research.att.com/~kshirley/summarytrees/gauss

[timelyportfolio]

@kshirley thanks so much for the feedback and the example. Unsurprisingly, there were lots of bugs in that first implementation.

1. and 2.

Numbers 1. and 2. arose from simple selector bugs that I had introduced as I moved to classes instead of id, since many times htmlwidgets will appear along with other HTML and id conflicts can arise very easily. They should now be resolved.

3. Sizing, the big question

I thought my nifty viewBox solution was a feature :), but I guess not. While setting svg to some massive height and width to allow the tree to grow is one approach. It seems like a very suboptimal viewing experience especially when considering the case of integrating the visualization in say a research paper with rmarkdown or in a slideshow where you want the vis to be contained within some bounds. In either case, I'm wondering if adding zoom/pan might help improve the experience. Also, I could easily add a parameter such as fit, so that both our approaches could be provided. Just in case you did not know, htmlwidgets provide a width and height argument that allows any CSS valid height and width size to customize.

In terms of other options regarding sizing, htmlwidgets give us a fairly robust sizingPolicy. networkD3 for instance by default fills the browser, while qtlcharts specifies a pretty big region.

Lots I guess to think about and debate.

[kshirley]

Re: #1 and #2 - nice. Looks good!

You're right about #3, I think the main challenge is going to be figuring out the sizing.

Adding zoom/pan would be cool -- kind of like a leaflet map visualization?

I hadn't actually looked at the viewBox code that you added before -- I just took a look at it this iteration, and since I'm not familiar with that attribute of an svg element, I'm not really sure what it's doing. Does it basically just re-scale all svg elements to a new scale? I'm not doubting that it's a feature! I'll just have to learn more about it :)

Going forward - when I use the htmlwidget version, I still see a couple funny things.

First, the tree's root is typically very low on the y-axis, and the entropy profile (the light gray box with the curve running from lower-left to top-right) is also placed low, rather than directly below the top div that contains the input elements for k. One possible explanation for this: did you see the shift variable that I'm using in line 507 of summarytrees_htmlwidget.js? I use it to measure the height of the tree, and then shift everything down. The details are a bit complicated and ad hoc. The goal is to move the root of the tree down far enough to avoid overlapping with the entropy profile box, whose height and width are set to legend.width. Is it possible that this shift variable is pushing down the root of the whole tree in the htmlwidget version? Or is there another reason that the tree is not pushed to the top of the svg?

Second, there is a slight "jump", or non-smooth transition whenever I change k in the htmlwidget. Do you see this? And is there a way to remove it so that transitions start smoothly from the current summary tree to the next?

I'll be offline for about a week starting in a couple days, but I'll be ready for more after that. Keep me posted with Q's, comments, or if you want me to try some stuff. I'm really pumped you're taking an interest in this!

[timelyportfolio]

I'll try to make the transition smoother, work on the placement of the elements, and also add the sizing option. Hope to have it by the end of the weekend.

Thanks for making the changes, so the vignettes work.

Kent

On Sat, Aug 15, 2015 at 2:02 PM, Kenny Shirley notifications@github.com wrote:

Re: #1 https://github.com/kshirley/summarytrees/issues/1 and #2 https://github.com/kshirley/summarytrees/issues/2 - nice. Looks good!

You're right about #3, I think the main challenge is going to be figuring out the sizing.

Adding zoom/pan would be cool -- kind of like a leaflet map visualization?

I hadn't actually looked at the viewBox code that you added before -- I just took a look at it this iteration, and since I'm not familiar with that attribute of an svg element, I'm not really sure what it's doing. Does it basically just re-scale all svg elements to a new scale? I'm not doubting that it's a feature! I'll just have to learn more about it :)

Going forward - when I use the htmlwidget version, I still see a couple funny things.

First, the tree's root is typically very low on the y-axis, and the entropy profile (the light gray box with the curve running from lower-left to top-right) is also placed low, rather than directly below the top div that contains the input elements for k. One possible explanation for this: did you see the shift variable that I'm using in line 507 of summarytrees_htmlwidget.js? I use it to measure the height of the tree, and then shift everything down. The details are a bit complicated and ad hoc. The goal is to move the root of the tree down far enough to avoid overlapping with the entropy profile box, whose height and width are set to legend.width. Is it possible that this shift variable is pushing down the root of the whole tree in the htmlwidget version? Or is there another reason that the tree is not pushed to the top of the svg?

Second, there is a slight "jump", or non-smooth transition whenever I change k in the htmlwidget. Do you see this? And is there a way to remove it so that transitions start smoothly from the current summary tree to the next?

I'll be offline for about a week starting in a couple days, but I'll be ready for more after that. Keep me posted with Q's, comments, or if you want me to try some stuff. I'm really pumped you're taking an interest in this!

— Reply to this email directly or view it on GitHub https://github.com/kshirley/summarytrees/issues/1#issuecomment-131425567 .

[kshirley]

Cool! Yeah, those vignettes were a little sloppy - my bad.

Let me know if you have any Q's,

k

[timelyportfolio]

@kshirley sorry this has taken forever, but I would like to try to get an htmlwidget implementation at least to a sort-of releaseable state. I revisited, and one maybe easy thing to do is transition the viewBox change. Here is a live example http://bl.ocks.org/timelyportfolio/6725ab949bab58c2f9f9.

I would still very much like to implement pan/zoom, so I'll play with that also.

[kshirley]

Hey, no worries! (I haven't done any updates in a few months either). The viewBox transition solution looks nice -- it's not choppy anymore. I'll play with it a bit now, too. Eventually I'd like to anchor the "entropy profile" in the upper left, as well as the root of the tree, so that when you increase k, it "grows" from left to right. I find it visually easier to follow if the tree itself isn't centered, so that the levels of the tree remain at fixed horizontal locations. Anyway... I'll get back into this, too. Keep me posted, and I'll probably get back to you with some Q's soon.

[timelyportfolio]

Great to be working on this again. I made a change to add pan and zoom which even more stresses the need for an anchored entropy legend. Hopefully, I can get that up and going by the end of the day.

[timelyportfolio]

@kshirley, here is an iteration with the legend separated http://bl.ocks.org/timelyportfolio/f8c2158b7416bb827926 . Still far from perfect but I think an improvement.

[kshirley]

Whoa - this is really nice! The entropy profile fixed in place is great. Then I realized you implemented zoom and pan - so even though the root isn't fixed on the left edge, I can drag it over there as I wish (avoiding overlap with the entropy profile), and when I've increased "k" so that the nodes (and labels) become small, I can zoom to make them whatever size I want (in my original version, I set the node rectangle's height as a tuning parameter, and used defaults like 12 or 14 pixels). The zooming is

One small thing: can we increase the responsiveness of the panning? Right now I have to click and hold, then drag my mouse most of the way across the touchpad just to move the tree part way across the screen.

Also, can we make the height and width of the view pane an argument for the user to enter? I still envision setting the height and width to very large values, like 2000 or 3000 pixels, so that I can open the viz in my browser (not embedded in a document) and really pan around to explore a summary tree with a few hundred nodes. Also, for some trees that are very tall, the 100-node summary tree might only have a depth of 2 or 3 levels, but each level will extend very far vertically, so we might want to let the user specify that the viz have a height of, say, 2000 pixels.

Last, I still like the having the root anchored-in-place as an option, so that when you change "k", the tree shrinks and grows from a fixed origin. Would this be possible to incorporate?

[timelyportfolio]

Thanks for the feedback. It is perfect. I think I can solve the panning problem by eliminating my viewBox "solution" and instead scaling/zoom to fit the tree to the container.

I'll play some with the best way to make really big height and width. I think it might just work with the height and width arguments, but I just realized that these are not documented. I'll add them to the roxygen and test.

Rooting to the left will be a little more difficult, but I agree would be very nice. I'll try to work on it.

[timelyportfolio]

Ok, I found a bug in my zoom code. This should be a much more pleasant experience http://bl.ocks.org/timelyportfolio/5396a974db3c6ad9c26c. I also added the height and width parameters to documentation.