use htmlwidgets ?

[jeroen]

Currently we simply call the rstudio viewer function to preview an image. Perhaps we can instead wrap it into an html widget so that the same figure will automatically be inserted into the document by knitr or shiny?

Not sure if this makes any sense at all, hopefully @timelyportfolio or @jcheng5 can chime in.

[jeroen]

Perhaps we can combine with https://github.com/ropensci/magick/issues/19 to make an htmlwidget based on jcrop.

[vbonhomme]

I would loooove having magick running along with jcrop and within shiny app and I'll start using it the day it's available. Not sure how I can help but I'm really willing to if I can !

[timelyportfolio]

@vbonhomme would you want it to be like a gadget, so you select coordinates with jcrop, and then those coordinates get passed to R variable which can be used with magick? Will you be clipping images from file or images from R graphics?

[vbonhomme]

I have been interested in combining shiny, imagemagick and a fast viewer for years now.

My field of research, morphometrics suffe from a very (pain in the ass) time-consuming task : digitization of shapes. Consecutively, the horizon of big data is far, and part of the problem is having a powerful and dedicated tool, within R.

The minimal primer I'm looking for combines :

1. a GUI/wrapper to (image)magick + some of Momocs. shiny would be ideal, or anything very close to R (where we would like to stay for the entire acquisition-> morphometrics -> statistics pipe)
2. a fast viewer (jcrop or Rstudio viewer are nice) for say .jpg images on which one can do many magick things but also place and edit landmarks, splines, outlines, etc.

Do you think that's feasible? Where would you start? I'd love having your view on it.

V

[timelyportfolio]

I am fascinated, but until today did not even know there was a word "morphometrics". Besides crop, what functions from magick would be most applicable? I guess I should start by reading the textbook :)

[vbonhomme]

I discovered the word like 10 yr ago and I'm still in love with it ;-)

I'm first searching for a general process to combine 1+2 above, and firstly by having a fast viewer embedded in a shiny app. This part may just be a trick to embed a viewer in shiny ui (thus in html).

Then I think I can develop a wrapper for segmentation and pretty much everything you have ported in magick. I'm quite serene on my skills for this.

But after that I would absolutely need drawing features : to place landmarks, draw splines and bezier curves, etc. A general mechanism to retrieve coordinates from the viewer, draw stuff back from them, etc. This part is very exotic to me, not proficient in Cpp, js, all that magical stuff to me.

What I would like to have, for me but also for the community, is an open-source alternative to tpsdig. Within R, there is StereoMorph by my friend Aaron but I'm looking for something simpler and on which we could have full control with lot of R. I created myself a primer as an ImageJ plugin Momacs but again, having the entire pipe "data acquisition -> analysis" open-source, and in R, and convenient is very important.

The textbook is easy to find and I can also send it to you !

[vbonhomme]

ping @timelyportfolio: any pointer in your mind to implement this? thanks a bunch.

[timelyportfolio]

Still getting a feel for the domain and having a blast with @aaronolsen repos. I am starting to envision something much greater than a cropper helper, and in this case something like fabric.js could prove very useful. Thoughts? Perhaps, we should start a repo or org, so we don't clutter up magick.

[vbonhomme]

Very excited and if we success in having such an editor that we can call from R/Rstudio that would be very popular in the field you can trust me ;-)

[aaronolsen]

I'm currently working on pulling shiny from StereoMorph's dependencies and replacing it with the more basic httpuv package (on which shiny is dependent). I only used shiny to easily create the local server port that allows communication between R and the web browser - the interactive interface itself I designed from scratch. So if I could get the server running with just httpuv that would be preferable. I'd ultimately like to expand the image analysis tools available through the StereoMorph app but haven't had the time. The tools would be primarily directed at collecting shape data from images in pixels and scaled coordinates.

[timelyportfolio]

@aaronolsen, have you looked at jug or fiery?

[timelyportfolio]

@vbonhomme, @aaronolsen where should we put such a thing -- Github org or repo?

[vbonhomme]

if @aaronolsen is ok, perhaps there as long as the "beast" is built on StereoMorph? Or we can start an org for open source, chainable, R-based morphometrics ?

[aaronolsen]

Oh wow - yes either jug or fiery looks like a good solution for what I wanted to do. Thanks! I have no experience collaborating on github so I'm not good a good person to ask about this. I certainly don't mind having these tools available through StereoMorph and adding any contributors whose branches are merged as package authors. I'm just not sure how easy it will be for people to contribute in a way that doesn't break the existing StereoMorph operations. Whatever you all think is best since you all are more experienced with this.

There are a lot of ideas I have in the pipeline for the StereoMorph digitizing app. We're using it to digitize video frames for a couple different studies but I'd like it to handle consecutive image loads faster. And interpolate and predict undigitized shapes. Also, I'd like to try modifying it to work with CT slice data. We'll see if I actually have the time for any of this! : )

[timelyportfolio]

@vbonhomme and @aaronolsen, as I work more with leaflet and Leaflet.Draw, I wonder if some of the shape workflows can fit within these tools.

# devtools::install_github("rstudio/leaflet")
# devtools::install_github("bhaskarvk/leaflet.extras")

library(shapes)
library(leaflet)
library(leaflet.extras)

data(apes)

oneape <- apes$x[,,1] %>%
  .[,c(2,1)]

leaflet(
  data=oneape,
  leafletOptions(
    crs = leafletCRS(crsClass="L.CRS.Simple")
  ),
  width = 400,
  height = 400,
  padding = 0
) %>%
  fitBounds(-50,-50,200,200) %>%
  addCircleMarkers(group="landmarks") %>%
  addDrawToolbar(targetGroup="landmarks", editOptions=editToolbarOptions())

This workflow could easily be wrapped in a Shiny gadget to return the edited/added landmarks.

[vbonhomme]

hi there ! well, we really need the raster image behind and I'm not sure leaflet is really appropriate here don't you think?

2017-01-10 19:00 GMT+01:00 timelyportfolio notifications@github.com:

@vbonhomme https://github.com/vbonhomme and @aaronolsen https://github.com/aaronolsen, as I work more with leaflet and Leaflet.Draw, I wonder if some of the shape workflows can fit within these tools.

devtools::install_github("rstudio/leaflet")

devtools::install_github("bhaskarvk/leaflet.extras")

library(shapes) library(leaflet) library(leaflet.extras)

data(apes)

oneape <- apes$x[,,1] %>% .[,c(2,1)]

leaflet( data=oneape, leafletOptions( crs = leafletCRS(crsClass="L.CRS.Simple") ), width = 400, height = 400, padding = 0 ) %>% fitBounds(-50,-50,200,200) %>% addCircleMarkers(group="landmarks") %>% addDrawToolbar(targetGroup="landmarks", editOptions=editToolbarOptions())

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ropensci/magick/issues/16#issuecomment-271649218, or mute the thread https://github.com/notifications/unsubscribe-auth/AEgoFsHmv-JuKJ-UFEcL8XiyL9B_Tqthks5rQ8cjgaJpZM4JrZ8b .

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

[timelyportfolio]

Agree that leaflet might not be the best tool, but it is capable of plotting a raster image layer behind. I'll try to demo that as well. Have a lot to learn about shapes/morphometrics.

[vbonhomme]

otherwise EBImage has a nice and fast viewer : https://github.com/aoles/EBImage/blob/master/inst/viewer/viewer.js

   #source("http://bioconductor.org/biocLite.R")
   #biocLite("EBImage")
   library(EBImage)
   lena <- readImage(system.file("images", "lena-color.png", package="EBImage"))
  display(lena)

we would "just" need to add basic editing capabilities (points, polylines, splines, bezier curves) and i/o from the console (like exporting groups of coordinates previously set, etc.)

all the best,

v

[vbonhomme]

Hi @timelyportfolio,

Still thinking and struggling about it. I would like to add a raster in the background, then draw markers and retrieve them in xy coordinates. I had lot of troubles (ie > 6hrs), trying to figure out how to do it in pixels and not in lat long. I have tried many things around leafletCRS(crsClass="L.CRS.Simple") and fitBounds() but, besides headaches, nothing satisfying. Any idea on how to do it, eg combine image display and xy coordinates of landmarks on it? Cheers from southern France.

v

[timelyportfolio]

Hi @vbonhomme, thanks for the ping, I still think this would be incredibly nice for R, but I have unfortunately not been able to work on it. In terms of httpuv, @kcf-jackson has done some experiments recently at https://github.com/kcf-jackson/jsReact, so these might be helfpul.

I will see if there is a way to retrieve xy instead of latlong. I think there is a way. However, I also know that the leaflet solution is probably not best. I saw this recently https://rmfisher.github.io/react-measurements/ and was intrigued.

@aaronolsen, I am in awe of your work.

[jeroen]

I've started working on some experimental widgets in https://github.com/jeroen/imwidgets

[timelyportfolio]

@jeroen, how exciting! Please let me know how I can help.

[jeroen]

One thing I can't figure out of there is a way to use local image paths in the html widget (other than serializing them to base64png). Currently this works:

imwidgets::cropper("https://jeroen.github.io/images/frink.png")

But I would like to show a local image obviously. Shiny has renderImage and rstudio has the local http server, but is there a htmlwidgets system for this?

[timelyportfolio]

There is not a htmlwidgets system that I remember, but @tim-salabim3 was able to make it work with mapview in popup.R. While this works when viewing locally, it does not pair well with saveWidget. I am wondering if we can build on top of htmlDependency for this (see lines). The RStudio part to render html in their Viewer and creates a temporary directory for the content is in these lines.

I think there is a clue here.

I'll keep digging on this, since we now need for imwidgets and mapview.

Should we start a new issue at imwidgets?

[kcf-jackson]

Hi @vbonhomme and @timelyportfolio, not sure if I understand the problem correctly; the following code may help regarding having a raster image background and retrieving mouse-click coordinates. Remove the quick hack comment to make the data persistent when you stop the app. I hope the code is self-explanatory.

# rm(list = ls())
# devtools::install_github("kcf-jackson/jsReact")
library(magrittr)
library(jsReact)

my_html <- create_html() %>% 
  add_js_library('p5') %>% 
  add_script(
    "
    function preload() {
      var src = 'https://cdn.rawgit.com/rmfisher/react-measurements/863d8563/src/demo/images/pollen.jpg?raw=true';
      img = loadImage(src);
    }
    function setup() {
      createCanvas(600, 600);
      image(img, 0, 0);
    }
    function mouseClicked() {
      fill(255);
      ellipse(mouseX, mouseY, 10, 10);
      ws.send(JSON.stringify({x: mouseX, y: mouseY}))
    }
    "
  )

# empty_df <- c()  # quick hack
my_r_fun <- function(msg) {
  cat(msg$x, msg$y, "\n") 
  # empty_df <<- rbind(empty_df, c(msg$x, msg$y))  # quick hack
}

preview_app(my_html, my_r_fun, T)

Remarks

• Change mouseClicked() to mouseDragged(), then you can 'draw' instead of 'click'.
• Note that Rstudio viewer often has much lower framerate than the browser, so use 'show in new window' (browser) if you experience any lag.
• With little modification, one can wrap the whole thing into a function call, which takes an image link as an argument and outputs a data.frame of mouse-click coordinates.

[kcf-jackson]

@jeroen To serve local images, (you ~probably~ definitely already know) you need to have your image and html serving from the same directory. (I didn't know I was talking to the author of V8 and friends!)

In your htmlwidget case, I haven't tried it but I think the trick is to find where the html will live at the time of rendering. This can be done by passing a function to the preRenderHook argument; the function looks something like this: preRenderHook = function(widget) {print(getwd())}. This on my computer gives me:

"/private/var/folders/96/m5m4mk2d6bn0z5hzwfr7t_4c0000gn/T/RtmprSGEfQ/viewhtml971c265a8ae1"

What's left is to bind your image file to the function, which may look like:

bind_img <- function(local_full_img_path) {
  function(widget) {
    server_dir <- getwd()
    file.copy(local_full_img_path, server_dir)
    # you may also need to update the widget$x$url param to 'server_dir/your_img_name'
  }
}  

Then, the preRenderHook argument becomes preRenderHook = bind_img(your_img).

Hope it works.

[vbonhomme]

Hi @timelyportfolio, @jeroen, @timelyportfolio, @kcf-jackson Thanks for all these updates and good news. All of this is amazing. I got the feeling that I'd be able to manage myself this app of dreams, providing we use Shiny and leaflet (and not proficient at all in js and leaflet has a good navigation interface yet it's a bit hacked for our purpose). Except you really dont think it's a good idea, that's the path I'll try. The only things missing now is: i) how to project properly an image in leaflet with xy (pixels) coordinates, and ii) how to retrieve/edit/display (if previous have been digitized) features (polylines, markers, etc.) on the spirit of what @timelyportfolio have proposed on SO here. And thanks for your response !

@timelyportfolio (and others): would you be keen to follow my advancements somewhere else ? perhaps on a dedicated repo? I have poor time (Father again <3!) in the days, but some in the nights... and from January I'll have plenty.

[vbonhomme]

@timelyportfolio and others. I wasn't aware of the progress made around mapedit! Quite amazing. I guess, I'll work around this. O fcourse, I still maintain my offer around helping me develop a soft for morphometrics if you're interesting in helping a poor lost frog !

[jeroen]

Merging this with https://github.com/ropensci/magick/issues/19