search

DataSlingers / clustRviz

Compute Convex (Bi)Clustering Solutions via Algorithmic Regularization
https://DataSlingers.github.io/clustRviz/
GNU General Public License v3.0
19 stars 14 forks source link

No visible plots for interactive CARP plots #56

Open alexpghayes opened 6 years ago

alexpghayes commented 6 years ago

I tried to play with a carp fit on the first 100 points from the data in #54, but the Shiny app wasn't displaying any graphics, just the regularization slider.

alexpghayes commented 6 years ago

I also gave up on type = "dynamic_path" after 2 or so minutes of rendering.

michaelweylandt commented 6 years ago

Will check out the lack of graphics issue tomorrow. Possibly a slowness issue, but my default test data has n = 44 so I can't imagine there being too big a difference.

(I have long term goal of removing shiny and doing it all in D3, but I'm not knowledgeable enough about D3 to do it right now)

Re: type = "dynamic_path" -- can you send session info? (Either the devtools version or the base R one) I've seen that hang when the wrong version of gganimate was being invoked and/or gifski was not installed properly, but want to confirm.

alexpghayes commented 6 years ago 
devtools::session_info()
Session info --------------------------------------------------------------------------
 setting  value                       
 version  R version 3.5.0 (2018-04-23)
 system   x86_64, mingw32             
 ui       RStudio (1.2.999)           
 language (EN)                        
 collate  English_United States.1252  
 tz       America/Chicago             
 date     2018-10-11                  

Packages ------------------------------------------------------------------------------
 package      * version    date       source                                 
 animation      2.5        2017-03-30 CRAN (R 3.5.1)                         
 assertthat     0.2.0      2017-04-11 CRAN (R 3.5.0)                         
 backports      1.1.2      2017-12-13 CRAN (R 3.5.0)                         
 base         * 3.5.0      2018-04-23 local                                  
 bindr          0.1.1      2018-03-13 CRAN (R 3.5.0)                         
 bindrcpp       0.2.2      2018-03-29 CRAN (R 3.5.0)                         
 class          7.3-14     2015-08-30 CRAN (R 3.5.0)                         
 cluster        2.0.7-1    2018-04-13 CRAN (R 3.5.0)                         
 clustRviz    * 0.1.0      2018-10-07 Github (DataSlingers/clustRviz@39dc90a)
 colorspace     1.3-2      2016-12-14 CRAN (R 3.5.0)                         
 compiler       3.5.0      2018-04-23 local                                  
 crayon         1.3.4      2017-09-16 CRAN (R 3.5.0)                         
 datasets     * 3.5.0      2018-04-23 local                                  
 dendextend     1.8.0      2018-05-03 CRAN (R 3.5.0)                         
 DEoptimR       1.0-8      2016-11-19 CRAN (R 3.5.0)                         
 devtools       1.13.6     2018-06-27 CRAN (R 3.5.0)                         
 digest         0.6.17     2018-09-12 CRAN (R 3.5.1)                         
 diptest        0.75-7     2016-12-05 CRAN (R 3.5.0)                         
 dplyr          0.7.6      2018-06-29 CRAN (R 3.5.1)                         
 evaluate       0.11       2018-07-17 CRAN (R 3.5.1)                         
 farver         1.0.0.9999 2018-09-27 Github (thomasp85/farver@5439336)      
 flexmix        2.3-14     2017-04-28 CRAN (R 3.5.0)                         
 fpc            2.1-11.1   2018-07-20 CRAN (R 3.5.1)                         
 gganimate      0.9.9.9999 2018-09-27 Github (thomasp85/gganimate@b1dce00)   
 ggplot2        3.0.0      2018-07-03 CRAN (R 3.5.1)                         
 ggrepel        0.8.0      2018-05-09 CRAN (R 3.5.1)                         
 gifski         0.8.3      2018-07-20 CRAN (R 3.5.1)                         
 glue           1.3.0      2018-08-26 Github (tidyverse/glue@4e74901)        
 graphics     * 3.5.0      2018-04-23 local                                  
 grDevices    * 3.5.0      2018-04-23 local                                  
 grid           3.5.0      2018-04-23 local                                  
 gridExtra      2.3        2017-09-09 CRAN (R 3.5.0)                         
 gtable         0.2.0      2016-02-26 CRAN (R 3.5.0)                         
 gtools         3.8.1      2018-06-26 CRAN (R 3.5.0)                         
 hms            0.4.2      2018-03-10 CRAN (R 3.5.0)                         
 htmltools      0.3.6      2017-04-28 CRAN (R 3.5.0)                         
 httpuv         1.4.5      2018-07-19 CRAN (R 3.5.1)                         
 kernlab        0.9-27     2018-08-10 CRAN (R 3.5.1)                         
 knitr          1.20       2018-02-20 CRAN (R 3.5.0)                         
 later          0.7.5      2018-09-18 CRAN (R 3.5.1)                         
 lattice        0.20-35    2017-03-25 CRAN (R 3.5.0)                         
 lazyeval       0.2.1      2017-10-29 CRAN (R 3.5.0)                         
 magrittr       1.5        2014-11-22 CRAN (R 3.5.0)                         
 MASS           7.3-49     2018-02-23 CRAN (R 3.5.0)                         
 Matrix         1.2-14     2018-04-13 CRAN (R 3.5.0)                         
 mclust         5.4.1      2018-06-27 CRAN (R 3.5.0)                         
 memoise        1.1.0      2017-04-21 CRAN (R 3.5.0)                         
 methods      * 3.5.0      2018-04-23 local                                  
 mime           0.5        2016-07-07 CRAN (R 3.5.0)                         
 modeltools     0.2-22     2018-07-16 CRAN (R 3.5.1)                         
 munsell        0.5.0      2018-06-12 CRAN (R 3.5.0)                         
 mvtnorm        1.0-8      2018-05-31 CRAN (R 3.5.0)                         
 nnet           7.3-12     2016-02-02 CRAN (R 3.5.0)                         
 packrat        0.4.9-3    2018-06-01 CRAN (R 3.5.0)                         
 pillar         1.3.0      2018-07-14 CRAN (R 3.5.1)                         
 pkgconfig      2.0.2      2018-08-16 CRAN (R 3.5.1)                         
 plyr           1.8.4      2016-06-08 CRAN (R 3.5.0)                         
 png            0.1-7      2013-12-03 CRAN (R 3.5.0)                         
 prabclus       2.2-6      2015-01-14 CRAN (R 3.5.0)                         
 prettyunits    1.0.2      2015-07-13 CRAN (R 3.5.0)                         
 progress       1.2.0      2018-06-27 Github (r-lib/progress@8428528)        
 promises       1.0.1      2018-04-13 CRAN (R 3.5.0)                         
 purrr          0.2.5      2018-05-29 CRAN (R 3.5.0)                         
 R6             2.2.2      2017-06-17 CRAN (R 3.5.0)                         
 RColorBrewer   1.1-2      2014-12-07 CRAN (R 3.5.0)                         
 Rcpp           0.12.19.1  2018-10-01 Github (RcppCore/Rcpp@4680c31)         
 rlang          0.2.2.9001 2018-10-01 Github (r-lib/rlang@cdb713c)           
 rmarkdown      1.10       2018-06-11 CRAN (R 3.5.0)                         
 robustbase     0.93-3     2018-09-21 CRAN (R 3.5.1)                         
 rprojroot      1.3-2      2018-01-03 CRAN (R 3.5.0)                         
 rstudioapi     0.7        2017-09-07 CRAN (R 3.5.0)                         
 scales         1.0.0      2018-08-09 CRAN (R 3.5.1)                         
 shiny          1.1.0      2018-05-17 CRAN (R 3.5.0)                         
 stats        * 3.5.0      2018-04-23 local                                  
 stats4         3.5.0      2018-04-23 local                                  
 stringi        1.1.7      2018-03-12 CRAN (R 3.5.0)                         
 stringr        1.3.1      2018-05-10 CRAN (R 3.5.0)                         
 tibble         1.4.2      2018-01-22 CRAN (R 3.5.0)                         
 tidyr          0.8.1      2018-05-18 CRAN (R 3.5.0)                         
 tidyselect     0.2.4      2018-02-26 CRAN (R 3.5.0)                         
 tools          3.5.0      2018-04-23 local                                  
 trimcluster    0.1-2.1    2018-07-20 CRAN (R 3.5.1)                         
 tweenr         0.1.5.9999 2018-09-27 Github (thomasp85/tweenr@48ea2fd)      
 utils        * 3.5.0      2018-04-23 local                                  
 viridis        0.5.1      2018-03-29 CRAN (R 3.5.0)                         
 viridisLite    0.3.0      2018-02-01 CRAN (R 3.5.0)                         
 whisker        0.3-2      2013-04-28 CRAN (R 3.5.0)                         
 withr          2.1.2      2018-03-15 CRAN (R 3.5.0)                         
 xtable         1.8-3      2018-08-29 CRAN (R 3.5.1)                         
 zoo            1.8-4      2018-09-19 CRAN (R 3.5.1)
alexpghayes commented 6 years ago

I also tried some static plots for the scaled data and there was a ton of overplotting. I imagine the absolute number of points in the paths grows with the number of data points and this slows things down a lot? Maybe there's some heuristic to no use all the points on the paths / not label things when there are 2000 points that will speed stuff up as well.

michaelweylandt commented 6 years ago

So I've looked into this and there's definitely some room for improvement -- we plot the centroid for each point at each iteration, which (as you note) leads to overplotting for fused centroids. Once "U smoothing" (#63) is added to the post-processing, we can add a distinct() filter before plotting which should speed things up majorly.

That said, I think there are always going to be some problems with larger data sets + shiny graphics since ggplot2 (or really grid) is slow (vs base graphics). I've got some long term ideas about moving away from ggplot2 but they aren't likely to happen for a while.

alexpghayes commented 6 years ago

RStudio also just hired someone to make ggplot2 faster, which with any luck will result in noticeable speed improvements across the board.