Wordcloud example doesn't work

bglgithub17 commented 6 years ago

The example data in the Wordcloud tool give me the error shown in the screenshot below. I'm running RStudio Server in CentOS and running the MetaBox app in Chrome. More details of my environment and R session are below the screenshot. Any help would be appreciated.

R version 3.4.3 (2017-11-30) Platform: x86_64-redhat-linux-gnu (64-bit) Running under: CentOS Linux 7 (Core)

Matrix products: default BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] opencpu_2.0.5 metabox_1.2 ggplot2_3.0.0 devtools_1.13.6

loaded via a namespace (and not attached): [1] Rcpp_0.12.16 pillar_1.2.1 compiler_3.4.3 git2r_0.23.0 plyr_1.8.4 bindr_0.1.1 sys_1.6
[8] tools_3.4.3 digest_0.6.15 lattice_0.20-35 jsonlite_1.5 evaluate_0.10.1 memoise_1.1.0 tibble_1.4.2
[15] gtable_0.2.0 pkgconfig_2.0.1 rlang_0.2.1 parallel_3.4.3 curl_3.2 yaml_2.1.18 bindrcpp_0.2.2
[22] withr_2.1.2 httr_1.3.1 stringr_1.3.1 dplyr_0.7.6 knitr_1.20 rappdirs_0.3.1 rprojroot_1.3-2 [29] grid_3.4.3 tidyselect_0.2.4 glue_1.3.0 R6_2.2.2 rmarkdown_1.10 purrr_0.2.5 magrittr_1.5
[36] webutils_0.6 backports_1.1.2 scales_0.5.0 htmltools_0.3.6 assertthat_0.2.0 mime_0.5 colorspace_1.3-2 [43] httpuv_1.3.6.2 labeling_0.3 stringi_1.2.4 openssl_1.0.1 lazyeval_0.2.1 munsell_0.4.3

kwanjeeraw commented 6 years ago

Hi, It's because we don't have pathway database running anymore. There need to be local database installation on your machine. The function still works for Mesh annotation though.

bglgithub17 commented 6 years ago

Thanks, but I'm still having problems. First, when I try the Mesh option, I get the same error: Second, I'm confused about installing a local database. What database should be installed locally, and where can I obtain this database? The only mention of a local database I see in the Metabox documentation is Neo4j, which it says is used for biological network queries and pathway enrichment analysis. So I did the following on a CentOS server: 1) Installed Neo4j community server edition following the instructions at http://yum.neo4j.org/stable/, using "sudo" when necessary. 2) Started the neo4j server: sudo neo4j start 3) Opened my browser in an Xming window and went to http://localhost:7474/browser/ 4) I didn't see any way to set the initial neo4j password from the web app as the documentation says, so I set it using the neo4j-admin command: sudo neo4j-admin set-initial-password grinn
5) When I refreshed my browser and entered my password, I saw the following screen, which seemed to suggest that Neo4j is running successfully: 6) Here is where I'm really confused. The Metabox documentation says in Linux to create a file called "database.R" and only that it should be in HOME/USER_NAME/SUBFOLDER_NAME/, but doesn't say how Metabox or Neo4j will find the "SUBFOLDER_NAME" directory. I put database.R in my $HOME directory and am starting Metabox by running ocpu_start_app("metabox") in RStudio with my working directory set to the same directory. The documentation also says put the following line in it: database.location = "/path/to/database/". I'm confused what database this refers to. I tried database.location = "/var/lib/neo4j/data/databases/graph.db/" and database.location = "http://localhost:7474/var/lib/neo4j/data/databases/graph.db/", but in both cases I see the following screen in Metabox:

I would greatly appreciate it if you could help me get back on track. Thanks. --Ben.

kwanjeeraw commented 6 years ago

I have no error on my system, don't really know how to fix it. Here is my system info. R version 3.3.1 (2016-06-21) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows >= 8 x64 (build 9200)

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 [4] LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] opencpu_2.0.5 metabox_1.2

loaded via a namespace (and not attached): [1] Rcpp_0.12.17 lattice_0.20-35 mime_0.5 rappdirs_0.3.1 plyr_1.8.4 grid_3.3.1 sys_1.5
[8] jsonlite_1.5 gtable_0.2.0 magrittr_1.5 scales_0.5.0 evaluate_0.10.1 pillar_1.2.3 ggplot2_2.2.1
[15] rlang_0.2.0 stringi_1.2.2 curl_3.2 lazyeval_0.2.1 webutils_0.6 tools_3.3.1 stringr_1.3.1
[22] munsell_0.4.3 parallel_3.3.1 colorspace_1.3-2 openssl_1.0.1 tibble_1.4.2

The database can be downloaded from here https://sourceforge.net/projects/grinn/files/human_v6.zip/download Unzip and put into a folder. When you start Neo4j server, select this human_v6.

I put this in database.R database.location = "http://localhost:7474/db/data/"

After finish installation, Neo4j should be like this.

And the metabox will show neo4j location.

bglgithub17 commented 6 years ago

Thanks for your patience. Before, I hadn't read anywhere in the Metabox documentation where it said to download the human_v6 database or where it could be downloaded from, so thanks for pointing that out.
You said when you start the Neo4j server, select the human_v6 database. How do you do this? I start/restart the Neo4j server using the command: sudo neo4j start (or restart). But I don't see a way to select the human_v6 directory. I searched around and found that the neo4j-admin import command. I ran this: sudo neo4j-admin import --mode=database --database=human_v6 --from=human_v6. This copied the human_v6 directory to the /var/lib/neo4j/data/databases directory, which is where graph.db already was, so now I have both in that directory: ls /var/lib/neo4j/data/databases/ graph.db human_v6 I then restarted neo4j, but this didn't change anything. In your Neo4j browser, in the Database tab, where yours says "human_v6", mine says "graph.db". Why is this? Thanks again.

bglgithub17 commented 6 years ago

Update: I'm getting closer. I got human_v6 to load in the Neo4j browser by doing the following:

Stopped Neo4j: sudo neo4j stop
Uncomment and edit line 9 in /etc/neo4j/neo4j.conf: dbms.active_database=human_v6
Restart Neo4j: sudo neo4j start The Neo4j browser now shows that human_v6 is loaded as the database However, I don't see the information that you are seeing; there's nothing in the "Node labels" or "Relationship types" sections. I also see that my database is about 112 Kb and yours is over 1 Gb. Any idea why? Metabox is at least recognizing that I have a local database installed; it shows the message that my local database installation is http://localhost:7474/db/data/. However, the WordCloud tool is still giving me the same popup error message when I try the example data, whether I choose the Pathway or Mesh options.

kwanjeeraw commented 6 years ago

The database was created under Neo4j 3.0. So it has to be upgraded by uncommenting dbms.allow_upgrade=true

I just tested it with current Neo4j version 3.4.5. Luckily it is working fine.

Trying to check the error message, but couldn't find any bug. I guess it might be weird return from the query.

bglgithub17 commented 6 years ago

And it's working! The Metabox documentation had confused me. I'm experienced in Linux and bioinformatics/computing but not with databases and their terminology. Below, here is what made sense to me and what was confusing. Maybe this will help you to improve the documentation. I don't mean to criticize your documentation, just offering my feedback in case you find it helpful.

Install Neo4j. This was clear to me, although the Metabox documentation doesn't say whether to install the server or desktop edition, or community or enterprise. Hopefully most users would know which one makes sense for their system.
Download the precompiled database. This confused me as someone with very little experience in databases. The Metabox documentation doesn't say what kind of database or where to find it. So I assumed it was just talking about the actual Neo4j database. Neo4j is referred to as a database throughout the web and its website uses the phrase "Neo4j databases". The Metabox documentation also says to move this database to "your Neo4j directory", but it doesn't say where that directory is. It's /var/lib/neo4j/data/databases/, which was difficult to find.
Start the Neo4j server. This was clear, but there were steps I had to do that were not mentioned in the documentation. First, edit the configuration file /etc/neo4j/neo4j.conf. The changes I had to make were dbms.allow_upgrade=true and dbms.active_database=human_v6. I only figured out the second requirement after looking in the log file /var/log/neo4j/neo4j.log. Second, I had to set the initial admin password before starting the Neo4j server, rather than after it, as the Metabox documentation says. I set the password using sudo neo4j-admin set-initial-password grinn. Then started Neo4j using sudo neo4j start
The database.R file doesn't appear to be necessary if you're using http://localhost:7474/db/data/ as the database location. After following these steps, Metabox is working. The only thing that didn't work for me is installing the dependency WGCNA in R/RStudio in CentOS. It worked fine in Windows 8, however. So I think most of the functionality in Metabox should work for me except anything that relies on WGCNA. I'm getting a segmentation fault when R tries to test whether the package can be loaded:

** testing if installed package can be loaded
sh: line 1: 31816 Segmentation fault      (core dumped) '/usr/lib64/R/bin/R' --no-save --slave 2>&1 < '/tmp/RtmpoioHFf/file7af82b85dbe2'
==========================================================================
*
*  Package WGCNA 1.63 loaded.
*
*    Important note: It appears that your system supports multi-threading,
*    but it is not enabled within WGCNA in R. 
*    To allow multi-threading within WGCNA with all available cores, use 
*
*          allowWGCNAThreads()
*
*    within R. Use disableWGCNAThreads() to disable threading if necessary.
*    Alternatively, set the following environment variable on your system:
*
*          ALLOW_WGCNA_THREADS=<number_of_processors>
*
*    for example 
*
*          ALLOW_WGCNA_THREADS=48
*
*    To set the environment variable in linux bash shell, type 
*
*           export ALLOW_WGCNA_THREADS=48
*
*     before running R. Other operating systems or shells will
*     have a similar command to achieve the same aim.
*
==========================================================================

 *** caught segfault ***
address (nil), cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...
ERROR: loading failed
* removing ‘/home/bernest/R/x86_64-redhat-linux-gnu-library/3.4/WGCNA’

 *** caught segfault ***
address (nil), cause 'memory not mapped'

Traceback:
 1: q("no", status = status, runLast = FALSE)
 2: do_exit(status = status)
 3: do_exit_on_error()
 4: errmsg("loading failed")
 5: do_install_source(pkg_name, instdir, pkg, desc)
 6: do_install(pkg)
 7: tools:::.install_packages()
An irrecoverable exception occurred. R is aborting now ...
/usr/lib64/R/bin/INSTALL: line 34: 31479 Done                    echo 'tools:::.install_packages()'
     31480 Segmentation fault      (core dumped) | R_DEFAULT_PACKAGES= LC_COLLATE=C "${R_HOME}/bin/R" $myArgs --slave --args ${args}

The downloaded source packages are in
    '/tmp/RtmprfAT2e/downloaded_packages'
installation path not writeable, unable to update packages: bit, data.table, DBI, evaluate, fastcluster, foreign, Formula,
  futile.options, ggplot2, glue, highr, htmlTable, htmlwidgets, httpuv, iterators, lambda.r, MALDIquant, MASS, matrixStats,
  mgcv, munsell, mvtnorm, pillar, RANN, Rcpp, RcppArmadillo, RCurl, rlang, robustbase, rrcov, RSQLite, shiny, sourcetools,
  stringi, stringr, survival, utf8, yaml
Old packages: 'igraph', 'maptools', 'opencpu', 'openssl', 'tm', 'XML'
Update all/some/none? [a/s/n]: 
n
Warning message:
In install.packages(pkgs = doing, lib = lib, ...) :
  installation of package 'WGCNA' had non-zero exit status

However, this is not a problem with Metabox; it's either with my R or Linux configuration or the WGCNA package. Thanks again for your help.

kwanjeeraw commented 6 years ago

Many thanks for your feedback. This should be useful for linux and mac. The database was implemented on neo4j 3.0 and it is available upon request. There are quite a change in the current neo4j. I'll find time to check and update the manual. WGCNA in metabox is used for computing correlation network, in case you need. Good luck :)

bglgithub17 commented 6 years ago

Just an update on WGCNA for anyone interested. I did a full upgrade of my R version to R 3.5, and a full upgrade of Bioconductor with biocLite("BiocUpgrade") and all packages with biocLite(). WGCNA then installed without problems.

kwanjeeraw / metabox

Wordcloud example doesn't work #7