search

rfhb / ctrdata

Aggregate and analyse information on clinical trials from public registers
https://rfhb.github.io/ctrdata/
Other
41 stars 5 forks source link

Loading to MongoDB fail #13

Closed JoeFuzz83 closed 5 years ago

JoeFuzz83 commented 5 years ago

Hi,

This is my first time using r and mongo so please for give me if there's something I've missed. I get error when importing into mongoDB, object; 'ids' not found.

e.g. ctrLoadQueryIntoDb(queryterm = q)

"Search should return 3 results in this example"

(2/3) Converting to JSON ... (3/3) Importing JSON into mongoDB ... Error in dbCTRLoadJSONFiles(dir = tempDir, mongo = mongo) : object 'ids' not found In addition: Warning message: In system(euctr2json, intern = TRUE) : running command 'cmd.exe /c c:\cygwin\bin\bash.exe

I've taken a look at the dbCTRLoadJSONFiles script to see if I could make any sense if the issue but I'm too novice at this point. https://rdrr.io/cran/ctrdata/src/R/main.R#sym-dbCTRLoadJSONFiles

Kind regards,

Joe

==================================================== I'm running under the following setup;

R version 3.5.3 (2019-03-11) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 17134)

attached base packages: [1] stats graphics grDevices utils datasets methods
[7] base

other attached packages: [1] mongolite_2.0.1 ctrdata_0.18

loaded via a namespace (and not attached): [1] httr_1.4.0 clipr_0.5.0 compiler_3.5.3 magrittr_1.5
[5] R6_2.4.0 tools_3.5.3 curl_3.3 Rcpp_1.0.1
[9] xml2_1.2.0 jsonlite_1.6 rvest_0.3.3 openssl_1.3
[13] askpass_1.1

=== CYGWIN_NT-10.0 DESKTOP-12J62K2 3.0.6(0.338/5/3) 2019-04-06 16:18 x86_64 mongodb-win32-x86_64-2008plus-ssl-4.0.8-signed

rfhb commented 5 years ago

Thanks - would you mind to post the queryterm, please? It will help to quicker find the issue.

JoeFuzz83 commented 5 years ago

This is the initial test run I used. where q <- ctrGetQueryUrlFromBrowser() = https://www.clinicaltrialsregister.eu/ctr-search/search?query=&country=ie&age=newborn&status=completed&phase=phase-two

ctrLoadQueryIntoDb(queryterm = q)

And I've also tried a few passes based on examples given with the same outcome.

ctrLoadQueryIntoDb(queryterm = "https://www.clinicaltrialsregister.eu/ctr-search/search?query=&country=ie&age=newborn&status=completed&phase=phase-two", register = "EUCTR",querytoupdate = 0L, forcetoupdate = FALSE, euctrresults = FALSE,annotation.text = "", annotation.mode = "append", details = TRUE,parallelretrievals = 10, debug = FALSE, collection = "ctrdata",uri = "mongodb://localhost/users",password = Sys.getenv("ctrdatamongopassword"), verbose = FALSE)

ctrLoadQueryIntoDb(queryterm = "2010-024264-18",

Thanks!

J

rfhb commented 5 years ago

A working cygwin installation in c:\cygwin is missing.

When you load package ctrdata (in R, call library(ctrdata), does it report a missing or non-working cygwin installation?

cygwin can be installed as per the README (in R, call ctrdata::installCygwinWindowsDoInstall()).

I will improve how this is checked and reported as error when functions are used without loading the package first.

JoeFuzz83 commented 5 years ago

I have C:\cygwin64\ rather than C:\cygwin\ in my directory. I'll do the manual install and see how that goes. Thanks, J

JoeFuzz83 commented 5 years ago

And just to note I didn't get an error message when I callws library(ctrdata).

rfhb commented 5 years ago
JoeFuzz83 commented 5 years ago

Hi Ralf,

I reinstalled ctrdata with no issues and manually installed cygwin per README (dir c:\cygwin this time) but unfortunately I sill have the same issue. Please see below.

q <- ctrGetQueryUrlFromBrowser()

" https://www.clinicaltrialsregister.eu/ctr-search/search?query=Cancer&country=ie&age=adult&status=completed&phase=phase-four "

ctrLoadQueryIntoDb(queryterm = q)

install.packages("ctrdata") Installing package into ‘C:/Users/Joe McCafferty/Documents/R/win-library/3.5’ (as ‘lib’ is unspecified) installing the source package ‘ctrdata’

trying URL 'https://cran.rstudio.com/src/contrib/ctrdata_0.18.tar.gz' Content type 'application/x-gzip' length 1320571 bytes (1.3 MB) downloaded 1.3 MB

Call library(ctrdata) returned no issues.

library(ctrdata) doesn't throw up any issues

JoeFuzz83 commented 5 years ago

I've installed the development version and ran > ctrLoadQueryIntoDb(queryterm = q) but I get the same result unfortunately.

ctrLoadQueryIntoDb(queryterm = q) q=https://www.clinicaltrialsregister.eu/ctr-search/search?query=Cancer&age=newborn&phase=phase-four

rfhb commented 5 years ago

Thanks (I hoped you had not seen my last comment, which I quickly removed since I still needed to work more on it as I had to change my Windows testing environment.)

The issue should now be fixed, by refactoring how Windows paths are escaped in a call of cygwin's bash, with commit 23c8807.

Please try again after installing the current development version with devtools::install_github("rfhb/ctrdata"), thank you.

JoeFuzz83 commented 5 years ago

It's working perfectly now, thanks Ralf!

Very much appreciated,

Joe

rfhb commented 5 years ago

Thanks for raising the issue, Windows is not my primary environment.