search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.78k stars 6.79k forks source link

Error in mx.io.CSVIter #6854

Closed cehagmann closed 7 years ago

cehagmann commented 7 years ago

I am trying to run a VDCNN on text, based on code here: https://github.com/Azure/Cortana-Intelligence-Gallery-Content/tree/master/Tutorials/Deep-Learning-for-Text-Classification-in-Azure/R Everytime I try to run mx.io.CSVIter (in CustomCSVIter or by itself) I get a "no file or directory" error and the stacktrace below. That folder definitely exists. I have tried this on two different Macbooks running 0.9.4 and 0.10.0. Same result, but it is not consistent. One time I was able to get past this point in the CNN demo with amazon product review sample data. Any ideas how to make it work consistently? I have tried re-installing mxnet, but same result.

Code Example

csv_iter <- mx.io.CSVIter(data.csv=data.csv, data.shape=feature.len+1, batch.size=batch.size)

Environment info

Operating System: Mac OSX El Capitan / Sierra

Package used: R

MXNet version: 0.9.4

R sessionInfo():

Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] grid      parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] readr_1.0.0        quanteda_0.9.9-24  mxnet_0.9.4        caret_6.0-73       ggplot2_2.2.1      lattice_0.20-34    httr_1.2.1        
 [8] bindrcpp_0.1       googlesheets_0.2.1 LiblineaR_2.10-8   stringr_1.2.0      tokenizers_0.1.4   jsonlite_1.3       tm_0.7            
[15] NLP_0.1-10         fields_8.10        maps_3.1.1         spam_1.4-0         purrr_0.2.2        text2vec_0.5.0     tidytext_0.1.2    
[22] dplyr_0.7.0        plyr_1.8.4         doParallel_1.0.10  iterators_1.0.8    foreach_1.4.3     

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.11        tidyr_0.6.1         visNetwork_1.0.3    class_7.3-14        assertthat_0.1      digest_0.6.12       psych_1.6.12       
 [8] slam_0.1-40         R6_2.2.1            cellranger_1.1.0    MatrixModels_0.4-1  stats4_3.3.1        e1071_1.6-8         rlang_0.1.1        
[15] lazyeval_0.2.0      curl_2.3            minqa_1.2.4         data.table_1.10.4   SparseM_1.74        car_2.1-4           nloptr_1.0.4       
[22] Matrix_1.2-8        splines_3.3.1       lme4_1.1-12         foreign_0.8-67      htmlwidgets_0.8     munsell_0.4.3       broom_0.4.2        
[29] compiler_3.3.1      janeaustenr_0.1.4   mnormt_1.5-5        mgcv_1.8-17         htmltools_0.3.5     nnet_7.3-12         openssl_0.9.6      
[36] tibble_1.3.3        codetools_0.2-15    MASS_7.3-45         ModelMetrics_1.1.0  SnowballC_0.5.1     nlme_3.1-131        gtable_0.2.0       
[43] magrittr_1.5        scales_0.4.1        RcppParallel_4.3.20 stringi_1.1.2       reshape2_1.4.2      ca_0.70             xml2_1.1.1         
[50] fastmatch_1.1-0     tools_3.3.1         glue_1.0.0          pbkrtest_0.4-6      colorspace_1.3-2    bindr_0.1           quantreg_5.29

Error Message:


Stack trace returned 33 entries:
[bt] (0) 0   libmxnet.so                         0x000000013d00a3fc _ZN4dmlc15LogMessageFatalD2Ev + 44
[bt] (1) 1   libmxnet.so                         0x000000013d005ea9 _ZN4dmlc15LogMessageFatalD1Ev + 9
[bt] (2) 2   libmxnet.so                         0x000000013d90024c _ZN4dmlc2io15LocalFileSystem13ListDirectoryERKNS0_3URIEPNSt3__16vectorINS0_8FileInfoENS5_9allocatorIS7_EEEE + 1404
[bt] (3) 3   libmxnet.so                         0x000000013d8e847c _ZN4dmlc2io14InputSplitBase17InitInputFileInfoERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE + 1260
[bt] (4) 4   libmxnet.so                         0x000000013d8e7dd7 _ZN4dmlc2io14InputSplitBase4InitEPNS0_10FileSystemEPKcm + 87
[bt] (5) 5   libmxnet.so                         0x000000013d8f9e31 _ZN4dmlc10InputSplit6CreateEPKcjjS2_ + 913
[bt] (6) 6   libmxnet.so                         0x000000013d90276e _ZN4dmlc4data15CreateCSVParserIjEEPNS_6ParserIT_EERKNSt3__112basic_stringIcNS6_11char_traitsIcEENS6_9allocatorIcEEEERKNS6_3mapISC_SC_NS6_4lessISC_EENSA_INS6_4pairISD_SC_EEEEEEjj + 46
[bt] (7) 7   libmxnet.so                         0x000000013d901a89 _ZN4dmlc4data13CreateParser_IjEEPNS_6ParserIT_EEPKcjjS7_ + 761
[bt] (8) 8   libmxnet.so                         0x000000013d69bc43 _ZN5mxnet2io7CSVIter4InitERKNSt3__16vectorINS2_4pairINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEESA_EENS8_ISB_EEEE + 179
[bt] (9) 9   libmxnet.so                         0x000000013d696b25 _ZN5mxnet2io11BatchLoader4InitERKNSt3__16vectorINS2_4pairINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEESA_EENS8_ISB_EEEE + 213
[bt] (10) 10  libmxnet.so                         0x000000013d690bb5 _ZN5mxnet2io14PrefetcherIter4InitERKNSt3__16vectorINS2_4pairINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEESA_EENS8_ISB_EEEE + 165
[bt] (11) 11  libmxnet.so                         0x000000013d62250c MXDataIterCreateIter + 588
[bt] (12) 12  mxnet.so                            0x000000013872f817 _ZN5mxnet1R22DataIterCreateFunctionclEPP7SEXPREC + 1207
[bt] (13) 13  Rcpp.so                             0x000000011b03a046 _Z23InternalFunction_invokeP7SEXPREC + 182
[bt] (14) 14  libR.dylib                          0x0000000101016d6a do_External + 378
[bt] (15) 15  libR.dylib                          0x000000010104bd6c Rf_eval + 988
[bt] (16) 16  libR.dylib                          0x00000001010a3d43 do_begin + 451
[bt] (17) 17  libR.dylib                          0x000000010104bf9e Rf_eval + 1550
[bt] (18) 18  libR.dylib                          0x00000001010a0a45 Rf_applyClosure + 1493
[bt] (19) 19  libR.dylib                          0x000000010104bea7 Rf_eval + 1303
[bt] (20) 20  libR.dylib                          0x00000001010a3d43 do_begin + 451
[bt] (21) 21  libR.dylib                          0x000000010104bf9e Rf_eval + 1550
[bt] (22) 22  libR.dylib                          0x00000001010a0a45 Rf_applyClosure + 1493
[bt] (23) 23  libR.dylib                          0x000000010104bea7 Rf_eval + 1303
[bt] (24) 24  libR.dylib                          0x00000001010a48b2 do_set + 1858
[bt] (25) 25  libR.dylib                          0x000000010104bf9e Rf_eval + 1550
[bt] (26) 26  libR.dylib                          0x00000001010d49e3 Rf_ReplIteration + 675
[bt] (27) 27  libR.dylib                          0x00000001010d5ff5 R_ReplConsole + 149
[bt] (28) 28  libR.dylib                          0x00000001010d5f22 run_Rmainloop + 82
[bt] (29) 29  rsession                            0x0000000100749c59 _ZN7rstudio1r7session12runEmbeddedRERKNS_4core8FilePathES5_bb7SA_TYPERKNS1_9CallbacksEPNS1_17InternalCallbacksE + 425
[bt] (30) 30  rsession                            0x0000000100729a59 _ZN7rstudio1r7session3runERKNS1_8ROptionsERKNS1_10RCallbacksE + 3161
[bt] (31) 31  rsession                            0x000000010003e828 main + 24168
[bt] (32) 32  rsession                            0x0000000100003434 start + 52

Error in mx.varg.io.CSVIter(list(...)) : 
  [08:54:49] src/io/local_filesys.cc:86: LocalFileSystem.ListDirectory ~/Dropbox/mxnetRcnn/data error: No such file or directory

Stack trace returned 33 entries:
[bt] (0) 0   libmxnet.so                         0x000000013d00a3fc _ZN4dmlc15LogMessageFatalD2Ev + 44
[bt] (1) 1   libmxnet.so                         0x000000013d005ea9 _ZN4dmlc15LogMessageFatalD1Ev + 9
[bt] (2) 2   libmxnet.so                         0x000000013d90024c _ZN4dmlc2io15LocalFileSystem13ListDirectoryERKNS0_3URIEPNSt3__16vectorINS0_8FileInfoENS5_9allocatorIS7_EEEE + 1404
[bt] (3) 3   libmxnet.so                         0x000000013d8e847c _ZN4dmlc2io14InputSplitBase17InitInputFileInfoERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE + 1260
[bt] (4) 4   libmxnet.so                         0x000000013d8e7dd7 _ZN4dmlc2io14InputSplitBase4InitEPNS0_10FileSystemEPKcm + 87
[bt] (5) 5   libmxnet.so                         0x000000013d8f9e31 _ZN4dmlc10InputSplit6CreateEPKcjjS2_ + 913
[bt] (6) 6
thirdwing commented 7 years ago

Can you try the full path? Not starting with "~".

I think that will be the problem.

cehagmann commented 7 years ago

I made that change and now R encounters a fatal error and force-quits on me.

cehagmann commented 7 years ago

It was crashing because of a change I made to feature length. Needed to be feature.len + 1. Working fine now.