search

neurodata / primitives-interfaces

A mirror of D3M's primitives-interfaces
Apache License 2.0
1 stars 4 forks source link

Primitives don't work, ask for CRAN mirror #33

Closed remram44 closed 5 years ago

remram44 commented 6 years ago

The primitives appear extremely broken in the Docker image. They ask to select a CRAN mirror on the terminal when I run d3m.primitives.jhu_primitives.SeededGraphMatching. Not only does that freeze the process, but if stdin is closed, it will enter in a loop asking over and over, spewing hundreds of megabytes per second in the log.

Downloading code at runtime is also definitely against the rules.

2018-08-05 22:54:40,809:INFO:score-231:d3m_ta2_nyu.workflow.execute:Executing module d3m.primitives.jhu_primitives.SeededGraphMatching, inputs: inputs
--- Please select a CRAN mirror for use in this session ---

/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Loading required package: igraph

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)

  warnings.warn(x, RRuntimeWarning)
Secure CRAN mirrors

 1: 0-Cloud [https]                   2: Algeria [https]
 3: Australia (Canberra) [https]      4: Australia (Melbourne 1) [https]
 5: Australia (Melbourne 2) [https]   6: Australia (Perth) [https]
 7: Austria [https]                   8: Belgium (Ghent) [https]
 9: Brazil (PR) [https]              10: Brazil (RJ) [https]
11: Brazil (SP 1) [https]            12: Brazil (SP 2) [https]
13: Bulgaria [https]                 14: Chile 1 [https]
15: Chile 2 [https]                  16: China (Guangzhou) [https]
17: China (Lanzhou) [https]          18: China (Shanghai 1) [https]
19: China (Shanghai 2) [https]       20: Colombia (Cali) [https]
21: Czech Republic [https]           22: Denmark [https]
23: East Asia [https]                24: Ecuador (Cuenca) [https]
25: Ecuador (Quito) [https]          26: Estonia [https]
27: France (Lyon 1) [https]          28: France (Lyon 2) [https]
29: France (Marseille) [https]       30: France (Montpellier) [https]
31: France (Paris 2) [https]         32: Germany (Erlangen) [https]
33: Germany (Göttingen) [https]      34: Germany (Münster) [https]
35: Greece [https]                   36: Iceland [https]
37: India [https]                    38: Indonesia (Jakarta) [https]
39: Ireland [https]                  40: Italy (Padua) [https]
41: Japan (Tokyo) [https]            42: Japan (Yonezawa) [https]
43: Korea (Busan) [https]            44: Korea (Seoul 1) [https]
45: Korea (Ulsan) [https]            46: Malaysia [https]
47: Mexico (Mexico City) [https]     48: Norway [https]
49: Philippines [https]              50: Serbia [https]
51: Spain (A Coruña) [https]         52: Spain (Madrid) [https]
53: Sweden [https]                   54: Switzerland [https]
55: Turkey (Denizli) [https]         56: Turkey (Mersin) [https]
57: UK (Bristol) [https]             58: UK (London 1) [https]
59: USA (CA 1) [https]               60: USA (IA) [https]
61: USA (KS) [https]                 62: USA (MI 1) [https]
63: USA (NY) [https]                 64: USA (OR) [https]
65: USA (TN) [https]                 66: USA (TX 1) [https]
67: Vietnam [https]                  68: (other mirrors)

Selection: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py", line 158, in consoleRead
    text = input(prompt)
EOFError: EOF when reading a line
Enter an item from the menu, or 0 to exit

Selection: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py", line 158, in consoleRead
    text = input(prompt)
EOFError: EOF when reading a line
Enter an item from the menu, or 0 to exit

Selection: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py", line 158, in consoleRead
    text = input(prompt)
EOFError: EOF when reading a line
Enter an item from the menu, or 0 to exit
jagterberg commented 6 years ago

see my response: https://gitlab.datadrivendiscovery.org/jpl/primitives_repo/merge_requests/496#note_8215

mitar commented 6 years ago

Copying response here as well:

I do not remember any discussion about this before.

I remember only that you said that the version of R in 17.10 is good one and you can use it.

So, things you can do here instead of requiring custom Dockerfile, is that you use post-install and post-devel hooks in setup.py to do stuff. That is what some other performers are doing. So in there you could configure R.

Please do not install R though in this way. If there is an Ubuntu package, please just use it and find a way how that can work for you. I am not sure why official package would not work correctly. This is pretty strange.

Also, do you have to install packages through R or are there Ubuntu packages corresponding to those R packages?

Finally, configuring stuff for current user (~/.Rprofile) is bound to fail because stuff can be run as different users. You have to configure things globally. In any case, you should not have to configure a mirror at runtime or download anything at runtime.

jagterberg commented 6 years ago

Reposting from GitLab:

I originally discussed this with Sujen when debugging our own pipelines.

I think that we can get Ubuntu packages given that library(XXX) from R still works. The issue is that R-igraph either isn't compiled properly or not downloaded, so library(igraph) is failing at runtime.

The real fix will come when we just have no R dependencies (which began just prior to the workshop, and is set to finish before the next evaluation).

hhelm10 commented 5 years ago

R dependency was completely removed.