search

kbenoit / sophistication

R package associated with Benoit, Munger and Spirling (2017) paper(s)
42 stars 7 forks source link

Package installation instructions need to describe spacyr in more detail #11

Closed lcombs closed 4 years ago

lcombs commented 4 years ago

I received the following error message when attempting to install the package for the first time:

Finding a python executable with spaCy installed...
Error: package or namespace load failed for 'sophistication':
 .onAttach failed in attachNamespace() for 'sophistication', details:
  call: set_spacy_python_option(python_executable, virtualenv, condaenv, 
  error: spaCy or language model en_core_web_sm is not installed in any of python executables.
Error: loading failed
Execution halted

After troubleshooting, I found that this would be a solution to my problem:

# install and load spacyr
devtools::install_github("quanteda/spacyr", build_vignettes = FALSE)
library("spacyr")
spacy_install()
spacy_initialize()
# source: https://cran.r-project.org/web/packages/spacyr/readme/README.html

After this, I was able to install sophistication using devtools::install_github("kbenoit/sophistication"). However, I also needed to install the following dependency first: install.packages('quanteda.textmodels').

Additionally, other team members had new issues on installation:

ArthurSpirling commented 4 years ago

@kbenoit -- @lcombs is one of my team at CDS working on filing issues for sophistication. I think she has some more in the pipe.

kbenoit commented 4 years ago

For diagnosing Installation issues, we always need the output of sessionInfo(). Many times, it's because a user's system is old or the R version is old. sessionInfo() will reveal that. Each problem should be logged specifically so we can try to fix it.

You should not need quanteda.textmodels in order to use the sophistication package. Can you indicate what failure was caused when you had not first installed quanteda.textmodels?

RcppArmadillo is not part of sophistication (but is required by quanteda.textmodels). If you experienced problems, please file an issue at https://github.com/quanteda/quanteda.textmodels. (But that package looks pretty clean: https://cran.r-project.org/web/checks/check_results_quanteda.textmodels.html)

Just to be clear, I'm happy to help and delighted to see you working on this @lcombs. The best way to file issues is to clearly separate the problems and provide as much info as possible so that they can be reproduced or understood. (Hence the suggestion on sessionInfo().)

lcombs commented 4 years ago

Thank you for your comments! It's my first time posting an issue, so I appreciate the feedback. I'll separate the remaining comments into separate issues moving forward.

For the main problem, it might very well be that my machine is old. Nonetheless, I've added the session information below. We did have another team member with the same issue, but for Windows machine. We should all be running the same version of R since we just recently re-installed the same version together.

> sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Mojave 10.14.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

Random number generation:
 RNG:     Mersenne-Twister 
 Normal:  Inversion 
 Sample:  Rounding 

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] rstudioapi_0.11   magrittr_1.5      usethis_1.6.1     devtools_2.3.0   
 [5] pkgload_1.1.0     R6_2.4.1          rlang_0.4.6       fansi_0.4.1      
 [9] tools_4.0.0       pkgbuild_1.0.8    sessioninfo_1.1.1 cli_2.0.2        
[13] withr_2.2.0       ellipsis_0.3.1    remotes_2.1.1     assertthat_0.2.1 
[17] digest_0.6.25     rprojroot_1.3-2   crayon_1.3.4      processx_3.4.2   
[21] callr_3.4.3       fs_1.4.1          ps_1.3.3          testthat_2.3.2   
[25] memoise_1.1.0     glue_1.4.1        compiler_4.0.0    desc_1.2.0       
[29] backports_1.1.7   prettyunits_1.1.1
kbenoit commented 4 years ago

Please provide the output from your install.packages() failure, including the call that triggered the failure. e.g.

> devtools::install_github("kbenoit/sophistication")
Downloading GitHub repo kbenoit/sophistication@master
Skipping 1 packages ahead of CRAN: quanteda
✓  checking for file ‘/private/var/folders/92/64fddl_57nddq_wwqpjnglwn48rjsn/T/Rtmpj3rf80/remotesb0035b8e7cf/kbenoit-sophistication-dd20d91/DESCRIPTION’ ...
─  preparing ‘sophistication’: (407ms)
✓  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘sophistication_0.70.tar.gz’
   Warning: invalid uid value replaced by that for user 'nobody'
   Warning: invalid gid value replaced by that for user 'nobody'

Installing package into ‘/Users/kbenoit/Library/R/4.0/library’
(as ‘lib’ is unspecified)
* installing *source* package ‘sophistication’ ...
** using staged installation
** R
** data
*** moving datasets to lazyload DB
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
spacy python option is already set, spacyr will use:
    condaenv = "spacy_condaenv"
successfully initialized (spaCy Version: 2.0.16, language model: en_core_web_sm)
(python options: type = "condaenv", value = "spacy_condaenv")
** testing if installed package can be loaded from final location
spacy python option is already set, spacyr will use:
    condaenv = "spacy_condaenv"
successfully initialized (spaCy Version: 2.0.16, language model: en_core_web_sm)
(python options: type = "condaenv", value = "spacy_condaenv")
** testing if installed package keeps a record of temporary installation path
* DONE (sophistication)
Adding ‘sophistication_0.70.tgz’ to the cache

If you have a problem installing spacyr, better to file the issue on that repo.

lcombs commented 4 years ago

Thanks for bearing with me...

Here is the error I receive if I go back to the original install on my system:

> devtools::install_github("kbenoit/sophistication")
Downloading GitHub repo kbenoit/sophistication@master
Installing 1 packages: spacyr
trying URL 'https://cran.rstudio.com/bin/macosx/contrib/4.0/spacyr_1.2.1.tgz'
Content type 'application/x-gzip' length 267722 bytes (261 KB)
==================================================
downloaded 261 KB

The downloaded binary packages are in
    /var/folders/w5/p8sm469n19n_gtsshmv6_lym0000gn/T//RtmpDiSmbN/downloaded_packages
✓  checking for file ‘/private/var/folders/w5/p8sm469n19n_gtsshmv6_lym0000gn/T/RtmpDiSmbN/remotes854579713c54/kbenoit-sophistication-dd20d91/DESCRIPTION’ (350ms)
─  preparing ‘sophistication’: (342ms)
✓  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘sophistication_0.70.tar.gz’

* installing *source* package ‘sophistication’ ...
** using staged installation
** R
** data
*** moving datasets to lazyload DB
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
Finding a python executable with spaCy installed...
Error: package or namespace load failed for ‘sophistication’:
 .onAttach failed in attachNamespace() for 'sophistication', details:
  call: set_spacy_python_option(python_executable, virtualenv, condaenv, 
  error: spaCy or language model en_core_web_sm is not installed in any of python executables.
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/sophistication’
Error: Failed to install 'sophistication' from GitHub:
  (converted from warning) installation of package ‘/var/folders/w5/p8sm469n19n_gtsshmv6_lym0000gn/T//RtmpDiSmbN/file8545227eeac8/sophistication_0.70.tar.gz’ had non-zero exit status
lcombs commented 4 years ago

As a follow-up, I resolved the issues by installing miniconda and taking the steps from this source: https://cran.r-project.org/web/packages/spacyr/readme/README.html .

> devtools::install_github("quanteda/spacyr", build_vignettes = FALSE, force=TRUE)
Downloading GitHub repo quanteda/spacyr@master
✓  checking for file ‘/private/var/folders/w5/p8sm469n19n_gtsshmv6_lym0000gn/T/RtmpDiSmbN/remotes854558c1f04c/quanteda-spacyr-06b4ead/DESCRIPTION’ ...
─  preparing ‘spacyr’:
✓  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘spacyr_1.2.1.tar.gz’

* installing *source* package ‘spacyr’ ...
** using staged installation
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (spacyr)
> library("spacyr")
> spacy_install()
Proceed? 

1: No
2: Yes

Selection: 2
Using existing conda environment  spacy_condaenv  for spaCy installation
. 
spaCy and language model(s): en_core_web_sm will be installed.  Installing spaCy...
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /Users/lisacombs/opt/miniconda3/envs/spacy_condaenv

  added / updated specs:
    - python

The following packages will be SUPERSEDED by a higher-priority channel:

  ca-certificates    conda-forge::ca-certificates-2020.6.2~ --> pkgs/main::ca-certificates-2020.1.1-0
  certifi            conda-forge::certifi-2020.6.20-py36h9~ --> pkgs/main::certifi-2020.6.20-py36_0
  openssl            conda-forge::openssl-1.1.1g-h0b31af3_0 --> pkgs/main::openssl-1.1.1g-h1de35cc_0

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /Users/lisacombs/opt/miniconda3/envs/spacy_condaenv

  added / updated specs:
    - python
    - spacy

The following packages will be UPDATED:

  ca-certificates     pkgs/main::ca-certificates-2020.1.1-0 --> conda-forge::ca-certificates-2020.6.20-hecda079_0

The following packages will be SUPERSEDED by a higher-priority channel:

  certifi               pkgs/main::certifi-2020.6.20-py36_0 --> conda-forge::certifi-2020.6.20-py36h9f0ad1d_0
  openssl              pkgs/main::openssl-1.1.1g-h1de35cc_0 --> conda-forge::openssl-1.1.1g-h0b31af3_0

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Installing model "en_core_web_sm"

Requirement already satisfied: en_core_web_sm==2.3.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.0/en_core_web_sm-2.3.0.tar.gz#egg=en_core_web_sm==2.3.0 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (2.3.0)
Requirement already satisfied: spacy<2.4.0,>=2.3.0 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from en_core_web_sm==2.3.0) (2.3.0)
Requirement already satisfied: thinc==7.4.1 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (7.4.1)
Requirement already satisfied: catalogue<1.1.0,>=0.0.7 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (1.0.0)
Requirement already satisfied: numpy>=1.15.0 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (1.18.5)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (3.0.2)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (2.0.3)
Requirement already satisfied: blis<0.5.0,>=0.4.0 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (0.4.1)
Requirement already satisfied: srsly<1.1.0,>=1.0.2 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (1.0.2)
Requirement already satisfied: setuptools in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (47.3.1.post20200616)
Requirement already satisfied: plac<1.2.0,>=0.9.6 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (0.9.6)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (1.0.0)
Requirement already satisfied: wasabi<1.1.0,>=0.4.0 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (0.7.0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (2.24.0)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (4.46.1)
Requirement already satisfied: importlib-metadata>=0.20; python_version < "3.8" in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from catalogue<1.1.0,>=0.0.7->spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (1.6.1)
Requirement already satisfied: chardet<4,>=3.0.2 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (2020.6.20)
Requirement already satisfied: idna<3,>=2.5 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from requests<3.0.0,>=2.13.0->spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (1.25.9)
Requirement already satisfied: zipp>=0.5 in ./opt/miniconda3/envs/spacy_condaenv/lib/python3.6/site-packages (from importlib-metadata>=0.20; python_version < "3.8"->catalogue<1.1.0,>=0.0.7->spacy<2.4.0,>=2.3.0->en_core_web_sm==2.3.0) (3.1.0)
✔ Download and installation successful
You can now load the model via spacy.load('en_core_web_sm')
Language model "en_core_web_sm" is successfully installed

Installation complete.
Condaenv: spacy_condaenv; Language model(s): en_core_web_sm

> spacy_initialize()
spaCy is already initialized
NULL
> # install sophistication
> devtools::install_github("kbenoit/sophistication")
Downloading GitHub repo kbenoit/sophistication@master
✓  checking for file ‘/private/var/folders/w5/p8sm469n19n_gtsshmv6_lym0000gn/T/RtmpDiSmbN/remotes85452e65b642/kbenoit-sophistication-dd20d91/DESCRIPTION’ ...
─  preparing ‘sophistication’:
✓  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘sophistication_0.70.tar.gz’

* installing *source* package ‘sophistication’ ...
** using staged installation
** R
** data
*** moving datasets to lazyload DB
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
Found 'spacy_condaenv'. spacyr will use this environment
successfully initialized (spaCy Version: 2.3.0, language model: en_core_web_sm)
(python options: type = "condaenv", value = "spacy_condaenv")
** testing if installed package can be loaded from final location
Found 'spacy_condaenv'. spacyr will use this environment
successfully initialized (spaCy Version: 2.3.0, language model: en_core_web_sm)
(python options: type = "condaenv", value = "spacy_condaenv")
** testing if installed package keeps a record of temporary installation path
* DONE (sophistication)
kbenoit commented 4 years ago

Could be worth updating the sophistication package installation instructions to reflect this.

kbenoit commented 4 years ago

@lcombs updating the documentation as per above would be a good trial for a first pull request. First, you create a branch, then from the branch you would modify the installation instructions (currently at README.Rmd), and then make the changes to your branch. When it's ready for review, you issue a PR, and I will review it before merging it into the master branch.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

lcombs commented 4 years ago

Merged PR with additional installation instructions into master