Closed jmcastagnetto closed 3 years ago
Just getting to this now, thank you for your patience! This is a bit strange, but, then, that's why we have unit tests.
Could you please run this script and tell me if you get zeros printed at the end?
# Set this to your own zaminfluence directory.
base_dir <- "/home/rgiordan/Documents/git_repos/zaminfluence"
setwd(base_dir)
library(devtools)
library(tidyverse)
library(zaminfluence)
devtools::load_all("zaminfluence") # Load even non-exported zaminfluence functions.
py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
# Test utility for generating data
source(file.path(base_dir, "zaminfluence/tests/testthat/utils.R"))
set.seed(42)
# Generate data and a regression model.
x_dim <- 3
beta_true <- runif(x_dim)
df <- GenerateRegressionData(10, beta_true, num_groups=NULL)
# Fit a regression model.
x_names <- sprintf("x%d", 1:x_dim)
reg_form <- formula(sprintf("y ~ %s - 1", paste(x_names, collapse=" + ")))
reg_fit <- lm(data = df, formula = reg_form, x=TRUE, y=TRUE)
py_main_set <- SetPythonRegressionVariables(reg_fit)
# If we are setting the python variables correctly, these differences should all be zero.
CheckPyMain <- function(py_main_check) {
print(py_main_check$x - df[, x_names] %>% as.matrix())
print(py_main_check$y - df$y %>% as.numeric())
print(py_main_check$beta - reg_fit$coefficients)
print(py_main_check$w0 - rep(1, nrow(df)))
}
CheckPyMain(py_main_set)
CheckPyMain(py_main)
For example, when I run this on a working installation, I get
> CheckPyMain(py_main_set)
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
> CheckPyMain(py_main)
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
For some context, the error you're getting is because the Python part (which computes the derivatives) is being asked to evaluate the sensitivity at values which are not the OLS solution. The most likely culprit is that the values are not being copied correctly from Python to R, so that's what we're checking here.
Sorry I just had time to run your script, and it works w/o a problem:
$ Rscript test.R
Loading required package: usethis
── Attaching packages ───────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.3.0 ──
✔ ggplot2 3.3.2 ✔ purrr 0.3.4
✔ tibble 3.0.4 ✔ dplyr 1.0.2
✔ tidyr 1.1.2 ✔ stringr 1.4.0
✔ readr 1.4.0 ✔ forcats 0.5.0
── Conflicts ──────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
Loading required package: reticulate
Loading required package: latex2exp
Loading required package: broom
Loading zaminfluence
Attaching package: ‘testthat’
The following object is masked from ‘package:dplyr’:
matches
The following object is masked from ‘package:purrr’:
is_null
The following object is masked from ‘package:tidyr’:
matches
The following object is masked from ‘package:devtools’:
test_file
Attaching package: ‘gridExtra’
The following object is masked from ‘package:dplyr’:
combine
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
Also, tried doing a git pull origin
, and see that there are updates in the README.md
and zaminfluence/DESCRIPTION
files.
Just in case, did a reinstall of both the python and the R parts, and got the same errors as before.
Thanks! Next, could I ask you to append the following code to the end of the previous debugging script and share what you get? Here, I'm just running the constituent parts of ComputeRegressionInfluence
, leading up to the function itself, to see what is going wrong. If this works, then I guess the problem is in the unit testing setup.
# Manually run the Python code in ComputeRegressionInfluence
reticulate::py_run_string("
reg_obj = regsens_rgiordandev.reg_obj
get_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix
sens_reg_obj = lambda beta, w: reg_obj(beta, y=y, x=x, w=w)
obs_w_sens = vittles.HyperparameterSensitivityLinearApproximation(
objective_fun=sens_reg_obj,
opt_par_value=beta,
hyper_par_value=w0,
validate_optimum=True,
grad_tol=1e-08)
get_betahat = obs_w_sens.get_opt_par_function()
def get_se(w):
betahat = get_betahat(w)
se_cov = get_standard_error_matrix(betahat, y, x, w=w, se_group=se_group)
return np.sqrt(np.diag(se_cov))
se = get_se(w0)
betahat_grad = obs_w_sens.get_dopt_dhyper()
se_grad = autograd.jacobian(get_se)(w0)
")
print(py_main$betahat_grad)
print(py_main$se_grad)
# Manually run the R code in ComputeRegressionInfluence
py_main <- SetPythonRegressionVariables(reg_fit, se_group=NULL)
reg <- broom::tidy(reg_fit)
reticulate::py_run_string("
betahat = regsens_rgiordandev.reg(y, x, w=w0)
se, betahat_grad, se_grad = regsens_rgiordandev.get_regression_w_grads(
betahat, y, x, w0, se_group=se_group)
")
print(py_main$betahat_grad)
print(py_main$se_grad)
# Just run ComputeRegressionInfluence
ComputeRegressionInfluence(reg_fit, se_group=NULL)
OK, done, the final script is
# Set this to your own zaminfluence directory.
base_dir <- "/home/jesus/devel/others/zaminfluence"
setwd(base_dir)
library(devtools)
library(tidyverse)
library(zaminfluence)
devtools::load_all("zaminfluence") # Load even non-exported zaminfluence functions.
py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
# Test utility for generating data
source(file.path(base_dir, "zaminfluence/tests/testthat/utils.R"))
set.seed(42)
# Generate data and a regression model.
x_dim <- 3
beta_true <- runif(x_dim)
df <- GenerateRegressionData(10, beta_true, num_groups=NULL)
# Fit a regression model.
x_names <- sprintf("x%d", 1:x_dim)
reg_form <- formula(sprintf("y ~ %s - 1", paste(x_names, collapse=" + ")))
reg_fit <- lm(data = df, formula = reg_form, x=TRUE, y=TRUE)
py_main_set <- SetPythonRegressionVariables(reg_fit)
# If we are setting the python variables correctly, these differences should all be zero.
CheckPyMain <- function(py_main_check) {
print(py_main_check$x - df[, x_names] %>% as.matrix())
print(py_main_check$y - df$y %>% as.numeric())
print(py_main_check$beta - reg_fit$coefficients)
print(py_main_check$w0 - rep(1, nrow(df)))
}
CheckPyMain(py_main_set)
CheckPyMain(py_main)
# new code below
# Manually run the Python code in ComputeRegressionInfluence
reticulate::py_run_string("
reg_obj = regsens_rgiordandev.reg_obj
get_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix
sens_reg_obj = lambda beta, w: reg_obj(beta, y=y, x=x, w=w)
obs_w_sens = vittles.HyperparameterSensitivityLinearApproximation(
objective_fun=sens_reg_obj,
opt_par_value=beta,
hyper_par_value=w0,
validate_optimum=True,
grad_tol=1e-08)
get_betahat = obs_w_sens.get_opt_par_function()
def get_se(w):
betahat = get_betahat(w)
se_cov = get_standard_error_matrix(betahat, y, x, w=w, se_group=se_group)
return np.sqrt(np.diag(se_cov))
se = get_se(w0)
betahat_grad = obs_w_sens.get_dopt_dhyper()
se_grad = autograd.jacobian(get_se)(w0)
")
print(py_main$betahat_grad)
print(py_main$se_grad)
# Manually run the R code in ComputeRegressionInfluence
py_main <- SetPythonRegressionVariables(reg_fit, se_group=NULL)
reg <- broom::tidy(reg_fit)
reticulate::py_run_string("
betahat = regsens_rgiordandev.reg(y, x, w=w0)
se, betahat_grad, se_grad = regsens_rgiordandev.get_regression_w_grads(
betahat, y, x, w0, se_group=se_group)
")
print(py_main$betahat_grad)
print(py_main$se_grad)
# Just run ComputeRegressionInfluence
ComputeRegressionInfluence(reg_fit, se_group=NULL)
And I see that the last call to reticulate::py_run_string()
seems to be the point where it breaks, and the first call generates some warnings because some values are NaN
Sorry I can't give more insight because I have not use reticulate
in my calculations.
$ Rscript test.R
Loading required package: usethis
── Attaching packages ───────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.3.0 ──
✔ ggplot2 3.3.2 ✔ purrr 0.3.4
✔ tibble 3.0.4 ✔ dplyr 1.0.2
✔ tidyr 1.1.2 ✔ stringr 1.4.0
✔ readr 1.4.0 ✔ forcats 0.5.0
── Conflicts ──────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
Loading required package: reticulate
Loading required package: latex2exp
Loading required package: broom
Loading zaminfluence
Attaching package: ‘testthat’
The following object is masked from ‘package:dplyr’:
matches
The following object is masked from ‘package:purrr’:
is_null
The following object is masked from ‘package:tidyr’:
matches
The following object is masked from ‘package:devtools’:
test_file
Attaching package: ‘gridExtra’
The following object is masked from ‘package:dplyr’:
combine
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/autograd-1.3-py3.8.egg/autograd/tracer.py:48: RuntimeWarning: invalid value encountered in sqrt
return f_raw(*args, **kwargs)
/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/autograd-1.3-py3.8.egg/autograd/numpy/numpy_vjps.py:99: RuntimeWarning: invalid value encountered in power
defvjp(anp.sqrt, lambda ans, x : lambda g: g * 0.5 * x**-0.5)
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.08487399 -0.07229308 0.148011153 0.03189895 0.016717574 0.05025406
[2,] -0.02707678 -0.25624932 -0.143256362 0.15593617 0.003747749 -0.41389411
[3,] 0.05874361 -0.54259092 -0.008895226 -0.05026861 -0.001068631 -0.61169193
[,7] [,8] [,9] [,10]
[1,] 0.4767400 -0.4690735 -0.06759143 -0.19953767
[2,] 0.1889611 0.5178116 0.06956875 -0.09554881
[3,] 0.7807653 0.3860833 -0.01931623 0.00823938
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
[2,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
[3,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Error in py_run_string_impl(code, local, convert) :
ValueError: The estimating equation is not zero at the proposed values. ||ee|| = 0.21276610243399807 > 1e-08 = solution_tol
Detailed traceback:
File "<string>", line 3, in <module>
File "/home/jesus/devel/others/zaminfluence/regression_sensitivity/regsens_rgiordandev/regression_sensitivity_lib.py", line 143, in get_regression_w_grads
obs_w_sens = vittles.HyperparameterSensitivityLinearApproximation(
File "/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/vittles-0.173-py3.8.egg/vittles/sensitivity_lib.py", line 365, in __init__
EstimatingEquationLinearApproximation.__init__(
File "/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/vittles-0.173-py3.8.egg/vittles/sensitivity_lib.py", line 186, in __init__
self.set_location(
File "/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/vittles-0.173-py3.8.egg/vittles/sensitivity_lib.py", line 215, in set_location
raise ValueError(err_msg)
Calls: <Anonymous> -> py_run_string_impl
Execution halted
Sorry, just to avoid ambiguity, could you please provide something that shows what command is being run, like the output from an interactive terminal, or
Rscript -e 'source("test.R", echo=TRUE)'
In case you're curious, reticulate
is just running a Python interpreter in the background, passing values back and forth with R through the list-like py_main
object. The zaminfluence
package computes derivatives by setting variables in the Python interpreter, computing derivatives using autograd
and vittles
, and then passing the results back to R
.
By the way, I assume the Python unit tests are passing, right?
source venv/bin/activate
cd $REPO/regression_sensitivity
python3 -m pytest
In the output above I put the command line used (vide supra)
Idem for the results of the python unit tests
Cheers
Yes, I understand. Could I please ask you to run
Rscript -e 'source("test.R", echo=TRUE)'
instead of Rscript test.R
? That way I can see all the output.
Sure thing, here it is:
$ Rscript -e 'source("test.R", echo=TRUE)'
> base_dir <- "/home/jesus/devel/others/zaminfluence"
> setwd(base_dir)
> library(devtools)
Loading required package: usethis
> library(tidyverse)
── Attaching packages ───────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.3.0 ──
✔ ggplot2 3.3.2 ✔ purrr 0.3.4
✔ tibble 3.0.4 ✔ dplyr 1.0.2
✔ tidyr 1.1.2 ✔ stringr 1.4.0
✔ readr 1.4.0 ✔ forcats 0.5.0
── Conflicts ──────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
> library(zaminfluence)
Loading required package: reticulate
Loading required package: latex2exp
Loading required package: broom
> devtools::load_all("zaminfluence")
Loading zaminfluence
Attaching package: ‘testthat’
The following object is masked from ‘package:dplyr’:
matches
The following object is masked from ‘package:purrr’:
is_null
The following object is masked from ‘package:tidyr’:
matches
The following object is masked from ‘package:devtools’:
test_file
> py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
> source(file.path(base_dir, "zaminfluence/tests/testthat/utils.R"))
Attaching package: ‘gridExtra’
The following object is masked from ‘package:dplyr’:
combine
> set.seed(42)
> x_dim <- 3
> beta_true <- runif(x_dim)
> df <- GenerateRegressionData(10, beta_true, num_groups = NULL)
> x_names <- sprintf("x%d", 1:x_dim)
> reg_form <- formula(sprintf("y ~ %s - 1", paste(x_names,
+ collapse = " + ")))
> reg_fit <- lm(data = df, formula = reg_form, x = TRUE,
+ y = TRUE)
> py_main_set <- SetPythonRegressionVariables(reg_fit)
> CheckPyMain <- function(py_main_check) {
+ print(py_main_check$x - df[, x_names] %>% as.matrix())
+ print(py_main_check$y - df$y %>% as.nume .... [TRUNCATED]
> CheckPyMain(py_main_set)
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
> CheckPyMain(py_main)
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
> reticulate::py_run_string("\nreg_obj = regsens_rgiordandev.reg_obj\nget_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix\nsens_ ..." ... [TRUNCATED]
/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/autograd-1.3-py3.8.egg/autograd/tracer.py:48: RuntimeWarning: invalid value encountered in sqrt
return f_raw(*args, **kwargs)
/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/autograd-1.3-py3.8.egg/autograd/numpy/numpy_vjps.py:99: RuntimeWarning: invalid value encountered in power
defvjp(anp.sqrt, lambda ans, x : lambda g: g * 0.5 * x**-0.5)
> print(py_main$betahat_grad)
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.08487399 -0.07229308 0.148011153 0.03189895 0.016717574 0.05025406
[2,] -0.02707678 -0.25624932 -0.143256362 0.15593617 0.003747749 -0.41389411
[3,] 0.05874361 -0.54259092 -0.008895226 -0.05026861 -0.001068631 -0.61169193
[,7] [,8] [,9] [,10]
[1,] 0.4767400 -0.4690735 -0.06759143 -0.19953767
[2,] 0.1889611 0.5178116 0.06956875 -0.09554881
[3,] 0.7807653 0.3860833 -0.01931623 0.00823938
> print(py_main$se_grad)
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
[2,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
[3,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
> py_main <- SetPythonRegressionVariables(reg_fit, se_group = NULL)
> reg <- broom::tidy(reg_fit)
> reticulate::py_run_string("\nbetahat = regsens_rgiordandev.reg(y, x, w=w0)\nse, betahat_grad, se_grad = regsens_rgiordandev.get_regression_w_grads(\ ..." ... [TRUNCATED]
Error in py_run_string_impl(code, local, convert) :
ValueError: The estimating equation is not zero at the proposed values. ||ee|| = 0.21276610243399807 > 1e-08 = solution_tol
Detailed traceback:
File "<string>", line 3, in <module>
File "/home/jesus/devel/others/zaminfluence/regression_sensitivity/regsens_rgiordandev/regression_sensitivity_lib.py", line 143, in get_regression_w_grads
obs_w_sens = vittles.HyperparameterSensitivityLinearApproximation(
File "/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/vittles-0.173-py3.8.egg/vittles/sensitivity_lib.py", line 365, in __init__
EstimatingEquationLinearApproximation.__init__(
File "/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/vittles-0.173-py3.8.egg/vittles/sensitivity_lib.py", line 186, in __init__
self.set_location(
File "/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/vittles-0.173-py3.8.egg/vittles/sensitivity_lib.py", line 215, in set_location
raise ValueError(err_msg)
Calls: source ... withVisible -> eval -> eval -> <Anonymous> -> py_run_string_impl
Execution halted
Confirms that the last reticulate
call is where it breaks.
Thanks for your patience! Before digging into the Python line by line, let's make sure that reticulate is picking up the right virtual environment. Could you please run (using Rscript -e 'source("test.R", echo=TRUE)'
):
# Set this to your own zaminfluence directory.
base_dir <- "/home/rgiordan/Documents/git_repos/zaminfluence"
setwd(base_dir)
library(devtools)
library(tidyverse)
library(zaminfluence)
devtools::load_all("zaminfluence") # Load even non-exported zaminfluence functions.
py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
# Manually run the Python code in ComputeRegressionInfluence. This is showing
# some problems.
PyPrint <- function(py_expr) {
reticulate::py_run_string(sprintf("foo = %s", py_expr))
print(py_main$foo)
}
reticulate::py_run_string("import pkg_resources")
reticulate::py_run_string("import sys")
PyPrint("sys.version")
PyPrint("vittles.__version__")
PyPrint("paragami.__version__")
PyPrint("pkg_resources.get_distribution(\"autograd\")")
Actually, please run the below script, which contains the version checks. Hopefully this will give some clues as to where things are going wrong.
# Set this to your own zaminfluence directory.
base_dir <- "/home/rgiordan/Documents/git_repos/zaminfluence"
setwd(base_dir)
library(devtools)
library(tidyverse)
library(zaminfluence)
devtools::load_all("zaminfluence") # Load even non-exported zaminfluence functions.
py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
# Manually run the Python code in ComputeRegressionInfluence. This is showing
# some problems.
PyPrint <- function(py_expr) {
reticulate::py_run_string(sprintf("foo = %s", py_expr))
print(py_main$foo)
}
reticulate::py_run_string("import pkg_resources")
reticulate::py_run_string("import sys")
PyPrint("sys.version")
PyPrint("vittles.__version__")
PyPrint("paragami.__version__")
PyPrint("pkg_resources.get_distribution(\"autograd\")")
#####################
# Test utility for generating data
source(file.path(base_dir, "zaminfluence/tests/testthat/utils.R"))
set.seed(42)
# Generate data and a regression model.
x_dim <- 3
beta_true <- runif(x_dim)
df <- GenerateRegressionData(10, beta_true, num_groups=NULL)
# Fit a regression model.
x_names <- sprintf("x%d", 1:x_dim)
reg_form <- formula(sprintf("y ~ %s - 1", paste(x_names, collapse=" + ")))
reg_fit <- lm(data = df, formula = reg_form, x=TRUE, y=TRUE)
py_main <- SetPythonRegressionVariables(reg_fit)
# If we are setting the python variables correctly, these differences should all be zero.
CheckPyMain <- function(py_main_check) {
print(py_main_check$x - df[, x_names] %>% as.matrix())
print(py_main_check$y - df$y %>% as.numeric())
print(py_main_check$beta - reg_fit$coefficients)
print(py_main_check$w0 - rep(1, nrow(df)))
}
CheckPyMain(py_main)
reticulate::py_run_string("
reg_obj = regsens_rgiordandev.reg_obj
get_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix
sens_reg_obj = lambda beta, w: reg_obj(beta, y=y, x=x, w=w)
obs_w_sens = vittles.HyperparameterSensitivityLinearApproximation(
objective_fun=sens_reg_obj,
opt_par_value=beta,
hyper_par_value=w0,
validate_optimum=True,
grad_tol=1e-08)
get_betahat = obs_w_sens.get_opt_par_function()
def get_se(w):
betahat = get_betahat(w)
print('get_se computed betahat = {}'.format(betahat))
se_cov = get_standard_error_matrix(betahat, y, x, w=w, se_group=se_group)
return np.sqrt(np.diag(se_cov))
")
PyPrint("get_betahat(w0)")
PyPrint("obs_w_sens._input_val0")
PyPrint("obs_w_sens._hyper_val0")
PyPrint("get_betahat(w0)")
PyPrint("obs_w_sens._input_val0")
PyPrint("obs_w_sens._hyper_val0")
PyPrint("get_se(w0)")
PyPrint("obs_w_sens._input_val0")
PyPrint("obs_w_sens._hyper_val0")
PyPrint("get_se(w0)")
PyPrint("obs_w_sens._input_val0")
PyPrint("obs_w_sens._hyper_val0")
PyPrint("obs_w_sens.get_dopt_dhyper()")
PyPrint("obs_w_sens._input_val0")
PyPrint("obs_w_sens._hyper_val0")
PyPrint("autograd.jacobian(get_se)(w0)")
PyPrint("obs_w_sens._input_val0")
PyPrint("obs_w_sens._hyper_val0")
OK, just finished some morning meetings, so I was able to run the script:
$ Rscript -e 'source("test2.R", echo=TRUE)'
> base_dir <- "/home/jesus/devel/others/zaminfluence"
> setwd(base_dir)
> library(devtools)
Loading required package: usethis
> library(tidyverse)
── Attaching packages ─────────────────────────────────────── tidyverse 1.3.0 ──
✔ ggplot2 3.3.2 ✔ purrr 0.3.4
✔ tibble 3.0.4 ✔ dplyr 1.0.2
✔ tidyr 1.1.2 ✔ stringr 1.4.0
✔ readr 1.4.0 ✔ forcats 0.5.0
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
> library(zaminfluence)
Loading required package: reticulate
Loading required package: latex2exp
Loading required package: broom
> devtools::load_all("zaminfluence")
Loading zaminfluence
Attaching package: ‘testthat’
The following object is masked from ‘package:dplyr’:
matches
The following object is masked from ‘package:purrr’:
is_null
The following object is masked from ‘package:tidyr’:
matches
The following object is masked from ‘package:devtools’:
test_file
> py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
> PyPrint <- function(py_expr) {
+ reticulate::py_run_string(sprintf("foo = %s", py_expr))
+ print(py_main$foo)
+ }
> reticulate::py_run_string("import pkg_resources")
> reticulate::py_run_string("import sys")
> PyPrint("sys.version")
[1] "3.8.5 (default, Jul 28 2020, 12:59:40) \n[GCC 9.3.0]"
> PyPrint("vittles.__version__")
[1] "0.173"
> PyPrint("paragami.__version__")
[1] "0.42"
> PyPrint("pkg_resources.get_distribution(\"autograd\")")
autograd 1.3
> source(file.path(base_dir, "zaminfluence/tests/testthat/utils.R"))
Attaching package: ‘gridExtra’
The following object is masked from ‘package:dplyr’:
combine
> set.seed(42)
> x_dim <- 3
> beta_true <- runif(x_dim)
> df <- GenerateRegressionData(10, beta_true, num_groups = NULL)
> x_names <- sprintf("x%d", 1:x_dim)
> reg_form <- formula(sprintf("y ~ %s - 1", paste(x_names,
+ collapse = " + ")))
> reg_fit <- lm(data = df, formula = reg_form, x = TRUE,
+ y = TRUE)
> py_main <- SetPythonRegressionVariables(reg_fit)
> CheckPyMain <- function(py_main_check) {
+ print(py_main_check$x - df[, x_names] %>% as.matrix())
+ print(py_main_check$y - df$y %>% as.nume .... [TRUNCATED]
> CheckPyMain(py_main)
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
> reticulate::py_run_string("\nreg_obj = regsens_rgiordandev.reg_obj\nget_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix\nsens_ ..." ... [TRUNCATED]
> PyPrint("get_betahat(w0)")
[1] 1.9036205 -0.1935714 -0.3037477
> PyPrint("obs_w_sens._input_val0")
[1] 1.9036205 -0.1935714 -0.3037477
> PyPrint("obs_w_sens._hyper_val0")
[1] 1 1 1 1 1 1 1 1 1 1
> PyPrint("get_betahat(w0)")
[1] 1.9036205 -0.1935714 -0.3037477
> PyPrint("obs_w_sens._input_val0")
[1] 1.9036205 -0.1935714 -0.3037477
> PyPrint("obs_w_sens._hyper_val0")
[1] 1 1 1 1 1 1 1 1 1 1
> PyPrint("get_se(w0)")
/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/autograd-1.3-py3.8.egg/autograd/tracer.py:48: RuntimeWarning: invalid value encountered in sqrt
return f_raw(*args, **kwargs)
get_se computed betahat = [ 1.90362055 -0.19357143 -0.30374769]
[1] 0.9887754 NaN NaN
> PyPrint("obs_w_sens._input_val0")
[1] 1.9036205 -0.1935714 -0.3037477
> PyPrint("obs_w_sens._hyper_val0")
[1] 1 1 1 1 1 1 1 1 1 1
> PyPrint("get_se(w0)")
get_se computed betahat = [ 1.90362055 -0.19357143 -0.30374769]
[1] 0.9887754 NaN NaN
> PyPrint("obs_w_sens._input_val0")
[1] 1.9036205 -0.1935714 -0.3037477
> PyPrint("obs_w_sens._hyper_val0")
[1] 1 1 1 1 1 1 1 1 1 1
> PyPrint("obs_w_sens.get_dopt_dhyper()")
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.08487399 -0.07229308 0.148011153 0.03189895 0.016717574 0.05025406
[2,] -0.02707678 -0.25624932 -0.143256362 0.15593617 0.003747749 -0.41389411
[3,] 0.05874361 -0.54259092 -0.008895226 -0.05026861 -0.001068631 -0.61169193
[,7] [,8] [,9] [,10]
[1,] 0.4767400 -0.4690735 -0.06759143 -0.19953767
[2,] 0.1889611 0.5178116 0.06956875 -0.09554881
[3,] 0.7807653 0.3860833 -0.01931623 0.00823938
> PyPrint("obs_w_sens._input_val0")
[1] 1.9036205 -0.1935714 -0.3037477
> PyPrint("obs_w_sens._hyper_val0")
[1] 1 1 1 1 1 1 1 1 1 1
> PyPrint("autograd.jacobian(get_se)(w0)")
/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/autograd-1.3-py3.8.egg/autograd/numpy/numpy_vjps.py:99: RuntimeWarning: invalid value encountered in power
defvjp(anp.sqrt, lambda ans, x : lambda g: g * 0.5 * x**-0.5)
get_se computed betahat = Autograd ArrayBox with value [ 1.90362055 -0.19357143 -0.30374769]
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
[2,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
[3,] NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
> PyPrint("obs_w_sens._input_val0")
[1] 1.9036205 -0.1935714 -0.3037477
> PyPrint("obs_w_sens._hyper_val0")
[1] 1 1 1 1 1 1 1 1 1 1
HTH
Ok, we're zooming in. Perhaps the regression data is getting corrupted. Could you please run this one?
# Set this to your own zaminfluence directory.
base_dir <- "/home/rgiordan/Documents/git_repos/zaminfluence"
setwd(base_dir)
library(devtools)
library(tidyverse)
library(zaminfluence)
devtools::load_all("zaminfluence") # Load even non-exported zaminfluence functions.
py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
# Manually run the Python code in ComputeRegressionInfluence. This is showing
# some problems.
PyPrint <- function(py_expr) {
reticulate::py_run_string(sprintf("foo = %s", py_expr))
print(py_main$foo)
}
reticulate::py_run_string("import pkg_resources")
reticulate::py_run_string("import sys")
PyPrint("sys.version")
PyPrint("vittles.__version__")
PyPrint("paragami.__version__")
PyPrint("pkg_resources.get_distribution(\"autograd\")")
#####################
# Test utility for generating data
source(file.path(base_dir, "zaminfluence/tests/testthat/utils.R"))
set.seed(42)
# Generate data and a regression model.
x_dim <- 3
beta_true <- runif(x_dim)
df <- GenerateRegressionData(10, beta_true, num_groups=NULL)
# Fit a regression model.
x_names <- sprintf("x%d", 1:x_dim)
reg_form <- formula(sprintf("y ~ %s - 1", paste(x_names, collapse=" + ")))
reg_fit <- lm(data = df, formula = reg_form, x=TRUE, y=TRUE)
py_main <- SetPythonRegressionVariables(reg_fit)
# If we are setting the python variables correctly, these differences should all be zero.
CheckPyMain <- function(py_main_check) {
print(py_main_check$x - df[, x_names] %>% as.matrix())
print(py_main_check$y - df$y %>% as.numeric())
print(py_main_check$beta - reg_fit$coefficients)
print(py_main_check$w0 - rep(1, nrow(df)))
}
CheckPyMain(py_main)
# Make deep copies of the correct values
reticulate::py_run_string("
from copy import deepcopy
x_copy = deepcopy(x)
y_copy = deepcopy(y)
w0_copy = deepcopy(w0)
beta_copy = deepcopy(beta)
")
reticulate::py_run_string("
reg_obj = regsens_rgiordandev.reg_obj
get_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix
sens_reg_obj = lambda beta, w: reg_obj(beta, y=y, x=x, w=w)
obs_w_sens = vittles.HyperparameterSensitivityLinearApproximation(
objective_fun=sens_reg_obj,
opt_par_value=beta,
hyper_par_value=w0,
validate_optimum=True,
grad_tol=1e-08)
get_betahat = obs_w_sens.get_opt_par_function()
def get_se(w):
betahat = get_betahat(w)
print('get_se computed betahat = {}'.format(betahat))
print('get_se x error = {}'.format(x_copy - x))
print('get_se y error = {}'.format(y_copy - y))
print('get_se w difference = {}'.format(w0_copy - w))
print('get_se se_group = {}'.format(se_group))
se_cov = get_standard_error_matrix(betahat, y, x, w=w, se_group=se_group)
print('get_se se_cov = {}'.format(se_cov))
return np.sqrt(np.diag(se_cov))
")
# Run it twice just to make sure the values are the same each time.
PyPrint("get_se(w0)")
PyPrint("get_se(w0)")
Here is the output of running the code above.
$ Rscript -e 'source("test3.R", echo=TRUE)'
> base_dir <- "/home/jesus/devel/others/zaminfluence"
> setwd(base_dir)
> library(devtools)
Loading required package: usethis
> library(tidyverse)
── Attaching packages ───────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.3.0 ──
✔ ggplot2 3.3.2 ✔ purrr 0.3.4
✔ tibble 3.0.4 ✔ dplyr 1.0.2
✔ tidyr 1.1.2 ✔ stringr 1.4.0
✔ readr 1.4.0 ✔ forcats 0.5.0
── Conflicts ──────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
> library(zaminfluence)
Loading required package: reticulate
Loading required package: latex2exp
Loading required package: broom
> devtools::load_all("zaminfluence")
Loading zaminfluence
Attaching package: ‘testthat’
The following object is masked from ‘package:dplyr’:
matches
The following object is masked from ‘package:purrr’:
is_null
The following object is masked from ‘package:tidyr’:
matches
The following object is masked from ‘package:devtools’:
test_file
> py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
> PyPrint <- function(py_expr) {
+ reticulate::py_run_string(sprintf("foo = %s", py_expr))
+ print(py_main$foo)
+ }
> reticulate::py_run_string("import pkg_resources")
> reticulate::py_run_string("import sys")
> PyPrint("sys.version")
[1] "3.8.5 (default, Jul 28 2020, 12:59:40) \n[GCC 9.3.0]"
> PyPrint("vittles.__version__")
[1] "0.173"
> PyPrint("paragami.__version__")
[1] "0.42"
> PyPrint("pkg_resources.get_distribution(\"autograd\")")
autograd 1.3
> source(file.path(base_dir, "zaminfluence/tests/testthat/utils.R"))
Attaching package: ‘gridExtra’
The following object is masked from ‘package:dplyr’:
combine
> set.seed(42)
> x_dim <- 3
> beta_true <- runif(x_dim)
> df <- GenerateRegressionData(10, beta_true, num_groups = NULL)
> x_names <- sprintf("x%d", 1:x_dim)
> reg_form <- formula(sprintf("y ~ %s - 1", paste(x_names,
+ collapse = " + ")))
> reg_fit <- lm(data = df, formula = reg_form, x = TRUE,
+ y = TRUE)
> py_main <- SetPythonRegressionVariables(reg_fit)
> CheckPyMain <- function(py_main_check) {
+ print(py_main_check$x - df[, x_names] %>% as.matrix())
+ print(py_main_check$y - df$y %>% as.nume .... [TRUNCATED]
> CheckPyMain(py_main)
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
> reticulate::py_run_string("\nfrom copy import deepcopy\nx_copy = deepcopy(x)\ny_copy = deepcopy(y)\nw0_copy = deepcopy(w0)\nbeta_copy = deepcopy(bet ..." ... [TRUNCATED]
> reticulate::py_run_string("\nreg_obj = regsens_rgiordandev.reg_obj\nget_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix\nsens_ ..." ... [TRUNCATED]
> PyPrint("get_se(w0)")
get_se computed betahat = [ 1.90362055 -0.19357143 -0.30374769]
get_se x error = [[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
get_se y error = [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
get_se w difference = [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
get_se se_group = None
get_se se_cov = [[ 0.97767675 -0.46919679 2.4775263 ]
[ 1.00699179 -0.31824127 -0.00303038]
[ 0.06094292 -2.56488699 -0.04013961]]
/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/autograd-1.3-py3.8.egg/autograd/tracer.py:48: RuntimeWarning: invalid value encountered in sqrt
return f_raw(*args, **kwargs)
[1] 0.9887754 NaN NaN
> PyPrint("get_se(w0)")
get_se computed betahat = [ 1.90362055 -0.19357143 -0.30374769]
get_se x error = [[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
get_se y error = [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
get_se w difference = [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
get_se se_group = None
get_se se_cov = [[ 0.97767675 -0.46919679 2.4775263 ]
[ 1.00699179 -0.31824127 -0.00303038]
[ 0.06094292 -2.56488699 -0.04013961]]
[1] 0.9887754 NaN NaN
And don't worry if you need me to run more stuff, I've been there trying to debug what was happening with code I wrote in other people's config, so I will try to help with what I can.
Thanks again for your patience. I do wish I could reproduce your error on my side so I could iterate more quickly. Unfortunately my guesses about where things are going wrong are all coming up blank.
Starting after CheckPyMain(py_main)
in the previous script I sent, could you run the attached code? In a sense I'm hoping this will fail in the same way so I don't have to go and have you debug the Python library.
############# Starting after CheckPyMain(py_main):
# Make deep copies of the correct values
reticulate::py_run_string("
from copy import deepcopy
x_copy = deepcopy(x)
y_copy = deepcopy(y)
w0_copy = deepcopy(w0)
beta_copy = deepcopy(beta)
")
reticulate::py_run_string("
reg_obj = regsens_rgiordandev.reg_obj
get_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix
sens_reg_obj = lambda beta, w: reg_obj(beta, y=y, x=x, w=w)
obs_w_sens = vittles.HyperparameterSensitivityLinearApproximation(
objective_fun=sens_reg_obj,
opt_par_value=beta,
hyper_par_value=w0,
validate_optimum=True,
grad_tol=1e-08)
get_betahat = obs_w_sens.get_opt_par_function()
def get_se(w):
betahat = get_betahat(w)
print('get_se computed betahat = {}'.format(betahat))
print('get_se x error = {}'.format(x_copy - x))
print('get_se y error = {}'.format(y_copy - y))
print('get_se w difference = {}'.format(w0_copy - w))
print('get_se se_group = {}'.format(se_group))
se_cov = get_standard_error_matrix(betahat, y, x, w=w, se_group=se_group)
print('get_se se_cov = {}'.format(se_cov))
return np.sqrt(np.diag(se_cov))
# Copied from regsens_rgiordandev.get_standard_error_matrix with debugging
def get_standard_error_matrix_local(betahat, y, x, w, se_group=None):
assert(se_group is None)
resid = y - x @ betahat
num_obs = len(y)
xtx_bar = np.einsum('ni,nj,n->ij', x, x, w) / num_obs
sigma2hat = np.sum(w * (resid ** 2)) / (num_obs - len(betahat))
xtx_inv = np.linalg.inv(xtx_bar)
se2 = sigma2hat * xtx_inv / num_obs
print('get_se_local betahat = {}'.format(betahat))
print('get_se_local y = {}'.format(y))
print('get_se_local x = {}'.format(x))
print('get_se_local w = {}'.format(w))
print('get_se_local resid = {}'.format(resid))
print('get_se_local xtx_bar = {}'.format(xtx_bar))
print('get_se_local sigma2hat = {}'.format(sigma2hat))
return se2
")
PyPrint("get_se(w0)")
reticulate::py_run_string("betahat = get_betahat(w0)")
PyPrint("get_standard_error_matrix_local(betahat, y, x, w=w0, se_group=se_group)")
OK, replaced the lines you suggested with the new code, and here is the result:
$ Rscript -e 'source("test4.R", echo=TRUE)'
> base_dir <- "/home/jesus/devel/others/zaminfluence"
> setwd(base_dir)
> library(devtools)
Loading required package: usethis
> library(tidyverse)
── Attaching packages ───────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.3.0 ──
✔ ggplot2 3.3.2 ✔ purrr 0.3.4
✔ tibble 3.0.4 ✔ dplyr 1.0.2
✔ tidyr 1.1.2 ✔ stringr 1.4.0
✔ readr 1.4.0 ✔ forcats 0.5.0
── Conflicts ──────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
> library(zaminfluence)
Loading required package: reticulate
Loading required package: latex2exp
Loading required package: broom
> devtools::load_all("zaminfluence")
Loading zaminfluence
Attaching package: ‘testthat’
The following object is masked from ‘package:dplyr’:
matches
The following object is masked from ‘package:purrr’:
is_null
The following object is masked from ‘package:tidyr’:
matches
The following object is masked from ‘package:devtools’:
test_file
> py_main <- InitializePython(file.path(base_dir, "venv/bin/python"))
> PyPrint <- function(py_expr) {
+ reticulate::py_run_string(sprintf("foo = %s", py_expr))
+ print(py_main$foo)
+ }
> reticulate::py_run_string("import pkg_resources")
> reticulate::py_run_string("import sys")
> PyPrint("sys.version")
[1] "3.8.5 (default, Jul 28 2020, 12:59:40) \n[GCC 9.3.0]"
> PyPrint("vittles.__version__")
[1] "0.173"
> PyPrint("paragami.__version__")
[1] "0.42"
> PyPrint("pkg_resources.get_distribution(\"autograd\")")
autograd 1.3
> source(file.path(base_dir, "zaminfluence/tests/testthat/utils.R"))
Attaching package: ‘gridExtra’
The following object is masked from ‘package:dplyr’:
combine
> set.seed(42)
> x_dim <- 3
> beta_true <- runif(x_dim)
> df <- GenerateRegressionData(10, beta_true, num_groups = NULL)
> x_names <- sprintf("x%d", 1:x_dim)
> reg_form <- formula(sprintf("y ~ %s - 1", paste(x_names,
+ collapse = " + ")))
> reg_fit <- lm(data = df, formula = reg_form, x = TRUE,
+ y = TRUE)
> py_main <- SetPythonRegressionVariables(reg_fit)
> CheckPyMain <- function(py_main_check) {
+ print(py_main_check$x - df[, x_names] %>% as.matrix())
+ print(py_main_check$y - df$y %>% as.nume .... [TRUNCATED]
> CheckPyMain(py_main)
x1 x2 x3
[1,] 0 0 0
[2,] 0 0 0
[3,] 0 0 0
[4,] 0 0 0
[5,] 0 0 0
[6,] 0 0 0
[7,] 0 0 0
[8,] 0 0 0
[9,] 0 0 0
[10,] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
[1] 0 0 0
[1] 0 0 0 0 0 0 0 0 0 0
> cat("***** 2020-12-15 CODE *****\n\n")
***** 2020-12-15 CODE *****
> reticulate::py_run_string("\nfrom copy import deepcopy\nx_copy = deepcopy(x)\ny_copy = deepcopy(y)\nw0_copy = deepcopy(w0)\nbeta_copy = deepcopy(bet ..." ... [TRUNCATED]
> reticulate::py_run_string("\nreg_obj = regsens_rgiordandev.reg_obj\nget_standard_error_matrix = regsens_rgiordandev.get_standard_error_matrix\nsens_ ..." ... [TRUNCATED]
> PyPrint("get_se(w0)")
get_se computed betahat = [ 1.90362055 -0.19357143 -0.30374769]
get_se x error = [[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
get_se y error = [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
get_se w difference = [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
get_se se_group = None
get_se se_cov = [[ 0.97767675 -0.46919679 2.4775263 ]
[ 1.00699179 -0.31824127 -0.00303038]
[ 0.06094292 -2.56488699 -0.04013961]]
/home/jesus/devel/others/zaminfluence/venv/lib/python3.8/site-packages/autograd-1.3-py3.8.egg/autograd/tracer.py:48: RuntimeWarning: invalid value encountered in sqrt
return f_raw(*args, **kwargs)
[1] 0.9887754 NaN NaN
> reticulate::py_run_string("betahat = get_betahat(w0)")
> PyPrint("get_standard_error_matrix_local(betahat, y, x, w=w0, se_group=se_group)")
get_se_local betahat = [ 1.90362055 -0.19357143 -0.30374769]
get_se_local y = [ 0.4537174 0.71119636 -0.61011909 0.68769675 -0.9676639 1.81117498
1.83409516 0.53692678 0.02391491 0.20022749]
get_se_local x = [[ 0.19683489 -0.32661248 0.34076902]
[ 0.00813278 -0.11974848 -0.52346165]
[-0.11451679 0.35797322 -0.09168743]
[ 0.10297558 0.39618512 -0.21569574]
[-0.49894614 -0.46455395 0.29983892]
[ 0.02337955 -0.10704423 -0.15892958]
[ 0.07145205 -0.02170856 0.23010505]
[-0.17587096 0.32199008 0.13169641]
[ 0.08549952 -0.44333114 0.20515593]
[ 0.30105951 0.40685042 -0.21779093]]
get_se_local w = [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
get_se_local resid = [ 0.11930341 0.51353448 -0.35067903 0.50284335 -0.01670877 1.69767398
1.76376929 0.97404891 -0.16234433 -0.36027446]
get_se_local xtx_bar = [[ 0.0446005 0.01902273 -0.01569602]
[ 0.01902273 0.10996104 -0.04313223]
[-0.01569602 -0.04313223 0.0720043 ]]
get_se_local sigma2hat = 1.107423007435789
[,1] [,2] [,3]
[1,] 0.97767675 -0.4691968 2.477526302
[2,] 1.00699179 -0.3182413 -0.003030384
[3,] 0.06094292 -2.5648870 -0.040139607
What is common is the warning given by the autograd
package. Will try to reinstall just that one and see if that helps (or not)
OK, recloned the repo into a new path, and rerun the procedure, now they are some new errors on the python part (when creating the modules wheels), and the R test seems to pass although there are some steps skipped.
Here is the whole session log
$ git clone https://github.com/rgiordan/zaminfluence.git
Cloning into 'zaminfluence'...
remote: Enumerating objects: 286, done.
remote: Counting objects: 100% (286/286), done.
remote: Compressing objects: 100% (167/167), done.
remote: Total 286 (delta 135), reused 252 (delta 108), pack-reused 0
Receiving objects: 100% (286/286), 89.87 KiB | 76.00 KiB/s, done.
Resolving deltas: 100% (135/135), done.
$ cd zaminfluence
$ export REPO=$(pwd)
$ python3 -m venv venv
$ source venv/bin/activate
$ python3 -m pip install -e regression_sensitivity
Obtaining file:///home/jesus/devel/others/zaminfluence/regression_sensitivity
Collecting autograd>=1.3
Using cached autograd-1.3.tar.gz (38 kB)
Collecting ipykernel
Using cached ipykernel-5.4.2-py3-none-any.whl (119 kB)
Collecting numpy
Using cached numpy-1.19.4-cp38-cp38-manylinux2010_x86_64.whl (14.5 MB)
Collecting paragami>=0.42
Using cached paragami-0.42-py3-none-any.whl (41 kB)
Collecting vittles>=0.171
Using cached vittles-0.173-py3-none-any.whl (26 kB)
Collecting future>=0.15.2
Using cached future-0.18.2.tar.gz (829 kB)
Collecting traitlets>=4.1.0
Using cached traitlets-5.0.5-py3-none-any.whl (100 kB)
Collecting tornado>=4.2
Using cached tornado-6.1-cp38-cp38-manylinux2010_x86_64.whl (427 kB)
Collecting ipython>=5.0.0
Using cached ipython-7.19.0-py3-none-any.whl (784 kB)
Collecting jupyter-client
Using cached jupyter_client-6.1.7-py3-none-any.whl (108 kB)
Collecting pytest
Using cached pytest-6.2.1-py3-none-any.whl (279 kB)
Collecting scipy
Using cached scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl (25.8 MB)
Collecting ipython-genutils
Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
Using cached prompt_toolkit-3.0.8-py3-none-any.whl (355 kB)
Collecting pexpect>4.3; sys_platform != "win32"
Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Requirement already satisfied: setuptools>=18.5 in ./venv/lib/python3.8/site-packages (from ipython>=5.0.0->ipykernel->regsens-rgiordandev==0.1) (44.0.0)
Collecting pygments
Using cached Pygments-2.7.3-py3-none-any.whl (950 kB)
Collecting pickleshare
Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting decorator
Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting backcall
Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting jedi>=0.10
Using cached jedi-0.17.2-py2.py3-none-any.whl (1.4 MB)
Collecting python-dateutil>=2.1
Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting pyzmq>=13
Using cached pyzmq-20.0.0-cp38-cp38-manylinux1_x86_64.whl (1.1 MB)
Collecting jupyter-core>=4.6.0
Using cached jupyter_core-4.7.0-py3-none-any.whl (82 kB)
Collecting pluggy<1.0.0a1,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting toml
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting packaging
Using cached packaging-20.8-py2.py3-none-any.whl (39 kB)
Collecting attrs>=19.2.0
Using cached attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting py>=1.8.2
Using cached py-1.10.0-py2.py3-none-any.whl (97 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting ptyprocess>=0.5
Using cached ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
Collecting parso<0.8.0,>=0.7.0
Using cached parso-0.7.1-py2.py3-none-any.whl (109 kB)
Collecting six>=1.5
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting pyparsing>=2.0.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Building wheels for collected packages: autograd, future
Building wheel for autograd (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/jesus/devel/others/zaminfluence/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ir6tp3hl/autograd/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ir6tp3hl/autograd/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-1et65h4u
cwd: /tmp/pip-install-ir6tp3hl/autograd/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for autograd
Running setup.py clean for autograd
Building wheel for future (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/jesus/devel/others/zaminfluence/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ir6tp3hl/future/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ir6tp3hl/future/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-4md3o1ra
cwd: /tmp/pip-install-ir6tp3hl/future/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for future
Running setup.py clean for future
Failed to build autograd future
Installing collected packages: future, numpy, autograd, ipython-genutils, traitlets, tornado, wcwidth, prompt-toolkit, ptyprocess, pexpect, pygments, pickleshare, decorator, backcall, parso, jedi, ipython, six, python-dateutil, pyzmq, jupyter-core, jupyter-client, ipykernel, pluggy, toml, iniconfig, pyparsing, packaging, attrs, py, pytest, scipy, paragami, vittles, regsens-rgiordandev
Running setup.py install for future ... done
Running setup.py install for autograd ... done
Running setup.py develop for regsens-rgiordandev
Successfully installed attrs-20.3.0 autograd-1.3 backcall-0.2.0 decorator-4.4.2 future-0.18.2 iniconfig-1.1.1 ipykernel-5.4.2 ipython-7.19.0 ipython-genutils-0.2.0 jedi-0.17.2 jupyter-client-6.1.7 jupyter-core-4.7.0 numpy-1.19.4 packaging-20.8 paragami-0.42 parso-0.7.1 pexpect-4.8.0 pickleshare-0.7.5 pluggy-0.13.1 prompt-toolkit-3.0.8 ptyprocess-0.6.0 py-1.10.0 pygments-2.7.3 pyparsing-2.4.7 pytest-6.2.1 python-dateutil-2.8.1 pyzmq-20.0.0 regsens-rgiordandev scipy-1.5.4 six-1.15.0 toml-0.10.2 tornado-6.1 traitlets-5.0.5 vittles-0.173 wcwidth-0.2.5
$ cd regression_sensitivity/
$ python3 -m pytest
============================================================== test session starts ===============================================================
platform linux -- Python 3.8.5, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /home/jesus/devel/others/zaminfluence/regression_sensitivity
collected 2 items
regsens_rgiordandev/tests/test_basics.py .. [100%]
=============================================================== 2 passed in 0.84s ================================================================
$ cd ../zaminfluence/tests/
$ ./testthat.R
Loading required package: reticulate
Loading required package: dplyr
Attaching package: ‘dplyr’
The following object is masked from ‘package:testthat’:
matches
The following objects are masked from ‘package:stats’:
filter, lag
The following objects are masked from ‘package:base’:
intersect, setdiff, setequal, union
Loading required package: ggplot2
Loading required package: latex2exp
Loading required package: stringr
Loading required package: broom
── Skip (test_end_to_end.R:16:1): python runs ────────────────────────────────────────────────────────────────────────────────────────────────────
Reason: empty test
── Skip (test_end_to_end.R:91:1): plotting works ─────────────────────────────────────────────────────────────────────────────────────────────────
Reason: empty test
── Warning (test_end_to_end.R:276:11): aggregated pairing works ──────────────────────────────────────────────────────────────────────────────────
Unknown or uninitialised column: `row_0`.
── Warning (test_end_to_end.R:277:11): aggregated pairing works ──────────────────────────────────────────────────────────────────────────────────
Unknown or uninitialised column: `row_1`.
── Warning (test_end_to_end.R:276:11): aggregated pairing works ──────────────────────────────────────────────────────────────────────────────────
Unknown or uninitialised column: `row_0`.
── Warning (test_end_to_end.R:277:11): aggregated pairing works ──────────────────────────────────────────────────────────────────────────────────
Unknown or uninitialised column: `row_1`.
── Warning (test_end_to_end.R:276:11): aggregated pairing works ──────────────────────────────────────────────────────────────────────────────────
Unknown or uninitialised column: `row_0`.
── Warning (test_end_to_end.R:277:11): aggregated pairing works ──────────────────────────────────────────────────────────────────────────────────
Unknown or uninitialised column: `row_1`.
── Warning (test_end_to_end.R:276:11): aggregated pairing works ──────────────────────────────────────────────────────────────────────────────────
Unknown or uninitialised column: `row_0`.
── Warning (test_end_to_end.R:277:11): aggregated pairing works ──────────────────────────────────────────────────────────────────────────────────
Unknown or uninitialised column: `row_1`.
── Skipped tests ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
● empty test (2)
══ testthat results ═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Warning (test_end_to_end.R:276:11): aggregated pairing works
Warning (test_end_to_end.R:277:11): aggregated pairing works
Warning (test_end_to_end.R:276:11): aggregated pairing works
Warning (test_end_to_end.R:277:11): aggregated pairing works
Warning (test_end_to_end.R:276:11): aggregated pairing works
Warning (test_end_to_end.R:277:11): aggregated pairing works
Warning (test_end_to_end.R:276:11): aggregated pairing works
Warning (test_end_to_end.R:277:11): aggregated pairing works
[ FAIL 0 | WARN 8 | SKIP 2 | PASS 85 ]
What is more the simple_examples.R
now runs w/o a hitch:
$ Rscript simple_examples.R
Attaching package: ‘dplyr’
The following objects are masked from ‘package:stats’:
filter, lag
The following objects are masked from ‘package:base’:
intersect, setdiff, setequal, union
Attaching package: ‘gridExtra’
The following object is masked from ‘package:dplyr’:
combine
Loading required package: reticulate
Loading required package: latex2exp
Loading required package: stringr
Loading required package: broom
Loading required package: car
Loading required package: carData
Attaching package: ‘car’
The following object is masked from ‘package:dplyr’:
recode
Loading required package: lmtest
Loading required package: zoo
Attaching package: ‘zoo’
The following objects are masked from ‘package:base’:
as.Date, as.Date.numeric
Loading required package: survival
change beta beta_pzse beta_mzse prop_removed
1 original 0.9183526 0.9865258 0.8501794 NA
2 sign -0.3595436 -0.2999995 -0.4190876 0.1875
3 sign and significance -0.5228577 -0.4641184 -0.5815969 0.2178
4 significance -0.2223906 -0.1616786 -0.2831025 0.1622
change beta beta_pzse beta_mzse prop_removed
1 original 0.48217551 0.53516292 0.4291881 NA
2 sign -0.17895735 -0.11675094 -0.2411638 0.0907
3 sign and significance -0.31502621 -0.24966329 -0.3803891 0.1129
4 significance -0.07877113 -0.01742793 -0.1401143 0.0725
change beta beta_pzse beta_mzse prop_removed
1 original 0.092414667 0.160886232 0.0239431030 NA
2 sign -0.001881734 0.065654834 -0.0694183030 0.0074
3 sign and significance -0.072743032 -0.005691437 -0.1397946283 0.0142
4 significance 0.067929299 0.136096415 -0.0002378169 0.0016
[1] 10000
# A tibble: 6 x 8
arm group se_grad beta_grad beta_pzse_grad beta_mzse_grad obs_per_row
<int> <int> <dbl> <dbl> <dbl> <dbl> <dbl>
1 0 1 4.33e-6 3.94e-4 0.000402 0.000385 6
2 0 2 -5.39e-6 -2.37e-4 -0.000248 -0.000227 7
3 0 3 -5.38e-6 -2.23e-4 -0.000234 -0.000213 4
4 0 4 -5.87e-6 -6.29e-5 -0.0000744 -0.0000514 3
5 0 5 -5.48e-6 -7.49e-4 -0.000760 -0.000739 7
6 0 6 -6.37e-6 2.35e-4 0.000222 0.000247 4
# … with 1 more variable: grouped_row <int>
# A tibble: 30 x 4
arm group beta_grad grouped_row
<int> <int> <dbl> <int>
1 0 46 0.00224 46
2 0 54 0.000213 54
3 0 152 0.00160 152
4 0 161 0.00157 161
5 0 179 0.00249 179
6 0 241 0.00213 241
7 0 247 0.00106 247
8 0 338 0.00180 337
9 0 361 0.00240 360
10 0 495 0.00200 494
# … with 20 more rows
0 1
15 15
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
205 199 197 172 203 209 193 191 187 203 187 205 182 195 209 178 206 195 197 191
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
193 192 222 214 226 189 194 174 197 229 206 250 193 201 204 210 218 198 208 177
40 41 42 43 44 45 46 47 48 49
202 181 184 224 218 189 212 199 179 213
vcovCL: 0.047676
zaminfluence: 0.047676
change beta beta_pzse beta_mzse prop_removed
1 original 0.9073391 1.00078136 0.8138968 NA
2 sign -0.1733668 0.08850384 -0.4352375 0.0893
3 sign and significance -0.4166331 -0.11799088 -0.7152752 0.1226
4 significance 0.0441852 0.31560318 -0.2272328 0.0756
:man_shrugging: ???
Seems like a bit of a Heisenbug :-)
So strange! Though arguably no stranger than the fact that you were seeing the errors you saw in the first place.
Incidentally, I see you have some wheel errors on your Python install. Could you please put my mind at ease, and try to do a clean install & test run one more time, this time upgrading pip
and installing wheel
before installing? (Some venv
versions seem to have wheel
, some don't. These steps are also listed in the new "troubleshooting" section of the install instructions.)
python3 -m pip install --upgrade pip
python3 -m pip install wheel
python3 -m pip install -e regression_sensitivity
The skipped tests and warnings are normal, I see them too. (I'll make an issue to remind myself to clean those up, actually.)
OK. I removed the pip cache (~/.cache/pip/), and then redid everything, including the python lines above:
jesus@aleph0:~/tmp/test$ git clone https://github.com/rgiordan/zaminfluence.git
Cloning into 'zaminfluence'...
remote: Enumerating objects: 286, done.
remote: Counting objects: 100% (286/286), done.
remote: Compressing objects: 100% (167/167), done.
remote: Total 286 (delta 135), reused 252 (delta 108), pack-reused 0
Receiving objects: 100% (286/286), 89.87 KiB | 216.00 KiB/s, done.
Resolving deltas: 100% (135/135), done.
jesus@aleph0:~/tmp/test$ cd zaminfluence/
✔ ~/tmp/test/zaminfluence [master {origin/master}|✔]
23:51 $ export REPO=$(pwd)
✔ ~/tmp/test/zaminfluence [master {origin/master}|✔]
23:51 $
✔ ~/tmp/test/zaminfluence [master {origin/master}|✔]
23:51 $ python3 -m venv venv
✔ ~/tmp/test/zaminfluence [master {origin/master}|✔]
23:51 $ source venv/bin/activate
(venv) ✔ ~/tmp/test/zaminfluence [master {origin/master}|✔]
23:51 $ python3 -m pip install --upgrade pip
Collecting pip
Downloading pip-20.3.3-py2.py3-none-any.whl (1.5 MB)
|████████████████████████████████| 1.5 MB 342 kB/s
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.0.2
Uninstalling pip-20.0.2:
Successfully uninstalled pip-20.0.2
Successfully installed pip-20.3.3
(venv) ✔ ~/tmp/test/zaminfluence [master {origin/master}|✔]
23:52 $ python3 -m pip install wheel
Collecting wheel
Downloading wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel
Successfully installed wheel-0.36.2
(venv) ✔ ~/tmp/test/zaminfluence [master {origin/master}|✔]
23:52 $ python3 -m pip install -e regression_sensitivity
Obtaining file:///home/jesus/tmp/test/zaminfluence/regression_sensitivity
Collecting autograd>=1.3
Downloading autograd-1.3.tar.gz (38 kB)
Collecting future>=0.15.2
Downloading future-0.18.2.tar.gz (829 kB)
|████████████████████████████████| 829 kB 1.0 MB/s
Collecting numpy
Downloading numpy-1.19.4-cp38-cp38-manylinux2010_x86_64.whl (14.5 MB)
|████████████████████████████████| 14.5 MB 241 kB/s
Collecting paragami>=0.42
Downloading paragami-0.42-py3-none-any.whl (41 kB)
|████████████████████████████████| 41 kB 6.3 kB/s
Collecting vittles>=0.171
Downloading vittles-0.173-py3-none-any.whl (26 kB)
Collecting ipykernel
Downloading ipykernel-5.4.2-py3-none-any.whl (119 kB)
|████████████████████████████████| 119 kB 1.2 MB/s
Collecting ipython>=5.0.0
Downloading ipython-7.19.0-py3-none-any.whl (784 kB)
|████████████████████████████████| 784 kB 1.1 MB/s
Requirement already satisfied: setuptools>=18.5 in ./venv/lib/python3.8/site-packages (from ipython>=5.0.0->ipykernel->regsens-rgiordandev==0.1) (44.0.0)
Collecting jedi>=0.10
Downloading jedi-0.17.2-py2.py3-none-any.whl (1.4 MB)
|████████████████████████████████| 1.4 MB 986 kB/s
Collecting parso<0.8.0,>=0.7.0
Downloading parso-0.7.1-py2.py3-none-any.whl (109 kB)
|████████████████████████████████| 109 kB 806 kB/s
Collecting pexpect>4.3
Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
|████████████████████████████████| 59 kB 216 kB/s
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
Downloading prompt_toolkit-3.0.8-py3-none-any.whl (355 kB)
|████████████████████████████████| 355 kB 1.2 MB/s
Collecting ptyprocess>=0.5
Downloading ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
Collecting tornado>=4.2
Downloading tornado-6.1-cp38-cp38-manylinux2010_x86_64.whl (427 kB)
|████████████████████████████████| 427 kB 1.7 MB/s
Collecting traitlets>=4.1.0
Downloading traitlets-5.0.5-py3-none-any.whl (100 kB)
|████████████████████████████████| 100 kB 465 kB/s
Collecting backcall
Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting decorator
Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting ipython-genutils
Downloading ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Collecting jupyter-client
Downloading jupyter_client-6.1.7-py3-none-any.whl (108 kB)
|████████████████████████████████| 108 kB 983 kB/s
Collecting jupyter-core>=4.6.0
Downloading jupyter_core-4.7.0-py3-none-any.whl (82 kB)
|████████████████████████████████| 82 kB 61 kB/s
Collecting python-dateutil>=2.1
Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
|████████████████████████████████| 227 kB 1.5 MB/s
Collecting pyzmq>=13
Downloading pyzmq-20.0.0-cp38-cp38-manylinux1_x86_64.whl (1.1 MB)
|████████████████████████████████| 1.1 MB 454 kB/s
Collecting six>=1.5
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting pickleshare
Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting pygments
Downloading Pygments-2.7.3-py3-none-any.whl (950 kB)
|████████████████████████████████| 950 kB 2.1 MB/s
Collecting pytest
Downloading pytest-6.2.1-py3-none-any.whl (279 kB)
|████████████████████████████████| 279 kB 1.1 MB/s
Collecting attrs>=19.2.0
Downloading attrs-20.3.0-py2.py3-none-any.whl (49 kB)
|████████████████████████████████| 49 kB 292 kB/s
Collecting pluggy<1.0.0a1,>=0.12
Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting py>=1.8.2
Downloading py-1.10.0-py2.py3-none-any.whl (97 kB)
|████████████████████████████████| 97 kB 374 kB/s
Collecting iniconfig
Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting packaging
Downloading packaging-20.8-py2.py3-none-any.whl (39 kB)
Collecting pyparsing>=2.0.2
Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
|████████████████████████████████| 67 kB 265 kB/s
Collecting scipy
Downloading scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl (25.8 MB)
|████████████████████████████████| 25.8 MB 223 kB/s
Collecting toml
Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting wcwidth
Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Building wheels for collected packages: autograd, future
Building wheel for autograd (setup.py) ... done
Created wheel for autograd: filename=autograd-1.3-py3-none-any.whl size=47988 sha256=de90e5fb6e14d77108c41e9893d5452a2522cfb276a22fc096ee117e2647a357
Stored in directory: /home/jesus/.cache/pip/wheels/85/f5/d2/3ef47d3a836b17620bf41647222825b065245862d12aa62885
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491059 sha256=0abba752cfe207e2ee3fec1ae47fb97faf6540e9b7c919cd181ed067aac63968
Stored in directory: /home/jesus/.cache/pip/wheels/8e/70/28/3d6ccd6e315f65f245da085482a2e1c7d14b90b30f239e2cf4
Successfully built autograd future
Installing collected packages: pyparsing, ipython-genutils, wcwidth, traitlets, toml, six, py, ptyprocess, pluggy, parso, packaging, numpy, iniconfig, future, attrs, tornado, scipy, pyzmq, python-dateutil, pytest, pygments, prompt-toolkit, pickleshare, pexpect, jupyter-core, jedi, decorator, backcall, autograd, paragami, jupyter-client, ipython, vittles, ipykernel, regsens-rgiordandev
Running setup.py develop for regsens-rgiordandev
Successfully installed attrs-20.3.0 autograd-1.3 backcall-0.2.0 decorator-4.4.2 future-0.18.2 iniconfig-1.1.1 ipykernel-5.4.2 ipython-7.19.0 ipython-genutils-0.2.0 jedi-0.17.2 jupyter-client-6.1.7 jupyter-core-4.7.0 numpy-1.19.4 packaging-20.8 paragami-0.42 parso-0.7.1 pexpect-4.8.0 pickleshare-0.7.5 pluggy-0.13.1 prompt-toolkit-3.0.8 ptyprocess-0.6.0 py-1.10.0 pygments-2.7.3 pyparsing-2.4.7 pytest-6.2.1 python-dateutil-2.8.1 pyzmq-20.0.0 regsens-rgiordandev scipy-1.5.4 six-1.15.0 toml-0.10.2 tornado-6.1 traitlets-5.0.5 vittles-0.173 wcwidth-0.2.5
(venv) ✔ ~/tmp/test/zaminfluence [master {origin/master}|✔]
23:54 $ cd regression_sensitivity/
(venv) ✔ ~/tmp/test/zaminfluence/regression_sensitivity [master {origin/master}|✔]
23:56 $ python3 -m pytest
============================================================== test session starts ===============================================================
platform linux -- Python 3.8.5, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /home/jesus/tmp/test/zaminfluence/regression_sensitivity
collected 2 items
regsens_rgiordandev/tests/test_basics.py .. [100%]
=============================================================== 2 passed in 1.35s ================================================================
(venv) ✔ ~/tmp/test/zaminfluence/regression_sensitivity [master {origin/master}|✔]
23:56 $ cd ../zaminfluence/tests/
(venv) ✔ ~/tmp/test/zaminfluence/zaminfluence/tests [master {origin/master}|✔]
23:56 $ ./testthat.R
Loading required package: reticulate
Loading required package: dplyr
Attaching package: ‘dplyr’
The following object is masked from ‘package:testthat’:
matches
The following objects are masked from ‘package:stats’:
filter, lag
The following objects are masked from ‘package:base’:
intersect, setdiff, setequal, union
Loading required package: ggplot2
Loading required package: latex2exp
Loading required package: stringr
Loading required package: broom
══ Skipped tests ═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
● empty test (2)
══ Warnings ══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
── Warning (test_end_to_end.R:276:11): aggregated pairing works ─────────────────────
Unknown or uninitialised column: `row_0`.
Backtrace:
1. df[infl_df$row_0, ] test_end_to_end.R:276:10
4. tibble:::`$.tbl_df`(infl_df, row_0)
── Warning (test_end_to_end.R:277:11): aggregated pairing works ─────────────────────
Unknown or uninitialised column: `row_1`.
Backtrace:
1. df[infl_df$row_1, ] test_end_to_end.R:277:10
4. tibble:::`$.tbl_df`(infl_df, row_1)
── Warning (test_end_to_end.R:276:11): aggregated pairing works ─────────────────────
Unknown or uninitialised column: `row_0`.
Backtrace:
1. df[infl_df$row_0, ] test_end_to_end.R:276:10
4. tibble:::`$.tbl_df`(infl_df, row_0)
── Warning (test_end_to_end.R:277:11): aggregated pairing works ─────────────────────
Unknown or uninitialised column: `row_1`.
Backtrace:
1. df[infl_df$row_1, ] test_end_to_end.R:277:10
4. tibble:::`$.tbl_df`(infl_df, row_1)
── Warning (test_end_to_end.R:276:11): aggregated pairing works ─────────────────────
Unknown or uninitialised column: `row_0`.
Backtrace:
1. df[infl_df$row_0, ] test_end_to_end.R:276:10
4. tibble:::`$.tbl_df`(infl_df, row_0)
── Warning (test_end_to_end.R:277:11): aggregated pairing works ─────────────────────
Unknown or uninitialised column: `row_1`.
Backtrace:
1. df[infl_df$row_1, ] test_end_to_end.R:277:10
4. tibble:::`$.tbl_df`(infl_df, row_1)
── Warning (test_end_to_end.R:276:11): aggregated pairing works ─────────────────────
Unknown or uninitialised column: `row_0`.
Backtrace:
1. df[infl_df$row_0, ] test_end_to_end.R:276:10
4. tibble:::`$.tbl_df`(infl_df, row_0)
── Warning (test_end_to_end.R:277:11): aggregated pairing works ─────────────────────
Unknown or uninitialised column: `row_1`.
Backtrace:
1. df[infl_df$row_1, ] test_end_to_end.R:277:10
4. tibble:::`$.tbl_df`(infl_df, row_1)
[ FAIL 0 | WARN 8 | SKIP 2 | PASS 85 ]
Wheel errors are gone, and all seems to be working. Now I have to figure out how to use this code effectively in my analysis :-)
Thanks for your help -- I think we can close this now.
I'm glad it worked out in the end! Please feel quite free to post another issue if you have more problems or questions. I'd really like this package to be useful, and I'll be happy to get feedback or provide help.
OS: Ubuntu 20.04.1 LTS R: 4.0.3 Python: 3.8.5
Everything installed as per the README.md
Also, when testing the Python bits, they seem to work, but give a deprecation warning:
Finally, editing
examples/simple_examples.R
to get the correct env var, and running it gives a similarnot zero
error as the test scripts: