Closed ealsop closed 3 years ago
@niklaslang and me ran into the same issue, although forcing dense arrays did not work either.
I am also having this same issue and was wondering if there was any resolution/fix?
I initially was getting the same error described in issue #145 but I was using diffxpy v0.7.4. I installed diffxpy v0.7.3 instead, and am now getting the ValueError: All arrays must be instances of SparseArray.
error. I tried including %env SPARSE_AUTO_DENSIFY=1
but it did not fix the problem.
Not sure if relevant, but since I saw them listed in issue #145: I have batchglm v0.7.4, sparse v0.12.0 and dask v2021.4.0, which I believe are all the most current versions. Do any of these need to be downgraded as well?
I am also having this same issue and was wondering if there was any resolution/fix?
I initially was getting the same error described in issue #145 but I was using diffxpy v0.7.4. I installed diffxpy v0.7.3 instead, and am now getting the
ValueError: All arrays must be instances of SparseArray.
error. I tried including%env SPARSE_AUTO_DENSIFY=1
but it did not fix the problem.Not sure if relevant, but since I saw them listed in issue #145: I have batchglm v0.7.4, sparse v0.12.0 and dask v2021.4.0, which I believe are all the most current versions. Do any of these need to be downgraded as well?
Hey,
looks like the issue lies within some seemingly unrelated dependencies. Diffxpy should be more strict here.
Try to force:
dask==2021.4.0
sparse==0.9.1
This resolved it on our end.
I'll likely submit a PR to pin sparse tomorrow or so. Please report back whether this also works for you, @oligomyeggo
Thanks @Zethson! Forcing dask==2021.4.0
and sparse==0.9.1
worked for me for both diffxpy==0.7.3
and diffxpy==0.7.4
. Though now I am hitting a new error which also seems potentially related (?). I am following this tutorial (the first portion, for the standard Wald test), and now everything is working up until the test.plot_vs_ttest()
function, which is returning the following error:
ValueError: Performing a mixed sparse-dense operation that would result in a dense array. Please make sure that func(sparse_fill_values, ndarrays) is a constant array.
This occurred with both diffxpy==0.7.3
and diffxpy==0.7.4
.
Thanks @Zethson! Forcing
dask==2021.4.0
andsparse==0.9.1
worked for me for bothdiffxpy==0.7.3
anddiffxpy==0.7.4
. Though now I am hitting a new error which also seems potentially related (?). I am following this tutorial (the first portion, for the standard Wald test), and now everything is working up until thetest.plot_vs_ttest()
function, which is returning the following error:
ValueError: Performing a mixed sparse-dense operation that would result in a dense array. Please make sure that func(sparse_fill_values, ndarrays) is a constant array.
This occurred with both
diffxpy==0.7.3
anddiffxpy==0.7.4
.
I'd suggest to create a new issue for this.
Running de.test.wald using an un-normalized adata.X matrix with only 2 samples and ~5000 cells to test differential expression in scanpy. All libraries (numby, scanpy, diffxpy, sparse and dask) are updated the most recent versions.
test = de.test.wald(data=adata, formula_loc="~ 1 + Group", factor_loc_totest="Group")
make initally make this run at all I needed to include:
%env SPARSE_AUTO_DENSIFY=1
Or I would receive the error "RuntimeError: Cannot convert a sparse array to dense automatically. To manually densify, use the todense method." mentioned in other issues.
With this solved I am now running through several iterations:
training location model: False training scale model: True iter 0: ll=74052044.544684 iter 1: ll=74052044.544684, converged: 0.00% (loc: 100.00%, scale update: False), in 0.00sec iter 2: ll=53700205.003682, converged: 16.18% (loc: 16.18%, scale update: True), in 883.76sec iter 3: ll=53700205.003682, converged: 16.18% (loc: 100.00%, scale update: False), in 0.01sec iter 4: ll=53407203.078969, converged: 84.27% (loc: 84.27%, scale update: True), in 692.52sec iter 5: ll=53407203.078969, converged: 84.27% (loc: 100.00%, scale update: False), in 0.00sec iter 6: ll=53376103.605985, converged: 94.97% (loc: 94.97%, scale update: True), in 178.97sec iter 7: ll=53376103.605985, converged: 94.97% (loc: 100.00%, scale update: False), in 0.00sec Fitting 1842 dispersion models: (progress not available with multiprocessing)
But getting this error / trace back:
ValueError Traceback (most recent call last)