tests: cannot import name 'Simulator' from 'batchglm.api.models'

[rekado]

Hi,

I'm trying to package batchglm for GNU Guix, but I'm having problems running the tests.

I'm only have tensorflow 1.9.0, so I replaced "tf.compat.v1." with "tf." throughout. I run the tests with python batchglm/unit_test/run_all_tests.py after installing and putting the installed location on PYTHONPATH.

I see errors like this:

======================================================================
ERROR: test_compute_jacobians_norm (batchglm.unit_test.test_jacobians_glm_all.Test_Jacobians_GLM_NORM)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_jacobians_glm_all.py", line 176, in test_compute_jacobians_norm
    self._test_compute_jacobians(sparse=False)
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_jacobians_glm_all.py", line 152, in _test_compute_jacobians
    self.simulate()
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_jacobians_glm_all.py", line 29, in simulate
    from batchglm.api.models import Simulator
ImportError: cannot import name 'Simulator' from 'batchglm.api.models' (/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/api/models/__init__.py)

I'm not a Python person, so I don't know if there should be a batchglm/api/models/simulator.py file (there is not) that defines the Simulator class. All I see is class definitions in batchglm/models/glm_beta/simulator.py and similar files, but none under the batchglm.api namespace.

There are other errors like this:

======================================================================
ERROR: test_compute_jacobians_beta (batchglm.unit_test.test_jacobians_glm_all.Test_Jacobians_GLM_BETA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_jacobians_glm_all.py", line 187, in test_compute_jacobians_beta
    self._test_compute_jacobians(sparse=False)
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_jacobians_glm_all.py", line 152, in _test_compute_jacobians
    self.simulate()
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_jacobians_glm_all.py", line 36, in simulate
    sim = Simulator(num_observations=num_observations, num_features=4)
TypeError: Can't instantiate abstract class Simulator with abstract methods eta_loc_j

and

======================================================================
ERROR: test_full_nb (batchglm.unit_test.test_graph_glm_all.TestGraphGlmNb)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_graph_glm_all.py", line 261, in test_full_nb
    self._test_full(sparse=True)
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_graph_glm_all.py", line 235, in _test_full
    self._test_full_a_and_b(sparse=sparse)
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_graph_glm_all.py", line 186, in _test_full_a_and_b
    return self.basic_test(
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_graph_glm_all.py", line 177, in basic_test
    self.basic_test_one_algo(
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_graph_glm_all.py", line 148, in basic_test_one_algo
    estimator = _TestGraphGlmAllEstim(
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/unit_test/test_graph_glm_all.py", line 55, in __init__
    estimator = Estimator(
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/train/numpy/glm_nb/estimator.py", line 59, in __init__
    init_a, init_b, train_loc, train_scale = init_par(
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/models/glm_nb/utils.py", line 120, in init_par
    groupwise_means, init_a, rmsd_a = closedform_nb_glm_logmu(
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/models/glm_nb/utils.py", line 30, in closedform_nb_glm_logmu
    return closedform_glm_mean(
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/models/base_glm/utils.py", line 118, in closedform_glm_mean
    linker_groupwise_means, mu, rmsd, rank, s = groupwise_solve_lm(
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/utils/linalg.py", line 93, in groupwise_solve_lm
    params = apply_fun(inverse_idx)
  File "/gnu/store/sszy0n09kniyp1y64svnw35cjgb6s35c-python-batchglm-0.7.4/lib/python3.8/site-packages/batchglm/models/base_glm/utils.py", line 109, in apply_fun
    groupwise_means = np.asarray(np.vstack([
  File "/gnu/store/kd16r2qb0s7g9ixc6agn485nhbgakdzj-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/core/_asarray.py", line 85, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/gnu/store/6xigjqkygh99fv5plrpyj6hshn8s96r6-python-dask-2.14.0/lib/python3.8/site-packages/dask/array/core.py", line 1341, in __array__
    x = np.array(x)
  File "/gnu/store/lgz9l3ygxl2gmg6ymk2lpwxpb99i05h1-python-sparse-0.12.0/lib/python3.8/site-packages/sparse/_sparse_array.py", line 229, in __array__
    raise RuntimeError(
RuntimeError: Cannot convert a sparse array to dense automatically. To manually densify, use the todense method.

But I think I should first figure out why the import doesn't work.

I'd appreciate any help!

[rekado]

If there's anything else I can provide to help diagnose this issue, please let me know.

[rekado]

Ping :)

I would be happy to try fixing this, but I don't understand the error. Any help welcome!

[rekado]

One of the problems appears to be that instantiations of Simulator do not provide an implementation of the eta_loc_j method.

I have tried building earlier versions, but to no avail. I'll give up on this as it seems that batchglm is not maintained.

[picciama]

@rekado sorry for the late reply. We have just recently started working on these issues again after other projects needed our full attention. The mentioned problem with eta_loc_j is fixed on dev (#118). Also the import issues should be fixed since we removed tensorflow support entirely on dev (#120) while support for tf2 will be added after merging #88.