bioFAM / mofapy2

Multi-omics factor analysis v2
https://biofam.github.io/MOFA2/
GNU Lesser General Public License v3.0
36 stars 28 forks source link

Model fails to train with nongaussian likelihood #30

Closed Ethan-Earlie closed 1 month ago

Ethan-Earlie commented 4 months ago

Hello,

I have been able to run mofa using GPU mode only when using the Gaussian likelihood. If I try to use Poisson, I consistently get an error similar to what others have raised in different issues:

Attempting to save the model at the current iteration... Saving model in /tmp/mofa_20240725-203015_interrupted.hdf5... Note: the model to be saved is not trained. Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/mofapy2/run/entry_point.py", line 49, in saver func(self, *args, *kwargs) File "/usr/local/lib/python3.11/dist-packages/mofapy2/run/entry_point.py", line 1020, in run train_model(self.model) File "/usr/local/lib/python3.11/dist-packages/mofapy2/build_model/train_model.py", line 28, in train_model model.iterate() File "/usr/local/lib/python3.11/dist-packages/mofapy2/core/BayesNet.py", line 224, in iterate elbo.iloc[0] = self.precompute() ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/mofapy2/core/BayesNet.py", line 193, in precompute self.nodes[n].precompute(self.options) File "/usr/local/lib/python3.11/dist-packages/mofapy2/core/nodes/multiview_nodes.py", line 105, in precompute self.nodes[m].precompute(options) File "/usr/local/lib/python3.11/dist-packages/mofapy2/core/nodes/nongaussian_nodes.py", line 189, in precompute self.updateExpectations() File "/usr/local/lib/python3.11/dist-packages/mofapy2/core/nodes/nongaussian_nodes.py", line 202, in updateExpectations self.E = self.params["zeta"] - sigmoid(self.params["zeta"])(1-self.obs/self.ratefn(self.params["zeta"])) / tau


File "cupy/_core/core.pyx", line 1697, in cupy._core.core._ndarray_base.__array_ufunc__
File "cupy/_core/_kernel.pyx", line 1286, in cupy._core._kernel.ufunc.__call__
File "cupy/_core/_kernel.pyx", line 159, in cupy._core._kernel._preprocess_args
File "cupy/_core/_kernel.pyx", line 145, in cupy._core._kernel._preprocess_arg
TypeError: Unsupported type <class 'numpy.ndarray'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

 File "run_mofa.py", line 57, in < module>
     ent.run()
   File "/usr/local/lib/python3.11/dist-packages/mofapy2/run/entry_point.py", line 59, in saver
 self.save(outfile=tmp_file)
   File "/usr/local/lib/python3.11/dist-packages/mofapy2/run/entry_point.py", line 1273, in save
    tmp.saveTrainingStats()
   File "/usr/local/lib/python3.11/dist-packages/mofapy2/build_model/save_model.py", line 533, in saveTrainingStats
     stats = self.model.getTrainingStats()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/dist-packages/mofapy2/core/BayesNet.py", line 387, in getTrainingStats
     return self.train_stats
            ^^^^^^^^^^^^^^^^
 AttributeError: 'BayesNet' object has no attribute 'train_stats'
gtca commented 1 month ago

Seems like it did fail due to the incomplete GPU support for the non-gaussian nodes.

Should work now in 0.7.2.