bug when running with exp_batch_idx_input argument

[Xiaojieqiu]

Thanks for this nice work! I find there may be a bug when we run predict function with a model trained with the one-hot matrix:

DI_model = DeepImpute.train(
            gene_expression, latent_dim, exp_batch_idx_input = lane_one_hot, T=2, batch_size=64, max_epoch=300)

# 4. latent representations and imputed expressions
latent_code, imputed_val, _ = DeepImpute.predict(
    gene_expression, DI_model)

The following include some running information (interestingly, the reconstruction error increases in my case) and the error message .

Finisheded epoch：100
Current reconstruction error is: 0.489985
Finisheded epoch：200
Current reconstruction error is: 0.4983297
Finisheded epoch：300
Current reconstruction error is: 0.5238996
Finish training 24886 samples after 300 epochs. The total training time is 2229.31636095047 seconds.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-226-3c14285a0bb8> in <module>
     20 # 4. latent representations and imputed expressions
     21 latent_code, imputed_val, _ = DeepImpute.predict(
---> 22     gene_expression, DI_model)

~/.local/lib/python3.6/site-packages/scscope/__init__.py in predict(test_data, model, batch_effect)
    261 
    262         latent_code_val, output_val, predicted_batch_effect = sess.run([latent_code[i], output[i], removed_batch_effect], feed_dict={
--> 263             test_data_holder: test_data, test_exp_batch_idx_holder: batch_effect_idx})
    264         if i == 0:
    265             latent_fea = latent_code_val

~/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    927     try:
    928       result = self._run(None, fetches, feed_dict, options_ptr,
--> 929                          run_metadata_ptr)
    930       if run_metadata:
    931         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
   1126                              'which has shape %r' %
   1127                              (np_val.shape, subfeed_t.name,
-> 1128                               str(subfeed_t.get_shape())))
   1129           if not self.graph.is_feedable(subfeed_t):
   1130             raise ValueError('Tensor %s may not be fed.' % subfeed_t)

ValueError: Cannot feed value of shape (24886, 1) for Tensor 'Placeholder_9:0', which has shape '(?, 6)'```

[feng-bao-ucsf]

Thanks for the feedback. It seems that the error was raised by a wrong input shape. Can you send us the part of the tested data (RNA-seq data and batch index data) so we can replicate this error for debug. Please send to fbao0110@gmail.com

Best, Feng

[fedeago]

Goodmorning, thanks for your work. I have the same problem. How can i fix it?

latent1, imputedval, = DeepImpute.predict(adata_test.X, DI_model1, batch_effect=batch_id_t)

---

ValueError Traceback (most recent call last)

in ----> 1 latent1, imputed_val, _ = DeepImpute.predict(adata_test.X, DI_model1, batch_effect=batch_id_t) 2 latent2, imputed_val, _ = DeepImpute.predict(adata_test.X, DI_model2, batch_effect=batch_id_t) 3 latent3, imputed_val, _ = DeepImpute.predict(adata_test.X, DI_model3, batch_effect=batch_id_t) 4 latent4, imputed_val, _ = DeepImpute.predict(adata_test.X, DI_model4, batch_effect=batch_id_t) 5 latent5, imputed_val, _ = DeepImpute.predict(adata_test.X, DI_model5, batch_effect=batch_id_t) c:\users\federico\anaconda3\envs\dca\lib\site-packages\scscope\__init__.py in predict(test_data, model, batch_effect) 265 266 latent_code_val, output_val, predicted_batch_effect = sess.run([latent_code[i], output[i], removed_batch_effect], feed_dict={ --> 267 test_data_holder: test_data, test_exp_batch_idx_holder: batch_effect_idx}) 268 if i == 0: 269 latent_fea = latent_code_val c:\users\federico\anaconda3\envs\dca\lib\site-packages\tensorflow\python\client\session.py in run(self, fetches, feed_dict, options, run_metadata) 927 try: 928 result = self._run(None, fetches, feed_dict, options_ptr, --> 929 run_metadata_ptr) 930 if run_metadata: 931 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr) c:\users\federico\anaconda3\envs\dca\lib\site-packages\tensorflow\python\client\session.py in _run(self, handle, fetches, feed_dict, options, run_metadata) 1126 'which has shape %r' % 1127 (np_val.shape, subfeed_t.name, -> 1128 str(subfeed_t.get_shape()))) 1129 if not self.graph.is_feedable(subfeed_t): 1130 raise ValueError('Tensor %s may not be fed.' % subfeed_t) ValueError: Cannot feed value of shape (1979, 2000) for Tensor 'Placeholder_2:0', which has shape '(?, 10058)

[feng-bao-ucsf]

Goodmorning, thanks for your work. I have the same problem. How can i fix it?

latent1, imputedval, = DeepImpute.predict(adata_test.X, DI_model1, batch_effect=batch_id_t)

ValueError Traceback (most recent call last) in ----> 1 latent1, imputedval, = DeepImpute.predict(adata_test.X, DI_model1, batch_effect=batch_id_t) 2 latent2, imputedval, = DeepImpute.predict(adata_test.X, DI_model2, batch_effect=batch_id_t) 3 latent3, imputedval, = DeepImpute.predict(adata_test.X, DI_model3, batch_effect=batch_id_t) 4 latent4, imputedval, = DeepImpute.predict(adata_test.X, DI_model4, batch_effect=batch_id_t) 5 latent5, imputedval, = DeepImpute.predict(adata_test.X, DI_model5, batch_effect=batch_id_t)

c:\users\federico\anaconda3\envs\dca\lib\site-packages\scscopeinit.py in predict(test_data, model, batch_effect) 265 266 latent_code_val, output_val, predicted_batch_effect = sess.run([latent_code[i], output[i], removed_batch_effect], feed_dict={ --> 267 test_data_holder: test_data, test_exp_batch_idx_holder: batch_effect_idx}) 268 if i == 0: 269 latent_fea = latent_code_val

c:\users\federico\anaconda3\envs\dca\lib\site-packages\tensorflow\python\client\session.py in run(self, fetches, feed_dict, options, run_metadata) 927 try: 928 result = self._run(None, fetches, feed_dict, options_ptr, --> 929 run_metadata_ptr) 930 if run_metadata: 931 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

c:\users\federico\anaconda3\envs\dca\lib\site-packages\tensorflow\python\client\session.py in _run(self, handle, fetches, feed_dict, options, run_metadata) 1126 'which has shape %r' % 1127 (np_val.shape, subfeed_t.name, -> 1128 str(subfeed_t.get_shape()))) 1129 if not self.graph.is_feedable(subfeed_t): 1130 raise ValueError('Tensor %s may not be fed.' % subfeed_t)

ValueError: Cannot feed value of shape (1979, 2000) for Tensor 'Placeholder_2:0', which has shape '(?, 10058)

Thank you for reporting this bug. We are woking on this problem. Could please provide a test dataset for validation? Really appreciate your help!