Overflow encountered in duality gap computation

[alyosama]

Hi

I was analyzing developmental time series data using wot and I have faced this issue with specific time points pairs. The tool was okay when I tried it with some time points pairs (e.g. 10 and 11 days) but when I tried to compute the transport maps on all-time points, I have found this issue "Overflow encountered in duality gap ".

How can I fix this issue ?

/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py:159: RuntimeWarning: invalid value encountered in double_scalars
  np.linalg.norm(_a - old_a * np.exp(u / epsilon_i)) / (1 + np.linalg.norm(_a)),
/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py:148: RuntimeWarning: overflow encountered in exp
  _a = a * np.exp(u / epsilon_i)
/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py:159: RuntimeWarning: overflow encountered in exp
  np.linalg.norm(_a - old_a * np.exp(u / epsilon_i)) / (1 + np.linalg.norm(_a)),
/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py:159: RuntimeWarning: invalid value encountered in subtract
  np.linalg.norm(_a - old_a * np.exp(u / epsilon_i)) / (1 + np.linalg.norm(_a)),
/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py:148: RuntimeWarning: overflow encountered in multiply
  _a = a * np.exp(u / epsilon_i)
/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py:134: RuntimeWarning: divide by zero encountered in true_divide
  b = (q / (K.T.dot(np.multiply(a, dx)))) ** alpha2 * np.exp(-v / (lambda2 + epsilon_i))
/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py:138: RuntimeWarning: divide by zero encountered in log
  u = u + epsilon_i * np.log(a)
/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py:140: RuntimeWarning: invalid value encountered in add
  K = np.exp((np.array([u]).T - C + np.array([v])) / epsilon_i)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-22-fdb24e4a9777> in <module>
      1 ot_model = wot.ot.OTModel(adata, epsilon = 0.05, lambda1 = 1, lambda2 = 50, growth_iters = 2)
----> 2 ot_model.compute_all_transport_maps(tmap_out='data/tmaps/serum')
      3 
      4 # we can speed this up by supplying g2 and doing 1 growth iter

/opt/miniconda3/lib/python3.7/site-packages/wot/ot/ot_model.py in compute_all_transport_maps(self, tmap_out, overwrite, output_file_format, with_covariates)
    186                 continue
    187 
--> 188             tmap = self.compute_transport_map(*day_pair)
    189             wot.io.write_dataset(tmap, output_file, output_format=output_file_format)
    190             if save_learned_growth:

/opt/miniconda3/lib/python3.7/site-packages/wot/ot/ot_model.py in compute_transport_map(self, t0, t1, covariate)
    229             logger.info('Computing transport map from {} {} to {} {}'.format(t0, covariate[0], t1, covariate[1]))
    230         config = {**self.ot_config, **local_config, 't0': t0, 't1': t1, 'covariate': covariate}
--> 231         return self.compute_single_transport_map(config)
    232 
    233     @staticmethod

/opt/miniconda3/lib/python3.7/site-packages/wot/ot/ot_model.py in compute_single_transport_map(self, config)
    302         else:
    303             config['G'] = np.ones(C.shape[0])
--> 304         tmap, learned_growth = wot.ot.compute_transport_matrix(solver=self.solver, **config)
    305         learned_growth.append(tmap.sum(axis=1))
    306         obs_growth = {}

/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py in compute_transport_matrix(solver, **params)
     28         params['G'] = row_sums
     29         learned_growth.append(row_sums)
---> 30         tmap = solver(**params)
     31         gc.collect()
     32 

/opt/miniconda3/lib/python3.7/site-packages/wot/ot/optimal_transport.py in optimal_transport_duality_gap(C, G, lambda1, lambda2, epsilon, batch_size, tolerance, tau, epsilon0, max_iter, **ignored)
    161 
    162     if np.isnan(duality_gap):
--> 163         raise RuntimeError("Overflow encountered in duality gap computation, please report this incident")
    164     return R / C.shape[1]
    165 

RuntimeError: Overflow encountered in duality gap computation, please report this incident
This can also be run from the command line as follows:

[alyosama]

Solved by normalizing the gene/cell-matrix

[lamdan2]

Solved by normalizing the gene/cell-matrix

Could you possibly elaborate? What kind of normalization did you do? Thanks :-)