search

gao-lab / GLUE

Graph-linked unified embedding for single-cell multi-omics data integration
MIT License
365 stars 55 forks source link

Integrate multiomics RNA and ATAC data with different number of cells: ValueError The value argument must be within the support #40

Closed elhaam closed 2 years ago

elhaam commented 2 years ago

Hi there, thanks for the very helpful package, GLUE!

I am integrating single-cell RNA-seq and ATAC-seq data where one cluster is missing in one dataset, so the sizes of the two datasets are different. When running GLUE, I get the following error, could you help me with this? Is the error happening because of the size difference between the number of cells in RNA-seq (707 cells) and ATAC-seq (1047 cells) matrices?

I followed the tutorials, but at this step I get an error: 

glue = scglue.models.fit_SCGLUE(
    {"rna": rna, "atac": atac}, graph,
    fit_kws={"directory": "glue"}
)
CLICK to see ERROR

[INFO] fit_SCGLUE: Pretraining SCGLUE model... [INFO] autodevice: Using GPU 1 as computation device. [INFO] SCGLUEModel: Setting `graph_batch_size` = 11614 [INFO] SCGLUEModel: Setting `max_epochs` = 1947 [INFO] SCGLUEModel: Setting `patience` = 163 [INFO] SCGLUEModel: Setting `reduce_lr_patience` = 82 [INFO] SCGLUETrainer: Using training directory: "glue/pretrain" ValueError Traceback (most recent call last) in 1 glue = scglue.models.fit_SCGLUE( 2 {"rna": rna, "atac": atac}, graph, ----> 3 fit_kws={"directory": "glue"} 4 ) ~/.local/lib/python3.6/site-packages/scglue/models/__init__.py in fit_SCGLUE(adatas, graph, model, init_kws, compile_kws, fit_kws, balance_kws) 89 pretrain = model(adatas, sorted(graph.nodes), **pretrain_init_kws) 90 pretrain.compile(**compile_kws) ---> 91 pretrain.fit(adatas, graph, **pretrain_fit_kws) 92 if "directory" in pretrain_fit_kws: 93 pretrain.save(os.path.join(pretrain_fit_kws["directory"], "pretrain.dill")) ~/.local/lib/python3.6/site-packages/scglue/models/scglue.py in fit(self, adatas, graph, edge_weight, edge_sign, neg_samples, val_split, data_batch_size, graph_batch_size, align_burnin, safe_burnin, max_epochs, patience, reduce_lr_patience, wait_n_lrs, directory) 1516 reduce_lr_patience=reduce_lr_patience, wait_n_lrs=wait_n_lrs, 1517 random_seed=self.random_seed, -> 1518 directory=directory 1519 ) 1520 ~/.local/lib/python3.6/site-packages/scglue/models/base.py in fit(self, *args, **kwargs) 332 Subclasses may override arguments for API definition. 333 """ --> 334 self.trainer.fit(*args, **kwargs) 335 336 def get_losses(self, *args, **kwargs) -> Mapping[str, float]: ~/.local/lib/python3.6/site-packages/scglue/models/glue.py in fit(self, data, graph, val_split, data_batch_size, graph_batch_size, align_burnin, safe_burnin, max_epochs, patience, reduce_lr_patience, wait_n_lrs, random_seed, directory, plugins) 609 train_loader, val_loader=val_loader, 610 max_epochs=max_epochs, random_seed=random_seed, --> 611 directory=directory, plugins=plugins 612 ) 613 finally: ~/.local/lib/python3.6/site-packages/scglue/models/base.py in fit(self, train_loader, val_loader, max_epochs, random_seed, directory, plugins) 198 # Start engines 199 torch.manual_seed(random_seed) --> 200 train_engine.run(train_loader, max_epochs=max_epochs) 201 202 torch.cuda.empty_cache() # Works even if GPU is unavailable ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in run(self, data, max_epochs, epoch_length, seed) 702 703 self.state.dataloader = data --> 704 return self._internal_run() 705 706 @staticmethod ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in _internal_run(self) 781 self._dataloader_iter = None 782 self.logger.error(f"Engine run is terminating due to exception: {e}") --> 783 self._handle_exception(e) 784 785 self._dataloader_iter = None ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in _handle_exception(self, e) 462 def _handle_exception(self, e: BaseException) -> None: 463 if Events.EXCEPTION_RAISED in self._event_handlers: --> 464 self._fire_event(Events.EXCEPTION_RAISED, e) 465 else: 466 raise e ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in _fire_event(self, event_name, *event_args, **event_kwargs) 419 kwargs.update(event_kwargs) 420 first, others = ((args[0],), args[1:]) if (args and args[0] == self) else ((), args) --> 421 func(*first, *(event_args + others), **kwargs) 422 423 def fire_event(self, event_name: Any) -> None: ~/.local/lib/python3.6/site-packages/scglue/models/base.py in _handle_exception(engine, e) 161 engine.terminate() 162 else: --> 163 raise e 164 165 # Compute metrics ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in _internal_run(self) 751 self._setup_engine() 752 --> 753 time_taken = self._run_once_on_dataset() 754 # time is available for handlers but must be update after fire 755 self.state.times[Events.EPOCH_COMPLETED.name] = time_taken ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in _run_once_on_dataset(self) 852 except Exception as e: 853 self.logger.error(f"Current run is terminating due to exception: {e}") --> 854 self._handle_exception(e) 855 856 return time.time() - start_time ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in _handle_exception(self, e) 462 def _handle_exception(self, e: BaseException) -> None: 463 if Events.EXCEPTION_RAISED in self._event_handlers: --> 464 self._fire_event(Events.EXCEPTION_RAISED, e) 465 else: 466 raise e ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in _fire_event(self, event_name, *event_args, **event_kwargs) 419 kwargs.update(event_kwargs) 420 first, others = ((args[0],), args[1:]) if (args and args[0] == self) else ((), args) --> 421 func(*first, *(event_args + others), **kwargs) 422 423 def fire_event(self, event_name: Any) -> None: ~/.local/lib/python3.6/site-packages/scglue/models/base.py in _handle_exception(engine, e) 161 engine.terminate() 162 else: --> 163 raise e 164 165 # Compute metrics ~/.local/lib/python3.6/site-packages/ignite/engine/engine.py in _run_once_on_dataset(self) 838 self.state.iteration += 1 839 self._fire_event(Events.ITERATION_STARTED) --> 840 self.state.output = self._process_function(self, self.state.batch) 841 self._fire_event(Events.ITERATION_COMPLETED) 842 ~/.local/lib/python3.6/site-packages/scglue/models/scglue.py in train_step(self, engine, data) 726 727 # Generator step --> 728 losses = self.compute_losses(data, epoch) 729 self.net.zero_grad(set_to_none=True) 730 losses["gen_loss"].backward() ~/.local/lib/python3.6/site-packages/scglue/models/scglue.py in compute_losses(self, data, epoch, dsc_only) 676 usamp[k], vsamp[getattr(net, f"{k}_idx")], xbch[k], l[k] 677 ).log_prob(x[k]).mean() --> 678 for k in net.keys 679 } 680 x_kl = { ~/.local/lib/python3.6/site-packages/scglue/models/scglue.py in (.0) 676 usamp[k], vsamp[getattr(net, f"{k}_idx")], xbch[k], l[k] 677 ).log_prob(x[k]).mean() --> 678 for k in net.keys 679 } 680 x_kl = { ~/.local/lib/python3.6/site-packages/torch/distributions/negative_binomial.py in log_prob(self, value) 90 def log_prob(self, value): 91 if self._validate_args: ---> 92 self._validate_sample(value) 93 94 log_unnormalized_prob = (self.total_count * F.logsigmoid(-self.logits) + ~/.local/lib/python3.6/site-packages/torch/distributions/distribution.py in _validate_sample(self, value) 275 assert support is not None 276 if not support.check(value).all(): --> 277 raise ValueError('The value argument must be within the support') 278 279 def _get_checked_instance(self, cls, _instance=None): ValueError: The value argument must be within the support 

> rna
AnnData object with n_obs × n_vars = 707 × 10856
    obs: 'n_genes', 'domain'
    var: 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'mean', 'std', 'chrom', 'chromStart', 'chromEnd', 'name', 'score', 'strand', 'thickStart', 'thickEnd', 'itemRgb', 'blockCount', 'blockSizes', 'blockStarts', 'gene_id', 'gene_type', 'hgnc_id', 'havana_gene', 'tag'
    uns: 'hvg', 'log1p', 'pca', 'neighbors', 'umap', '__scglue__'
    obsm: 'X_pca', 'X_umap'
    varm: 'PCs'
    layers: 'counts'
    obsp: 'distances', 'connectivities'
> atac
AnnData object with n_obs × n_vars = 1047 × 136771
    obs: 'domain'
    var: 'chrom', 'chromStart', 'chromEnd', 'highly_variable'
    uns: 'neighbors', 'umap', '__scglue__'
    obsm: 'X_lsi', 'X_umap'
    obsp: 'distances', 'connectivities'

Thanks!

elhaam commented 2 years ago

My issue is resolved using the comments in a previous issue.

Thanks again!