search

aertslab / scenicplus

SCENIC+ is a python package to build gene regulatory networks (GRNs) using combined or separate single-cell gene expression (scRNA-seq) and single-cell chromatin accessibility (scATAC-seq) data.
Other
186 stars 29 forks source link

localrule region_to_gene #408

Open TingTingShao opened 6 months ago

TingTingShao commented 6 months ago

Hi,

Before it worked fine, but this time, it shows error:

localrule region_to_gene:
    input: /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/ACC_GEX.h5mu, /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/search_space.tsv
    output: /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/region_to_gene_adj.tsv
    jobid: 10
    reason: Missing output files: /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/region_to_gene_adj.tsv
    threads: 20
    resources: tmpdir=/tmp

2024-05-23 16:29:25,943 SCENIC+      INFO     Reading multiome MuData.
/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/anndata/_core/anndata.py:522: FutureWarning: The dtype argument is deprecated and will be removed in late 2024.
  warnings.warn(
/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/anndata/_core/anndata.py:522: FutureWarning: The dtype argument is deprecated and will be removed in late 2024.
  warnings.warn(
2024-05-23 16:30:00,615 SCENIC+      INFO     Reading search space
2024-05-23 16:30:00,859 R2G          INFO     Calculating region to gene importances, using GBM method

Running using 20 cores:   0%|          | 0/3896 [00:00<?, ?it/s]
Running using 20 cores:   0%|          | 1/3896 [00:00<06:44,  9.63it/s]
Running using 20 cores:   0%|          | 19/3896 [00:00<00:36, 105.07it/s]
Running using 20 cores:   1%|          | 30/3896 [00:01<05:03, 12.74it/s] 
Running using 20 cores:   1%|          | 30/3896 [00:19<05:03, 12.74it/s]joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
    r = call_item()
        ^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 291, in __call__
    return self.fn(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/parallel.py", line 589, in __call__
    return [func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/parallel.py", line 589, in <listcomp>
    return [func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/scenicplus/enhancer_to_gene.py", line 140, in _score_regions_to_single_gene
    from arboreto import core as arboreto_core
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/arboreto/core.py", line 12, in <module>
    from dask.dataframe import from_delayed
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/__init__.py", line 100, in <module>
    from dask.dataframe import backends, dispatch, rolling
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/backends.py", line 15, in <module>
    from dask.dataframe.core import DataFrame, Index, Scalar, Series, _Frame
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/core.py", line 36, in <module>
    from dask.dataframe import methods
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/methods.py", line 34, in <module>
    from dask.dataframe.utils import is_dataframe_like, is_index_like, is_series_like
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/utils.py", line 20, in <module>
    from dask.dataframe import (  # noqa: F401 register pandas extension types
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/_dtypes.py", line 9, in <module>
    from dask.dataframe.extensions import make_array_nonempty, make_scalar
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/extensions.py", line 8, in <module>
    from dask.dataframe.accessor import (
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 126, in <module>
    class DatetimeAccessor(Accessor):
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 81, in __init_subclass__
    _bind_property(cls, pd_cls, attr, min_version)
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 35, in _bind_property
    setattr(cls, attr, property(derived_from(pd_cls, version=min_version)(func)))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/utils.py", line 987, in wrapper
    method.__doc__ = _derived_from(
                     ^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/utils.py", line 940, in _derived_from
    method_args = get_named_args(method)
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/dask/utils.py", line 701, in get_named_args
    s = inspect.signature(func)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/inspect.py", line 3263, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/inspect.py", line 3011, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/inspect.py", line 2599, in _signature_from_callable
    call = _descriptor_get(call, obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/inspect.py", line 2432, in _descriptor_get
    return get(descriptor, obj, type(obj))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: descriptor '__call__' for 'type' objects doesn't apply to a 'property' object
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/bin/scenicplus", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/scenicplus/cli/scenicplus.py", line 1137, in main
    args.func(args)
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/scenicplus/cli/scenicplus.py", line 330, in TF_to_gene
    infer_region_to_gene(
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/scenicplus/cli/commands.py", line 739, in infer_region_to_gene
    adj = calculate_regions_to_genes_relationships(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/scenicplus/enhancer_to_gene.py", line 261, in calculate_regions_to_genes_relationships
    region_to_gene_importances = _score_regions_to_genes(
                                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/scenicplus/enhancer_to_gene.py", line 219, in _score_regions_to_genes
    joblib.Parallel(
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/parallel.py", line 1952, in __call__
    return output if self.return_generator else list(output)
                                                ^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/parallel.py", line 1595, in _get_outputs
    yield from self._retrieve()
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/parallel.py", line 1699, in _retrieve
    self._raise_error_fast()
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/parallel.py", line 1734, in _raise_error_fast
    error_job.get_result(self.timeout)
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/parallel.py", line 736, in get_result
    return self._return_or_raise()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/leuven/351/vsc35107/miniconda3/envs/scenic2/lib/python3.11/site-packages/joblib/parallel.py", line 754, in _return_or_raise
    raise self._result
TypeError: descriptor '__call__' for 'type' objects doesn't apply to a 'property' object
[Thu May 23 16:32:11 2024]
Error in rule region_to_gene:
    jobid: 10
    input: /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/ACC_GEX.h5mu, /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/search_space.tsv
    output: /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/region_to_gene_adj.tsv
    shell:

        scenicplus grn_inference region_to_gene             --multiome_mudata_fname /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/ACC_GEX.h5mu             --search_space_fname /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/search_space.tsv             --temp_dir /lustre1/scratch/351/vsc35107/             --out_region_to_gene_adjacencies /lustre1/project/stg_00079/students/tingting/data/sun/snap2_allfragments/08/080results_scenicplus_pycistopic/region_to_gene_adj.tsv             --importance_scoring_method GBM             --correlation_scoring_method SR             --n_cpu 20

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Could you please help me with this?

thanks, tingting

franciskim-yonsei commented 6 months ago

Check out https://github.com/aertslab/scenicplus/issues/381#issuecomment-2100228630. You might want to update Dask. It worked for me.

TingTingShao commented 6 months ago

Hi,

Thanks for your reply.

I followed the solution, but it shows:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
distributed 2024.2.1 requires dask==2024.2.1, but you have dask 2024.4.1 which is incompatible.
scenicplus 1.0a1 requires dask[array,complete,dataframe,diagnostics,distributed]==2024.2.1, but you have dask 2024.4.1 which is incompatible.
Successfully installed dask-2024.4.1

Would this be a problem?

Thanks, tingting

franciskim-yonsei commented 6 months ago

Hi,

Thanks for your reply.

I followed the solution, but it shows:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
distributed 2024.2.1 requires dask==2024.2.1, but you have dask 2024.4.1 which is incompatible.
scenicplus 1.0a1 requires dask[array,complete,dataframe,diagnostics,distributed]==2024.2.1, but you have dask 2024.4.1 which is incompatible.
Successfully installed dask-2024.4.1

Would this be a problem?

Thanks, tingting

I encountered similar messages but just proceeded anyway. I haven't been troubled by any other errors regarding this matter, until now at least.

TingTingShao commented 6 months ago

Hi, Thanks for your reply. I followed the solution, but it shows:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
distributed 2024.2.1 requires dask==2024.2.1, but you have dask 2024.4.1 which is incompatible.
scenicplus 1.0a1 requires dask[array,complete,dataframe,diagnostics,distributed]==2024.2.1, but you have dask 2024.4.1 which is incompatible.
Successfully installed dask-2024.4.1

Would this be a problem? Thanks, tingting

I encountered similar messages but just proceeded anyway. I haven't been troubled by any other errors regarding this matter, until now at least.

I see

TingTingShao commented 6 months ago

I see, will try later with the run. Thanks!

tingting