broadinstitute / tensorqtl

Ultrafast GPU-enabled QTL mapper
BSD 3-Clause "New" or "Revised" License
162 stars 52 forks source link

TensorQTL analysis problem #71

Closed Juyeol-Jung closed 2 months ago

Juyeol-Jung commented 2 years ago

Hi, I'm user for analyzing sc-eQTL by using tensorQTL. Thanks for developing great tool to understand SNP's role for bioscience.

I've got problem to analyse sc-eQTL by tensorQTL's cis_independent mode. Based on error information, It is occurred by pandas-based q-value estimation but I cannot easily access this problem.

These are running process result during tensorQTL's running process.

[Jul 19 17:09:24] Running TensorQTL: cis-QTL mapping

To avoid creating the large chunks, set the option

with dask.config.set(**{'array.slicing.split_large_chunks': True}): ... array[indexer] self.bed = self.bed[:,ix] Traceback (most recent call last): File "/ess/discovery/users/juyeol/99.tools/python/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3621, in get_loc return self._engine.get_loc(casted_key) File "pandas/_libs/index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'qval'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/ess/discovery/users/juyeol/99.tools/python/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/ess/discovery/users/juyeol/99.tools/python/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/ess/discovery/users/juyeol/99.tools/python/lib/python3.9/site-packages/tensorqtl-1.0.5-py3.9.egg/tensorqtl/main.py", line 2, in tensorqtl.main() File "/ess/discovery/users/juyeol/99.tools/python/lib/python3.9/site-packages/tensorqtl-1.0.5-py3.9.egg/tensorqtl/tensorqtl.py", line 144, in main res_df = cis.map_independent(genotype_df, variant_df, summary_df, phenotype_df, phenotype_pos_df, covariates_df, File "/ess/discovery/users/juyeol/99.tools/python/lib/python3.9/site-packages/tensorqtl-1.0.5-py3.9.egg/tensorqtl/cis.py", line 653, in map_independent signif_df = cis_df[cis_df[fdr_col]<=fdr].copy() File "/ess/discovery/users/juyeol/99.tools/python/lib/python3.9/site-packages/pandas/core/frame.py", line 3505, in getitem indexer = self.columns.get_loc(key) File "/ess/discovery/users/juyeol/99.tools/python/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3623, in get_loc raise KeyError(key) from err KeyError: 'qval'

I hope this information is worth for contributing this problem's solution.

and can you give some information for rfunc's role in this analysis??

Thank you.

francois-a commented 2 years ago

Hi, TensorQTL uses the qvalue package from R to estimate FDR. For this to be run, R needs to be installed with this package, together with rpy2 for Python. This is then called here, and adds the qval column to the dataframe.

Juyeol-Jung commented 2 years ago

Thank you for your kindness reply.

Because of other work, I check your reply now and I try running tensorQTL.

abiddanda54gene commented 2 years ago

Hello @francois-a, I am running into a similar issue where I need the qval column for --mode cis_independent. I am not able to find rfunc for python and am unsure of how to get it. Could you please advise?

francois-a commented 2 years ago

rfunc is part of tensorQTL, and contains a wrapper for qvalue. You'll need R and rpy2, as described above.

jonathonl commented 1 year ago

I'm having the same issue as @abiddanda54gene. Both R and rpy2 are installed, but rfunc is failing to import.

$ python3 -c "import subprocess; print(subprocess.call('which R', shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)); import rpy2; import tensorqtl"
0
Warning: 'rfunc' cannot be imported. R and the 'rpy2' Python package are needed.
jonathonl commented 1 year ago

The issue is that Iterable as been removed from collections in python v3.10 (see https://stackoverflow.com/questions/72032032/importerror-cannot-import-name-iterable-from-collections-in-python). Modifying line 5 of rfunc.py to be from collections.abc import Iterable fixes the import error.

francois-a commented 1 year ago

Hi, thanks for pointing this out. Fixed in eb72058c776ecd473a9bef7afe88ad4385edc5e4.