search

YosefLab / Compass

In-Silico Modeling of Metabolic Heterogeneity using Single-Cell Transcriptomes
BSD 3-Clause "New" or "Revised" License
96 stars 29 forks source link

ValueError: too many values to unpack (expected 5) #43

Open dm8000 opened 3 years ago

dm8000 commented 3 years ago

Hello

I have been trying to use compass but I get the following error, but given my limited knowledge I'm not able to find the issue.

This is my script

. /tools/Miniconda3.7/etc/profile.d/conda.sh
conda activate /data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass

which compass

compass --data-mtx compass_adipocytes_only_input_transposed.tsv genes.csv --num-processes 10 --species mus_musculus

# To deactivate an active environment, use:
conda deactivate
exit

And here is the error:

[diogomoraes@lobster compass]$ ./metabolismanalysis.sh
/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/bin/compass
Warning: The arguments used in the temporary directory,  /data04/projects04/MarianaBoroni/adipocyte_int/bin/compass/_tmp , are different from current arguments. Cached results may not be compatible with current settings
Differing arguments:  ['data', 'data_mtx', 'species']
Enter y or yes if you want to continue Compass and used cached results.
 Otherwise rerun Compass after removing/renaming the temporary directory or changing the --temp-dir argument
y 
Evaluating Reaction Penalties...
Traceback (most recent call last):
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/bin/compass", line 8, in <module>
    sys.exit(entry())
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/compass/main.py", line 500, in entry
    args)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/compass/compass/penalties.py", line 57, in eval_reaction_penalties
    expression = utils.read_data(expression_file) #pd.read_csv(expression_file, sep='\t', index_col=0)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/compass/utils.py", line 54, in read_data
    return read_mtx(data[0], data[1], data[2])
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/compass/utils.py", line 60, in read_mtx
    mtx = scipy.io.mmread(mtx_file)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/scipy/io/mmio.py", line 72, in mmread
    return MMFile().read(source)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/scipy/io/mmio.py", line 423, in read
    self._parse_header(stream)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/scipy/io/mmio.py", line 487, in _parse_header
    self.__class__.info(stream)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/scipy/io/mmio.py", line 230, in info
    [asstr(part.strip()) for part in line.split()]
ValueError: too many values to unpack (expected 5)
schel337 commented 3 years ago

As far as I can tell, this indicates the gene expression matrix is not in the matrix market format scipy wants when reading the header. The first lines should look something like this:

%%MatrixMarket matrix coordinate real general % 40152 40 752673

Where 40152 is the number of rows, 40 is the number of columns, and 752673 is the number of nonzero entries. I would recommend trying to convert it to mtx format. If you want to send something formatted like the file you have here or to schel337@berkeley.edu I can take a look and give more details.

dm8000 commented 3 years ago

As far as I can tell, this indicates the gene expression matrix is not in the matrix market format scipy wants when reading the header. The first lines should look something like this:

%%MatrixMarket matrix coordinate real general % 40152 40 752673

Where 40152 is the number of rows, 40 is the number of columns, and 752673 is the number of nonzero entries. I would recommend trying to convert it to mtx format. If you want to send something formatted like the file you have here or to schel337@berkeley.edu I can take a look and give more details.

Thank you Schel

Now I feel stupid because I structured the data in a tsv format and forgot to change the script! This was definitely the issue.

However, I'm getting a different error this time:

Evaluating Reaction Penalties...
Traceback (most recent call last):
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/bin/compass", line 8, in <module>
    sys.exit(entry())
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/compass/main.py", line 500, in entry
    args)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/compass/compass/penalties.py", line 99, in eval_reaction_penalties
    input_knn=input_knn, output_knn=output_knn, latent_input=latent_input)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/compass/compass/penalties.py", line 162, in eval_reaction_penalties_shared
    reaction_expression = pd.concat(reaction_expression, axis=1)
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 284, in concat
    sort=sort,
  File "/data04/projects04/MarianaBoroni/lbbc_members/lib/conda_envs/compass/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 331, in __init__
    raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate
schel337 commented 3 years ago

This looks related to the issue here https://github.com/YosefLab/Compass/issues/45

I am not certain of the cause of this issue, but first I would suggest trying to parse the data in python using pandas.read_csv("expression.tsv", sep="\t", index_col=0) and see if the data has been read correctly.