biocore / songbird

Vanilla regression methods for microbiome differential abundance analysis
BSD 3-Clause "New" or "Revised" License
58 stars 25 forks source link

TypeError: SparseDataFrame() takes no arguments #128

Open sformel opened 4 years ago

sformel commented 4 years ago

Thanks for developing Songbird, it seems like it will be a useful tool. However, I’m trying to run it as a stand-alone program on our cluster here at Tulane and I’m getting the error below.
When I run the tutorial:

songbird multinomial \ --input-biom songbird-master/data/redsea/redsea.biom \ --metadata-file songbird-master/data/redsea/redsea_metadata.txt \ --formula "Depth+Temperature+Salinity+Oxygen+Fluorescence+Nitrate" \ --epochs 10000 \ --differential-prior 0.5 \ --summary-interval 1 \ --summary-dir results

I end up with the error

Traceback (most recent call last): File "/lustre/project/svanbael/steve/conda-envs/songbird_env/bin/songbird", line 225, in <module> songbird() File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/bin/songbird", line 154, in multinomial dense_table = table.to_dataframe().to_dense().T File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/biom/table.py", line 4068, in to_dataframe return constructor(mat, index=index, columns=columns) TypeError: SparseDataFrame() takes no arguments

I apologize if there is an obvious solution, I’m still learning Python. If it’s helpful, this is the output from “conda info”:

active environment : songbird_env active env location : /lustre/project/svanbael/steve/conda-envs/songbird_env shell level : 1 user config file : /home/sformel/.condarc populated config files : /home/sformel/.condarc conda version : 4.5.0 conda-build version : 3.7.2 python version : 3.6.4.final.0 base environment : /share/apps/anaconda/3/5.1.0 (read only) channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://conda.anaconda.org/bioconda/linux-64 https://conda.anaconda.org/bioconda/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/linux-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/pro/linux-64 https://repo.anaconda.com/pkgs/pro/noarch package cache : /lustre/project/svanbael/steve/conda_pkgs envs directories : /lustre/project/svanbael/steve/conda-envs /home/sformel/.conda/envs /share/apps/anaconda/3/5.1.0/envs platform : linux-64 user-agent : conda/4.5.0 requests/2.18.4 CPython/3.6.4 Linux/2.6.32-431.23.3.el6.x86_64 centos/6.5 glibc/2.12 UID:GID : 281003972:5043 netrc file : None offline mode : False

mortonjt commented 4 years ago

Hi @sformel thank you for reporting this error. There has been a dependency issue with pandas v1.0 lately. The easiest fix would be to restrict this version (i.e. conda install pandas==0.25.3).

It'll help to print the contents of env.yaml from conda env export env.yaml to confirm this.

sformel commented 4 years ago

@mortonjt THanks for the quick response! You were correct, downgrading pandas got me past that error, but now I've run up against another:

`Traceback (most recent call last): File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/matplotlib/font_manager.py", line 1341, in fontManager = json_load(_fmcache) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/matplotlib/font_manager.py", line 937, in json_load return json.load(fh, object_hook=_json_decode) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/json/init.py", line 296, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, kw) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/json/init.py", line 361, in loads return cls(kw).decode(s) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 503 column 7 (char 12283)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/lustre/project/svanbael/steve/conda-envs/songbird_env/bin/songbird", line 3, in import pandas as pd File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/init.py", line 55, in from pandas.core.api import ( File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/core/api.py", line 24, in from pandas.core.groupby import Grouper, NamedAgg File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/core/groupby/init.py", line 1, in from pandas.core.groupby.generic import ( # noqa: F401 File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/core/groupby/generic.py", line 44, in from pandas.core.frame import DataFrame File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/core/frame.py", line 115, in from pandas.core.series import Series File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/core/series.py", line 84, in import pandas.plotting File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/plotting/init.py", line 59, in from pandas.plotting._core import ( File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/plotting/_core.py", line 17, in import pandas.plotting._matplotlib # noqa File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/plotting/_matplotlib/init.py", line 3, in from pandas.plotting._matplotlib.boxplot import ( File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/plotting/_matplotlib/boxplot.py", line 14, in from pandas.plotting._matplotlib.core import LinePlot, MPLPlot File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/plotting/_matplotlib/core.py", line 34, in from pandas.plotting._matplotlib.tools import ( File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/pandas/plotting/_matplotlib/tools.py", line 5, in import matplotlib.table File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/matplotlib/table.py", line 25, in from .text import Text File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/matplotlib/text.py", line 14, in from .font_manager import FontProperties File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/matplotlib/font_manager.py", line 1343, in _rebuild() File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/matplotlib/font_manager.py", line 1336, in _rebuild json_dump(fontManager, _fmcache) File "/lustre/project/svanbael/steve/conda-envs/songbird_env/lib/python3.7/site-packages/matplotlib/font_manager.py", line 925, in json_dump _log.warning('Could not save font_manager cache {}'.format(e)) OSError: [Errno 122] Disk quota exceeded ` I'm not sure what the first error is, and I'm not sure where it's trying to write the file when it generates the second error, but in general I should have plenty of space on my account. If it's helpful, my matplotlib pkgs are:

matplotlib=3.2.1=0 matplotlib-base=3.2.1=py37h30547a4_0

mortonjt commented 4 years ago

hmm - that sounds like a matplotlib error. I haven't seen that before - I'd double check your disk space to make sure that you really do have enough space.

The matplotlib folk maybe in a better position to help with this sort of thing. This post should be a good place to get started.

sformel commented 4 years ago

The error messages never lie, do they? There was a huge hidden folder in there causing a ruckus. It's functioning now, thanks for your help!

mortonjt commented 4 years ago

Great!!

On Wed, Apr 8, 2020, 4:00 PM Stephen Formel notifications@github.com wrote:

The error messages never lie, do they? There was a huge hidden folder in there causing a ruckus. It's functioning now, thanks for your help!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/biocore/songbird/issues/128#issuecomment-611216628, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA75VXO4JUTA6E332LMMZ43RLTXWJANCNFSM4MECWTDA .

Samyr-Tachy commented 4 years ago

Instead of using SparseDataFrame() try using pd.DataFrame.sparse.from_spmatrix()