search

biocore / songbird

Vanilla regression methods for microbiome differential abundance analysis
BSD 3-Clause "New" or "Revised" License
58 stars 25 forks source link

Suppression of warnings and (optionally) output #106

Closed gibsramen closed 4 years ago

gibsramen commented 4 years ago

Addresses #105

Some users have mentioned that it would be useful to suppress the output of songbird.multinomial, including the TensorFlow deprecation warnings and tqdm progress bar. This PR should address most of these issues.

make test passes on my build (added a couple tests of the new functionality).

Main changes

See this SO question and this GitHub issue. Note that the latter apparently works only on TensorFlow 1.14.0. If/when this requirement is updated in Songbird, we may need to come back to this issue.

Python Warnings

Python FutureWarnings are now all filtered regardless of silent flag.

RuntimeWarnings also suppressed in standalone CLI.

$ songbird multinomial \
    --input-biom data/redsea/redsea.biom \
    --metadata-file data/redsea/redsea_metadata.txt \
    --formula "Depth+Temperature+Salinity+Oxygen+Fluorescence+Nitrate" \
    --epochs 10000 \
    --differential-prior 0.5 \
    --summary-interval 1 \
    --summary-dir results

Old behavior

/miniconda3/envs/evans_fxr/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/miniconda3/envs/evans_fxr/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
.
.
.
/miniconda3/envs/evans_fxr/lib/python3.6/site-packages/biom/table.py:4049: FutureWarning: SparseSeries is deprecated and will be removed in a future version.
Use a Series with sparse values instead.

    >>> series = pd.Series(pd.SparseArray(...))

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  for r in self.matrix_data.tocsr()]
/miniconda3/envs/evans_fxr/lib/python3.6/site-packages/biom/table.py:4052: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.
.
.
.
WARNING:tensorflow:From /miniconda3/envs/evans_fxr/bin/songbird:173: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-01-17 14:06:30.183343: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
WARNING:tensorflow:From /miniconda3/envs/evans_fxr/bin/songbird:176: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.
.
.
.
WARNING:tensorflow:From /miniconda3/envs/evans_fxr/lib/python3.6/site-packages/songbird/multinomial.py:163: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

  0%|                                                            | 0/80000 [00:00<?, ?it/s]

New behavior

WARNING:tensorflow:From /Users/gibs/projects/songbird/scripts/songbird:199: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-01-17 14:35:40.991584: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
WARNING:tensorflow:From /Users/gibs/projects/songbird/scripts/songbird:202: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

WARNING:tensorflow:From /Users/gibs/projects/songbird/songbird/multinomial.py:75: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.random.categorical` instead.
WARNING:tensorflow:From /Users/gibs/projects/songbird/songbird/multinomial.py:86: The name tf.random_normal is deprecated. Please use tf.random.normal instead.
.
.
.
WARNING:tensorflow:From /Users/gibs/projects/songbird/songbird/multinomial.py:167: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

  0%|                                                                | 0/80000 [00:00<?, ?it/s]

Silent flag

Both standalone and Q2 implementations can now specify whether or not to suppress output.

$ songbird multinomial \
    --input-biom data/redsea/redsea.biom \
    --metadata-file data/redsea/redsea_metadata.txt \
    --formula "Depth+Temperature+Salinity+Oxygen+Fluorescence+Nitrate" \
    --epochs 10000 \
    --differential-prior 0.5 \
    --summary-interval 1 \
    --summary-dir results \
        --silent; \
        echo "Finished"
Finished

~NOTE: To my knowledge, calling qiime songbird multinomial from the CLI defaults to not showing the tqdm progress bar. In this case, the quiet flag is redundant and provided for compatibility. Calling through the Artifact API, however, outputs the tqdm progress bar normally, so the flag can be used there.~

Turns out that you will see the progress bar from the Qiime2 invocation if you provide the --verbose option.

gibsramen commented 4 years ago

Hmmm... I addressed the flake8 issues but now it looks like there's some sort of GLIBC error in the travis check. Is this something my changes brought about? (Not super familiar with C stuff)

fedarko commented 4 years ago

@gibsramen I updated the changelog about this PR -- let me know if this description looks good/bad to you.