Receiving AssertionErrors after running `pytest ./tests/`

[michael-a-green]

❓ Questions and Help

I setup mmf following the instructions documented here: https://mmf.sh/docs/getting_started/installation/

1. conda create -n mmf python=3.7
2. conda activate mmf
3. git clone https://github.com/facebookresearch/mmf.git
4. cd mmf
5. pip install --editable .

I then run the tests with this command:

pytest ./tests/

I get the following assertion errors:

============================================================================================================================ FAILURES =============================================================================================================================
___________________________________________________________________________________________________________ TestModuleLayers.test_bert_classifier_head ____________________________________________________________________________________________________________

self = <tests.modules.test_layers.TestModuleLayers testMethod=test_bert_classifier_head>

    def test_bert_classifier_head(self):
        config = {}
        config["hidden_size"] = 768
        config["hidden_act"] = "gelu"
        config["layer_norm_eps"] = 1e-12
        config["hidden_dropout_prob"] = 0.1
        config = OmegaConf.create(config)
        clf = layers.ClassifierLayer("bert", 768, 1, config=config)
        self.assertEqual(len(list(clf.module.children())), 3)
        self.assertEqual(len(list(clf.parameters())), 6)

        inp = torch.rand(3, 768)

        output = clf(inp)
        self.assertEqual(output.size(), torch.Size((3, 1)))
>       np.testing.assert_almost_equal(
            output.squeeze().tolist(), [0.5452202, -0.0437842, -0.377468], decimal=3
        )
E       AssertionError: 
E       Arrays are not almost equal to 3 decimals
E       
E       Mismatched elements: 3 / 3 (100%)
E       Max absolute difference: 0.45450842
E       Max relative difference: 10.38064924
E        x: array([ 0.741,  0.411, -0.254])
E        y: array([ 0.545, -0.044, -0.377])

tests/modules/test_layers.py:112: AssertionError
____________________________________________________________________________________________________________________ TestModuleLayers.test_mlp ____________________________________________________________________________________________________________________

self = <tests.modules.test_layers.TestModuleLayers testMethod=test_mlp>

    def test_mlp(self):
        mlp = layers.ClassifierLayer("mlp", in_dim=300, out_dim=1)
        self.assertEqual(len(list(mlp.module.layers.children())), 1)
        self.assertEqual(len(list(mlp.parameters())), 2)

        inp = torch.rand(3, 300)

        output = mlp(inp)
        self.assertEqual(output.size(), torch.Size((3, 1)))
        np.testing.assert_almost_equal(
            output.squeeze().tolist(), [0.1949174, 0.4030975, -0.0109139]
        )

        mlp = layers.ClassifierLayer(
            "mlp", in_dim=300, out_dim=1, hidden_dim=150, num_layers=1
        )

        self.assertEqual(len(list(mlp.module.layers.children())), 5)
        self.assertEqual(len(list(mlp.parameters())), 6)

        inp = torch.rand(3, 300)

        output = mlp(inp)
        self.assertEqual(output.size(), torch.Size((3, 1)))
>       np.testing.assert_almost_equal(
            output.squeeze().tolist(), [-0.503411, 0.1725615, -0.6833304], decimal=3
        )
E       AssertionError: 
E       Arrays are not almost equal to 3 decimals
E       
E       Mismatched elements: 3 / 3 (100%)
E       Max absolute difference: 0.40513365
E       Max relative difference: 1.94583034
E        x: array([-0.637,  0.508, -0.278])
E        y: array([-0.503,  0.173, -0.683])

tests/modules/test_layers.py:93: AssertionError

Are these error messages OK? If not, what do I need to do to resolve them? Thank you.

Here's some info on the system I'm using. Please let me know if you need more info:

> python --version
Python 3.7.9

> conda --version
conda 4.8.3

> lsb_release -a
LSB Version:    core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0.fake-amd64:desktop-4.0.fake-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0.fake-amd64:graphics-4.0.fake-noarch
Distributor ID: openSUSE
Description:    openSUSE Tumbleweed
Release:    20200128
Codename:   n/a

> uname -a
Linux <REDACTED> 5.4.14-1-default #1 SMP Thu Jan 23 08:54:47 UTC 2020 (fc4ea7a) x86_64 x86_64 x86_64 GNU/Linux

Full Log is below:

> pytest ./tests/
======================================================================================================================= test session starts =======================================================================================================================
platform linux -- Python 3.7.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf
collected 122 items                                                                                                                                                                                                                                               

tests/common/test_batch_collator.py .                                                                                                                                                                                                                       [  0%]
tests/common/test_sample.py ....                                                                                                                                                                                                                            [  4%]
tests/configs/test_configs_for_keys.py ..                                                                                                                                                                                                                   [  5%]
tests/configs/test_zoo_urls.py ..                                                                                                                                                                                                                           [  7%]
tests/datasets/test_base_dataset.py .                                                                                                                                                                                                                       [  8%]
tests/datasets/test_mmf_dataset_builder.py ...                                                                                                                                                                                                              [ 10%]
tests/datasets/test_multi_dataset_loader.py ..                                                                                                                                                                                                              [ 12%]
tests/datasets/test_prediction_processors.py .                                                                                                                                                                                                              [ 13%]
tests/datasets/test_processors.py .....                                                                                                                                                                                                                     [ 17%]
tests/models/test_cnn_lstm.py .                                                                                                                                                                                                                             [ 18%]
tests/models/test_mmbt.py ......                                                                                                                                                                                                                            [ 22%]
tests/models/test_mmf_transformer.py ....                                                                                                                                                                                                                   [ 26%]
tests/models/test_vilbert.py ....                                                                                                                                                                                                                           [ 29%]
tests/models/test_visual_bert.py ...                                                                                                                                                                                                                        [ 31%]
tests/models/interfaces/test_interfaces.py .                                                                                                                                                                                                                [ 32%]
tests/modules/test_encoders.py ......                                                                                                                                                                                                                       [ 37%]
tests/modules/test_fusions.py ..........                                                                                                                                                                                                                    [ 45%]
tests/modules/test_layers.py F..F.                                                                                                                                                                                                                          [ 50%]
tests/modules/test_losses.py ..                                                                                                                                                                                                                             [ 51%]
tests/modules/test_metrics.py ............                                                                                                                                                                                                                  [ 61%]
tests/modules/test_optimizers.py ..                                                                                                                                                                                                                         [ 63%]
tests/trainers/test_device.py .                                                                                                                                                                                                                             [ 63%]
tests/trainers/test_fp16.py ..                                                                                                                                                                                                                              [ 65%]
tests/trainers/test_training_loop.py ...                                                                                                                                                                                                                    [ 68%]
tests/trainers/callbacks/test_logistics.py ....                                                                                                                                                                                                             [ 71%]
tests/trainers/callbacks/test_lr_scheduler.py .                                                                                                                                                                                                             [ 72%]
tests/utils/test_checkpoint.py .........                                                                                                                                                                                                                    [ 79%]
tests/utils/test_configuration.py ..                                                                                                                                                                                                                        [ 81%]
tests/utils/test_distributed.py .                                                                                                                                                                                                                           [ 81%]
tests/utils/test_download.py ...                                                                                                                                                                                                                            [ 84%]
tests/utils/test_file_io.py ....                                                                                                                                                                                                                            [ 87%]
tests/utils/test_general.py ..                                                                                                                                                                                                                              [ 89%]
tests/utils/test_logger.py ..                                                                                                                                                                                                                               [ 90%]
tests/utils/test_quality_checks.py ..                                                                                                                                                                                                                       [ 92%]
tests/utils/test_text.py ......                                                                                                                                                                                                                             [ 97%]
tests/utils/test_timer.py ...                                                                                                                                                                                                                               [100%]

============================================================================================================================ FAILURES =============================================================================================================================
___________________________________________________________________________________________________________ TestModuleLayers.test_bert_classifier_head ____________________________________________________________________________________________________________

self = <tests.modules.test_layers.TestModuleLayers testMethod=test_bert_classifier_head>

    def test_bert_classifier_head(self):
        config = {}
        config["hidden_size"] = 768
        config["hidden_act"] = "gelu"
        config["layer_norm_eps"] = 1e-12
        config["hidden_dropout_prob"] = 0.1
        config = OmegaConf.create(config)
        clf = layers.ClassifierLayer("bert", 768, 1, config=config)
        self.assertEqual(len(list(clf.module.children())), 3)
        self.assertEqual(len(list(clf.parameters())), 6)

        inp = torch.rand(3, 768)

        output = clf(inp)
        self.assertEqual(output.size(), torch.Size((3, 1)))
        np.testing.assert_almost_equal(
            #output.squeeze().tolist(), [0.5452202, -0.0437842, -0.377468], decimal=3
>           output.squeeze().tolist(), [0.5452202, -0.0437842, -0.377468], decimal=1
        )
E       AssertionError: 
E       Arrays are not almost equal to 1 decimals
E       
E       Mismatched elements: 2 / 3 (66.7%)
E       Max absolute difference: 0.45450842
E       Max relative difference: 10.38064924
E        x: array([ 0.7,  0.4, -0.3])
E        y: array([ 0.5, -0. , -0.4])

tests/modules/test_layers.py:115: AssertionError
____________________________________________________________________________________________________________________ TestModuleLayers.test_mlp ____________________________________________________________________________________________________________________

self = <tests.modules.test_layers.TestModuleLayers testMethod=test_mlp>

    def test_mlp(self):
        mlp = layers.ClassifierLayer("mlp", in_dim=300, out_dim=1)
        self.assertEqual(len(list(mlp.module.layers.children())), 1)
        self.assertEqual(len(list(mlp.parameters())), 2)

        inp = torch.rand(3, 300)

        output = mlp(inp)
        self.assertEqual(output.size(), torch.Size((3, 1)))
        np.testing.assert_almost_equal(
            output.squeeze().tolist(), [0.1949174, 0.4030975, -0.0109139]
        )

        mlp = layers.ClassifierLayer(
            "mlp", in_dim=300, out_dim=1, hidden_dim=150, num_layers=1
        )

        self.assertEqual(len(list(mlp.module.layers.children())), 5)
        self.assertEqual(len(list(mlp.parameters())), 6)

        inp = torch.rand(3, 300)

        output = mlp(inp)
        self.assertEqual(output.size(), torch.Size((3, 1)))
        np.testing.assert_almost_equal(
            #output.squeeze().tolist(), [-0.503411, 0.1725615, -0.6833304], decimal=3
>           output.squeeze().tolist(), [-0.503411, 0.1725615, -0.6833304], decimal=1
        )
E       AssertionError: 
E       Arrays are not almost equal to 1 decimals
E       
E       Mismatched elements: 2 / 3 (66.7%)
E       Max absolute difference: 0.40513365
E       Max relative difference: 1.94583034
E        x: array([-0.6,  0.5, -0.3])
E        y: array([-0.5,  0.2, -0.7])

tests/modules/test_layers.py:95: AssertionError
======================================================================================================================== warnings summary =========================================================================================================================
../../../../../../anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/site-packages/transformers/modeling_deberta.py:18
  /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/site-packages/transformers/modeling_deberta.py:18: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    from collections import Sequence

tests/utils/test_model.py:7
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/tests/utils/test_model.py:7: PytestCollectionWarning: cannot collect test class 'TestDecoderModel' because it has a __init__ constructor (from: tests/utils/test_model.py)
    class TestDecoderModel(nn.Module):

tests/utils/test_model.py:7
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/tests/utils/test_model.py:7: PytestCollectionWarning: cannot collect test class 'TestDecoderModel' because it has a __init__ constructor (from: tests/utils/test_text.py)
    class TestDecoderModel(nn.Module):

tests/common/test_sample.py::TestFunctions::test_to_device
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_checkpoint_scaler_loading
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_finalize_and_restore_from_it
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_finalize_and_resume_file
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_max_to_keep
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_pretrained_load
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_resets
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_save_and_load_state_dict
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_zoo_load
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/common/sample.py:405: UserWarning: You are not returning SampleList/Sample from your dataset. MMF expects you to move your tensors to cuda yourself.
    "You are not returning SampleList/Sample from your dataset. "

tests/configs/test_configs_for_keys.py::TestConfigsForKeys::test_dataset_configs_for_keys
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/tests/configs/test_configs_for_keys.py:51: UserWarning: Dataset vqa2_ocr has no default configuration defined. Skipping it. Make sure it is intentional
    ).format(builder_key)

tests/configs/test_configs_for_keys.py::TestConfigsForKeys::test_model_configs_for_keys
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/tests/configs/test_configs_for_keys.py:26: UserWarning: Model multihead has no default configuration defined. Skipping it. Make sure it is intentional
    ).format(model_key)

tests/configs/test_configs_for_keys.py::TestConfigsForKeys::test_model_configs_for_keys
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/tests/configs/test_configs_for_keys.py:26: UserWarning: Model top_down_bottom_up has no default configuration defined. Skipping it. Make sure it is intentional
    ).format(model_key)

tests/datasets/test_mmf_dataset_builder.py::TestMMFDatasetBuilder::test_train_split_alignment
tests/datasets/test_mmf_dataset_builder.py::TestMMFDatasetBuilder::test_train_split_len
tests/datasets/test_mmf_dataset_builder.py::TestMMFDatasetBuilder::test_train_split_non_overlap
  /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/site-packages/omegaconf/dictconfig.py:252: UserWarning: Keys with dot (split_train.seed) are deprecated and will have different semantic meaning the next major version of OmegaConf (2.1)
  See the compact keys issue for more details: https://github.com/omry/omegaconf/issues/152
  You can disable this warning by setting the environment variable OC_DISABLE_DOT_ACCESS_WARNING=1
    warnings.warn(message=msg, category=UserWarning)

tests/models/test_mmbt.py: 3 warnings
tests/models/test_mmf_transformer.py: 4 warnings
tests/models/test_vilbert.py: 4 warnings
tests/models/test_visual_bert.py: 3 warnings
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/models/base_model.py:115: UserWarning: No losses are defined in model configuration. You are expected to return loss in your return dict from forward.
    "No losses are defined in model configuration. You are expected "

tests/models/test_mmf_transformer.py::TestMMFTransformerTorchscript::test_finetune_bert_base
tests/models/test_mmf_transformer.py::TestMMFTransformerTorchscript::test_finetune_roberta_base
tests/models/test_mmf_transformer.py::TestMMFTransformerTorchscript::test_finetune_xlmr_base
tests/models/test_visual_bert.py::TestVisualBertTorchscript::test_finetune_model
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/modules/losses.py:99: UserWarning: Sample list has not field 'targets', are you sure that your ImDB has labels? you may have wanted to run with evaluation.predict=true
    "Sample list has not field 'targets', are you "

tests/models/test_visual_bert.py::TestVisualBertPretraining::test_pretrained_model
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_checkpoint_scaler_loading
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_finalize_and_restore_from_it
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_finalize_and_resume_file
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_max_to_keep
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_pretrained_load
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_resets
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_save_and_load_state_dict
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_zoo_load
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/models/base_model.py:184: UserWarning: 'losses' already present in model output. No calculation will be done in base model.
    "'losses' already present in model output. "

tests/modules/test_metrics.py::TestModuleMetrics::test_caption_bleu4
  /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/site-packages/nltk/decorators.py:68: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly
    regargs, varargs, varkwargs, defaults, formatvalue=lambda value: ""

tests/modules/test_metrics.py::TestModuleMetrics::test_caption_bleu4
  /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/site-packages/nltk/lm/vocabulary.py:13: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    from collections import Counter, Iterable

tests/modules/test_metrics.py::TestModuleMetrics::test_caption_bleu4
  /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
    return f(*args, **kwds)

tests/trainers/test_fp16.py::TestFp16::test_fp16_values
tests/trainers/test_fp16.py::TestFp16::test_fp16_works
tests/trainers/test_training_loop.py::TestTrainingLoop::test_epoch_over_updates
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/trainers/core/training_loop.py:237: UserWarning: Both max_updates and max_epochs are specified. Favoring max_epochs: 0.04
    + f"Favoring max_epochs: {max_epochs}"

tests/trainers/callbacks/test_lr_scheduler.py::TestLogisticsCallback::test_on_update_end
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/utils/build.py:255: UserWarning: No type for scheduler specified even though lr_scheduler is True, setting default to 'Pythia'
    "No type for scheduler specified even though lr_scheduler is True, "

tests/trainers/callbacks/test_lr_scheduler.py::TestLogisticsCallback::test_on_update_end
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/utils/build.py:261: UserWarning: scheduler attributes has no params defined, defaulting to {}.
    warnings.warn("scheduler attributes has no params defined, defaulting to {}.")

tests/trainers/callbacks/test_lr_scheduler.py::TestLogisticsCallback::test_on_update_end
  /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:123: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
    "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)

tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_checkpoint_scaler_loading
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_finalize_and_restore_from_it
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_max_to_keep
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_pretrained_load
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_resets
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_save_and_load_state_dict
  /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:200: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.
    warnings.warn(SAVE_STATE_WARNING, UserWarning)

tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_finalize_and_restore_from_it
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_resets
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_save_and_load_state_dict
  /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:218: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.
    warnings.warn(SAVE_STATE_WARNING, UserWarning)

tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_finalize_and_resume_file
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_zoo_load
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/utils/checkpoint.py:255: UserWarning: 'optimizer' key is not present in the checkpoint asked to be loaded. Skipping.
    "'optimizer' key is not present in the "

tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_finalize_and_resume_file
tests/utils/test_checkpoint.py::TestUtilsCheckpoint::test_zoo_load
  /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf/mmf/utils/checkpoint.py:298: UserWarning: 'lr_scheduler' key is not present in the checkpoint asked to be loaded. Setting lr_scheduler's last_epoch to current_iteration.
    "'lr_scheduler' key is not present in the "

-- Docs: https://docs.pytest.org/en/stable/warnings.html
===================================================================================================================== short test summary info =====================================================================================================================
FAILED tests/modules/test_layers.py::TestModuleLayers::test_bert_classifier_head - AssertionError: 
FAILED tests/modules/test_layers.py::TestModuleLayers::test_mlp - AssertionError: 
===================================================================================================== 2 failed, 120 passed, 67 warnings in 1461.25s (0:24:21) =====================================================================================================
(mmf2_venv) <USERNAME>@<HOSTNAME_REDACTED>:~/Dropbox/PERSONAL/Documents/Metis/git/my_mmf> which python

More info:

conda list
# packages in environment at /home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
attrs                     20.3.0                   pypi_0    pypi
ca-certificates           2020.10.14                    0  
certifi                   2020.6.20          pyhd3eb1b0_3  
chardet                   3.0.4                    pypi_0    pypi
click                     7.1.2                    pypi_0    pypi
demjson                   2.2.4                    pypi_0    pypi
editdistance              0.5.3                    pypi_0    pypi
fasttext                  0.9.1                    pypi_0    pypi
filelock                  3.0.12                   pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
gitdb                     4.0.5                    pypi_0    pypi
gitpython                 3.1.0                    pypi_0    pypi
idna                      2.10                     pypi_0    pypi
importlib-metadata        2.0.0                    pypi_0    pypi
iniconfig                 1.1.1                    pypi_0    pypi
joblib                    0.17.0                   pypi_0    pypi
ld_impl_linux-64          2.33.1               h53a641e_7  
libedit                   3.1.20191231         h14c3975_1  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.1.0                hdf63c60_0  
libstdcxx-ng              9.1.0                hdf63c60_0  
lmdb                      0.98                     pypi_0    pypi
mmf                       1.0.0rc12                 dev_0    <develop>
ncurses                   6.2                  he6710b0_1  
nltk                      3.4.5                    pypi_0    pypi
numpy                     1.19.4                   pypi_0    pypi
omegaconf                 2.0.1rc4                 pypi_0    pypi
openssl                   1.1.1h               h7b6447c_0  
packaging                 20.4                     pypi_0    pypi
pillow                    8.0.1                    pypi_0    pypi
pip                       20.2.4           py37h06a4308_0  
pluggy                    0.13.1                   pypi_0    pypi
protobuf                  3.14.0                   pypi_0    pypi
py                        1.9.0                    pypi_0    pypi
pybind11                  2.6.1                    pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
pytest                    6.1.2                    pypi_0    pypi
python                    3.7.9                h7579374_0  
pyyaml                    5.3.1                    pypi_0    pypi
readline                  8.0                  h7b6447c_0  
regex                     2020.11.13               pypi_0    pypi
requests                  2.23.0                   pypi_0    pypi
sacremoses                0.0.43                   pypi_0    pypi
scikit-learn              0.23.2                   pypi_0    pypi
scipy                     1.5.4                    pypi_0    pypi
sentencepiece             0.1.94                   pypi_0    pypi
setuptools                50.3.1           py37h06a4308_1  
six                       1.15.0                   pypi_0    pypi
sklearn                   0.0                      pypi_0    pypi
smmap                     3.0.4                    pypi_0    pypi
sqlite                    3.33.0               h62c20be_0  
termcolor                 1.1.0                    pypi_0    pypi
threadpoolctl             2.1.0                    pypi_0    pypi
tk                        8.6.10               hbc83047_0  
tokenizers                0.9.2                    pypi_0    pypi
toml                      0.10.2                   pypi_0    pypi
torch                     1.6.0                    pypi_0    pypi
torchtext                 0.5.0                    pypi_0    pypi
torchvision               0.7.0                    pypi_0    pypi
tqdm                      4.52.0                   pypi_0    pypi
transformers              3.4.0                    pypi_0    pypi
typing-extensions         3.7.4.3                  pypi_0    pypi
urllib3                   1.25.11                  pypi_0    pypi
wheel                     0.35.1             pyhd3eb1b0_0  
xz                        5.2.5                h7b6447c_0  
zipp                      3.4.0                    pypi_0    pypi
zlib                      1.2.11               h7b6447c_3  

> pip list
Package            Version             Location
------------------ ------------------- ---------------------------------------------------------
attrs              20.3.0
certifi            2020.6.20
chardet            3.0.4
click              7.1.2
demjson            2.2.4
editdistance       0.5.3
fasttext           0.9.1
filelock           3.0.12
future             0.18.2
gitdb              4.0.5
GitPython          3.1.0
idna               2.10
importlib-metadata 2.0.0
iniconfig          1.1.1
joblib             0.17.0
lmdb               0.98
mmf                1.0.0rc12           /home/<USERNAME>/Dropbox/PERSONAL/Documents/Metis/git/my_mmf
nltk               3.4.5
numpy              1.19.4
omegaconf          2.0.1rc4
packaging          20.4
Pillow             8.0.1
pip                20.2.4
pluggy             0.13.1
protobuf           3.14.0
py                 1.9.0
pybind11           2.6.1
pyparsing          2.4.7
pytest             6.1.2
PyYAML             5.3.1
regex              2020.11.13
requests           2.23.0
sacremoses         0.0.43
scikit-learn       0.23.2
scipy              1.5.4
sentencepiece      0.1.94
setuptools         50.3.1.post20201107
six                1.15.0
sklearn            0.0
smmap              3.0.4
termcolor          1.1.0
threadpoolctl      2.1.0
tokenizers         0.9.2
toml               0.10.2
torch              1.6.0
torchtext          0.5.0
torchvision        0.7.0
tqdm               4.52.0
transformers       3.4.0
typing-extensions  3.7.4.3
urllib3            1.25.11
wheel              0.35.1
zipp               3.4.0

[hackgoofer]

Hi, thanks for using mmf!

Do you mind doing a which pytest to make sure that your pytest is downloaded in the conda site packages?

Also, please paste the output of: python -m torch.utils.collect_env for us to better assist you.

[michael-a-green]

~> which pytest
/home/<USERNAME>/anaconda3_2020_07/envs/mmf2_venv/bin/pytest

~> python -m torch.utils.collect_env
Collecting environment information...
PyTorch version: 1.6.0
Is debug build: No
CUDA used to build PyTorch: 10.2

OS: openSUSE Tumbleweed
GCC version: (SUSE Linux) 9.2.1 20200109 [gcc-9-branch revision 280039]
CMake version: version 3.16.2

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: TITAN RTX
Nvidia driver version: 440.33.01
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.19.4
[pip3] torch==1.6.0
[pip3] torchtext==0.5.0
[pip3] torchvision==0.7.0
[conda] numpy                     1.19.4                   pypi_0    pypi
[conda] torch                     1.6.0                    pypi_0    pypi
[conda] torchtext                 0.5.0                    pypi_0    pypi
[conda] torchvision               0.7.0                    pypi_0    pypi

[apsdehal]

The error messages might be caused due to platform and GPU differences. Did you try a sample run to see if you can reproduce the results?

[michael-a-green]

The error messages might be caused due to platform and GPU differences. Did you try a sample run to see if you can reproduce the results?

Hi apsdehal,

Not sure what you mean by "sample run." How do I run that?

[hackgoofer]

Looks like your pytest is not installed in the mmf env, but rather mmf2_venv env, try install pytest in the mmf env and run pytest in the mmf env, and see if the error repros.

The sample run @apsdehal was talking about was probably the first step under here to make sure that mmf runs.