Error in project setup

[LDiazN]

Issue description

When i try to set up the project, the last step fails with the following error:

nox > Running session tests-3.8
nox > Creating virtualenv using python3.8 in .nox/tests-3-8
nox > Command /usr/bin/python3 -m virtualenv /home/luis/dev/C4V/c4v-py/.nox/tests-3-8 -p python3.8 failed with exit code 1:
ModuleNotFoundError: No module named 'virtualenv.seed.via_app_data'
nox > Session tests-3.8 failed.

when running: nox -s tests

Steps to reproduce the issue

1. Follow installation guide, it will crash at step 8

What's the expected result?

• Every test works just fine

What's the actual result?

• This error:

  nox > Running session tests-3.8
  nox > Creating virtualenv using python3.8 in .nox/tests-3-8
  nox > Command /usr/bin/python3 -m virtualenv /home/luis/dev/C4V/c4v-py/.nox/tests-3-8 -p python3.8 failed with exit code 1:
  ModuleNotFoundError: No module named 'virtualenv.seed.via_app_data'
  nox > Session tests-3.8 failed.

Additional details

• After installing nox, virtualenv will crash with this error:

  ModuleNotFoundError: No module named 'virtualenv.seed.via_app_data'

Reiinstalling virtualenv will fix it, but then nox it's uninstalled too. When reinstalling nox, the same error happens again

(i found this template report here )

[marianelamin]

Hey! I don't recall using virtual env. Did you try with pyenv? Also, just to we can help you better, could you tell us which OS you are using? Nice template, I know @m4r4c00ch0 is or will be working on them.

[dieko95]

@LDiazN Have you run poetry install? It looks like that nox can't find the virtual environment. Within c4v we're using poetry as package manager. These are some docs on how to use constarint:

• How to install Poetry
• How to install dependencies within a project: TL;DR In your terminal, cd to your repo's location and execute poetry install.

[LDiazN]

Hey! I don't recall using virtual env. Did you try with pyenv? Also, just to we can help you better, could you tell us which OS you are using?

@marianelamin I have both virtualenv and pyenv currently installed. I set up pyenv with pyenv local 3.8.2 before running nox tests, but i'm still having the same issue.

Oh and i forgot about my OS in template details, sorry about it. i use windows but i work with WSL, using Ubuntu 20.04

@LDiazN Have you run poetry install? It looks like that nox can't find the virtual environment. Within c4v we're using poetry as package manager. These are some docs on how to use constarint:

• How to install Poetry
• How to install dependencies within a project: TL;DR In your terminal, cd to your repo's location and execute poetry install.

@dieko95 I did so, but still not working. I think there may be some weird thing conflicting between pyenv and my previous installation of virtualenv

[LDiazN]

Ok i managed to make it work. I workaround it by creating a virtualenvironment pointing to a python version installed & managed by pyenv:

• i ran pyenv install 3.8.2
• i looked for this version's binary in /home/myuser/.pyenv/versions/3.8.2/bin/python
• i ran: mkvirtualenv -p=/home/myuser/.pyenv/versions/3.8.2/bin/python c4venv
• activate recently created virtualenv with workon c4venc

(P.S.: i manage my virtualenvs with virtualenvwrapper)

then after calling poetry install again and installing required dependencies, i run nox and i no longer get the previous error, but now i'm getting a new one that's actually not an error, but some test not passing for some reason:

----------- coverage: platform linux, python 3.8.2-final-0 -----------
Name                               Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------------------
src/c4v/__init__.py                    1      0      0      0   100%
src/c4v/data/angostura_loader.py      27      9      6      3    64%   49->50, 50-51, 52->53, 53-54, 80, 100->103, 103-108
src/c4v/data/baseline_models.py      105      7     26      1    94%   167, 170-176, 197->198, 198
src/c4v/data/data_loader.py           78      4     16      4    91%   77, 130->132, 132, 264->272, 272, 284->289, 317->328, 328
src/c4v/data/tweet_loader.py          23      8      6      2    59%   60->61, 61-71, 75->exit, 90-97
------------------------------------------------------------------------------
TOTAL                                234     28     54     10    86%

=============================================== short test summary info ================================================
FAILED tests/data/test_angostura_loader.py::test_angostura - FileNotFoundError: [Errno 2] No such file or directory: ...
FAILED tests/data/test_tweet_loader.py::test_tweet_loader - FileNotFoundError: [Errno 2] No such file or directory: '...
======================================= 2 failed, 3 passed, 17 warnings in 9.64s =======================================
nox > Command pytest --cov -m not e2e ./tests failed with exit code 1
nox > Session tests-3.8 failed.

[LDiazN]

I should have included the errors messages, here they are:

Error 1:

______________________________________________________________________________________________ test_angostura ______________________________________________________________________________________________

    def test_angostura():

        test = AngosturaLoader()
>       df = test.create_query(
            "SELECT * FROM `event-pipeline.angostura.sinluz_rawtweets` LIMIT 1"
        )

tests/data/test_angostura_loader.py:7: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/c4v/data/angostura_loader.py:101: in create_query
    self.create_connection()
src/c4v/data/angostura_loader.py:76: in create_connection
    credentials = service_account.Credentials.from_service_account_file(
.nox/tests-3-8/lib/python3.8/site-packages/google/oauth2/service_account.py:227: in from_service_account_file
    info, signer = _service_account_info.from_filename(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

filename = 'angostura_connection.json', require = ['client_email', 'token_uri']

    def from_filename(filename, require=None):
        """Reads a Google service account JSON file and returns its parsed info.

        Args:
            filename (str): The path to the service account .json file.
            require (Sequence[str]): List of keys required to be present in the
                info.

        Returns:
            Tuple[ Mapping[str, str], google.auth.crypt.Signer ]: The verified
                info and a signer instance.
        """
>       with io.open(filename, "r", encoding="utf-8") as json_file:
E       FileNotFoundError: [Errno 2] No such file or directory: 'angostura_connection.json'

.nox/tests-3-8/lib/python3.8/site-packages/google/auth/_service_account_info.py:72: FileNotFoundError 

Error 2:

____________________________________________________________________________________________ test_tweet_loader _____________________________________________________________________________________________

    def test_tweet_loader():

        test = TweetLoader()
        test_query = test.load_sinluz_tweets(batch_size=20, batch_number=2)

        # Loop over queried batches
>       for text_batch in test_query:

tests/data/test_tweet_loader.py:10: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/c4v/data/tweet_loader.py:89: in load_sinluz_tweets
    batch_df = self.angostura_loader.create_query(query)
src/c4v/data/angostura_loader.py:101: in create_query
    self.create_connection()
src/c4v/data/angostura_loader.py:76: in create_connection
    credentials = service_account.Credentials.from_service_account_file(
.nox/tests-3-8/lib/python3.8/site-packages/google/oauth2/service_account.py:227: in from_service_account_file
    info, signer = _service_account_info.from_filename(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

filename = 'angostura_connection.json', require = ['client_email', 'token_uri']

    def from_filename(filename, require=None):
        """Reads a Google service account JSON file and returns its parsed info.

        Args:
            filename (str): The path to the service account .json file.
            require (Sequence[str]): List of keys required to be present in the
                info.

        Returns:
            Tuple[ Mapping[str, str], google.auth.crypt.Signer ]: The verified
                info and a signer instance.
        """
>       with io.open(filename, "r", encoding="utf-8") as json_file:
E       FileNotFoundError: [Errno 2] No such file or directory: 'angostura_connection.json'

.nox/tests-3-8/lib/python3.8/site-packages/google/auth/_service_account_info.py:72: FileNotFoundError

[dieko95]

@LDiazN It looks that you don't have the key to connect to angostura, its name is angostura_connection.json. I will share it with you in slack