Failed to install client requirements.txt for client Unit testing

[brajabasi]

Hello Sir as per client unit testing requirement, python-nose is required, but it is breaking while installing requirements.txt

facebook@facebook-ThinkPad-T450s:~/fbcode/CommunityCellularManager/client$ sudo pip install -r requirements.txt The directory '/home/facebook/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/home/facebook/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Requirement already satisfied: argparse==1.2.1 in /usr/lib/python2.7 (from -r requirements.txt (line 1)) Collecting astroid==1.3.6 (from -r requirements.txt (line 2)) Downloading astroid-1.3.6-py2.py3-none-any.whl (182kB) 100% |████████████████████████████████| 184kB 170kB/s Collecting Babel==1.3 (from -r requirements.txt (line 3)) Downloading Babel-1.3.tar.gz (3.4MB) 100% |████████████████████████████████| 3.4MB 81kB/s Collecting coverage==3.7.1 (from -r requirements.txt (line 4)) Downloading coverage-3.7.1.tar.gz (284kB) 100% |████████████████████████████████| 286kB 84kB/s Collecting docopt==0.6.2 (from -r requirements.txt (line 5)) Downloading docopt-0.6.2.tar.gz Collecting eno==0.0.17 (from -r requirements.txt (line 6)) Downloading eno-0.0.17.tar.gz Collecting envoy==0.0.3 (from -r requirements.txt (line 7)) Downloading envoy-0.0.3-py2.py3-none-any.whl Collecting flup==1.0.3.dev-20110405 (from -r requirements.txt (line 8)) Downloading flup-1.0.3.dev-20110405.tar.gz (57kB) 100% |████████████████████████████████| 61kB 65kB/s Collecting humanize==0.5.1 (from -r requirements.txt (line 9)) Downloading humanize-0.5.1.tar.gz Collecting itsdangerous==0.22 (from -r requirements.txt (line 10)) Downloading itsdangerous-0.22.tar.gz Collecting logilab-common==0.63.2 (from -r requirements.txt (line 11)) Downloading logilab-common-0.63.2.tar.gz (196kB) 100% |████████████████████████████████| 204kB 47kB/s Collecting meld3==1.0.2 (from -r requirements.txt (line 12)) Downloading meld3-1.0.2-py2.py3-none-any.whl Collecting mock==1.0.1 (from -r requirements.txt (line 13)) Downloading mock-1.0.1.zip (861kB) 100% |████████████████████████████████| 870kB 59kB/s Collecting netifaces==0.8 (from -r requirements.txt (line 14)) Downloading netifaces-0.8.tar.gz Collecting nose==1.3.4 (from -r requirements.txt (line 15)) Downloading nose-1.3.4-py2-none-any.whl (154kB) 100% |████████████████████████████████| 163kB 70kB/s Collecting openbts==0.1.7 (from -r requirements.txt (line 16)) Downloading openbts-0.1.7.tar.gz Collecting Paste==1.7.5.1 (from -r requirements.txt (line 17)) Downloading Paste-1.7.5.1.tar.gz (523kB) 100% |████████████████████████████████| 532kB 107kB/s Collecting phonenumbers==6.2.0 (from -r requirements.txt (line 18)) Downloading phonenumbers-6.2.0.tar.gz (2.1MB) 100% |████████████████████████████████| 2.1MB 133kB/s Collecting psutil==3.1.1 (from -r requirements.txt (line 19)) Downloading psutil-3.1.1.tar.gz (247kB) 100% |████████████████████████████████| 256kB 187kB/s Collecting psycopg2==2.6 (from -r requirements.txt (line 20)) Downloading psycopg2-2.6.tar.gz (367kB) 100% |████████████████████████████████| 368kB 25kB/s Complete output from command python setup.py egg_info: running egg_info creating pip-egg-info/psycopg2.egg-info writing pip-egg-info/psycopg2.egg-info/PKG-INFO writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt' Error: pg_config executable not found.

Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:

    python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-Q7Xudx/psycopg2/

[9muir]

pg_config being missing probably indicates that Postgres is not (correctly) installed. Can you check that please?

[brajabasi]

I installed the postgresql and set it to PATH, but still getting error. Tried this command : python setup.py build_ext --pg-config '/home/facebook/.linuxbrew/Cellar/postgresql/9.6.1/bin/pg_config'

Getting this error: error: option --pg-config not recognized I have printed the $PATH with pg_config, but it is still failing to install psycopg2

writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt' Error: pg_config executable not found.

Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:

    python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-otdiqF/psycopg2/ facebook@facebook-ThinkPad-T450s:~/fbcode/CommunityCellularManager/client$ python setup.py build_ext --pg-config '/home/facebook/.linuxbrew/Cellar/postgresql/9.6.1/bin/pg_config' usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help

error: option --pg-config not recognized facebook@facebook-ThinkPad-T450s:~/fbcode/CommunityCellularManager/client$ echo $PATH /home/facebook/bin:/home/facebook/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/facebook/.linuxbrew/Cellar/postgresql/9.6.1/bin facebook@facebook-ThinkPad-T450s:~/fbcode/CommunityCellularManager/client$

[9muir]

How did you install the BTS-related package, e.g., endaga-openbts? There should be a dependency from python-endaga-core to Postgres and related components that would take care of all this stuff.

I suspect this is a case of the instructions in the README being targeted at people wanting to do active development rather than people that just want to setup the client. For the latter use case it's generally much simpler to skip all the steps required for building packages locally and instead just install from our package repos: from inside the VM run (as root or via sudo) apt-get install endaga-openbts.

Even if you want to actively develop the code, or need the latest version of the source, that's a good way to ensure all the base dependencies are satisfied before switching to local packages.

[brajabasi]

Follwed these steps from CLI:-

1) fab dev package 2) vagrant ssh openbts Inside OpenBTS VM:-

1 cd endaga-packages/
2  ls
3  cd ..
4  exit
5  cd endaga-packages/
6  ls
7  gdebi endaga-openbts_0.7.11_all.deb 
8  sudo gdebi endaga-openbts_0.7.11_all.deb 

[9muir]

That should have worked. What does dpkg -l | grep postgres show?

[brajabasi]

vagrant@endaga-client-openbts:~$ dpkg -l | grep postgres ii postgresql 9.3+154ubuntu1 all object-relational SQL database (supported version) ii postgresql-9.3 9.3.15-0ubuntu0.14.04 i386 object-relational SQL database, version 9.3 server ii postgresql-client-9.3 9.3.15-0ubuntu0.14.04 i386 front-end programs for PostgreSQL 9.3 ii postgresql-client-common 154ubuntu1 all manager for multiple PostgreSQL client versions ii postgresql-common 154ubuntu1 all PostgreSQL database-cluster manager ii postgresql-contrib 9.3+154ubuntu1 all additional facilities for PostgreSQL (supported version) ii postgresql-contrib-9.3 9.3.15-0ubuntu0.14.04 i386 additional facilities for PostgreSQL

[9muir]

Hmmm, that all looks good, but I noticed up above there is some mention of Postgres 9.6.1:

python setup.py build_ext --pg-config 'home/facebook/.linuxbrew/Cellar/postgresql/9.6.1/bin/pg_config'

You shouldn't need to use linuxbrew to install packages into the client VM, I suspect there may be conflicts between different versions. Can you create a clean VM that doesn't have any of that stuff and try again? Just the following steps should suffice to get a basic client VM instantiated:

vagrant up openbts
vagrant ssh openbts
apt-get install endaga-openbts # inside the VM

[brajabasi]

I followed above steps and then tried this command outside VM facebook@facebook-ThinkPad-T450s:~/fbcode/CommunityCellularManager/client$ sudo pip install -r requirements.txt

But getting the same error:- Please add the directory containing pg_config to the PATH or specify the full executable path with the option:

I didn't use linuxbrew inside vm, but outside VM to install postgres. postgress was showing some dependency on linuxbrew. Please let me know if I'm doing any mistake

[9muir]

You don't need to install Postgres outside the VM. The only requirements in the host ('outside the VM') are Vagrant, VirtualBox (or some other Vagrant backend) and Ansible. Fabric is required if you want to rebuild the binary packages from source, but you can work on the code and test it without doing that.

[9muir]

Here are some steps that I usually take to make testing and debugging easier:

Increase the logger's level of verbosity - run log_level set global notice inside the VM

Specify the BTS type to be 'fakebts', which will prevent all the OpenBTS services being stopped and started - run endaga_db_set bts.type fake inside the VM

Disable gprsd - supervisorctl remove gprsd

You have to restart the agent after running either of the first two commands.

[brajabasi]

I'm able to test fake_sms vagrant@endaga-client-openbts:~$ fake_phone_client IMSI123451234512345 5065:Endaga-Fake-Phone: sms 101 test 5065:Endaga-Fake-Phone: RESPONSE: 202

Targets pending: - 1) Nosetest for client 2) BTS tower getting registered, but not changing to ACTIVE

[9muir]

What are the contents of /var/log/syslog after increasing the logging level to 'notice'? That will help us troubleshoot 2)

[9muir]

To figure out 1), please can you create a clean openbts VM. Then capture the outputs of the following commands (maybe use your terminal app's facility to do so, or use tmux or screen):

1. apt-get install endaga-openbts
2. dpkg -l | grep postgres
3. pip install -r requirements.txt

Thanks

[brajabasi]

Please find the attached log of commands executed: Command_Log_Openbts.txt

[9muir]

That looks okay to me:

Successfully installed astroid coverage eno logilab-common meld3 mock nose openbts pylint six Twisted Adafruit-BBIO Flask Jinja2 MarkupSafe Werkzeug pyserial python-gsmmodem

Are you able to run nosetests now? I can see from the logs that Postgres was successfully installed too, so hopefully there are no longer any issues with installation.

[brajabasi]

Massive Thanks Steve. Following two unit tests passed, $ nosetests core.tests.billing_tests $ nosetests core.tests.billing_tests:GetCostTest This two test cases failed $ nosetests core.tests.billing_tests:GetCostTest.test_inbound_call $ nosetests fake_phone_integration_tests Please find the attached log

UnitTest.txt

[9muir]

yes, we're aware that some of the unit tests don't currently pass. Since the specific issue of not being able to install nosetests has been resolved I'm going to close this issue, please open a new one for particular tests (or preferably, create a PR with a fix to make them pass).