PCCproject / PCC-Uspace

The userspace implementations of PCC.
BSD 3-Clause "New" or "Revised" License
119 stars 61 forks source link

Cannot find the correct Python interpreter using a trained agent #10

Closed AirManH closed 5 years ago

AirManH commented 5 years ago

Env

I create a new conda env pcc, and install all modules required.

pip list prints that:

Package              Version  
-------------------- ---------
absl-py              0.7.1    
astor                0.8.0    
certifi              2019.6.16
cloudpickle          1.2.1    
cycler               0.10.0   
future               0.17.1   
gast                 0.2.2    
google-pasta         0.1.7    
grpcio               1.16.1   
gym                  0.14.0   
h5py                 2.9.0    
joblib               0.13.2   
Keras-Applications   1.0.8    
Keras-Preprocessing  1.1.0    
kiwisolver           1.1.0    
Markdown             3.1.1    
matplotlib           3.1.1    
mkl-fft              1.0.14   
mkl-random           1.0.2    
mkl-service          2.0.2    
mpi4py               3.0.2    
numpy                1.16.4   
opencv-python        4.1.0.25 
pandas               0.25.1   
pip                  19.2.2   
protobuf             3.8.0    
pyglet               1.3.2    
pyparsing            2.4.2    
python-dateutil      2.8.0    
pytz                 2019.2   
scipy                1.3.1    
setuptools           41.0.1   
six                  1.12.0   
stable-baselines     2.7.0    
tensorboard          1.14.0   
tensorflow           1.14.0   
tensorflow-estimator 1.14.0   
termcolor            1.1.0    
Werkzeug             0.15.5   
wheel                0.33.4   
wrapt                1.11.2

Problem

In terminal A(Server)

conda activate pcc
pcc_uspace_path='/home/mininet/src/PCC-Uspace'
server_port='9000'
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"${pcc_uspace_path}/src/core/"
"${pcc_uspace_path}/src/app/pccserver" recv ${server_port}
Starting sending rate = 2.24e+06
server is ready at port: 9000

In terminal B(Client)

conda activate pcc
pcc_uspace_path='/home/mininet/src/PCC-Uspace'
pcc_rl_path='/home/mininet/src/PCC-RL'
model_path='/home/mininet/src/pcc-model/20190725/'
# ls "${model_path}" outputs that:
# saved_model.pbtxt  variables
server_ip='127.0.0.1'
server_port='9000'
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"${pcc_uspace_path}/src/core/"
"${pcc_uspace_path}/src/app/pccclient" send ${server_ip} ${server_port} --pcc-rate-control=python -pyhelper=loaded_client -pypath="${pcc_rl_path}/src/udt-plugins/testing/" --history-len=10 --pcc-utility-calc=linear --model-path="${model_path}"

It prints:

Starting sending rate = 2.24e+06
ERROR: Could not load python module: loaded_client
Traceback (most recent call last):
  File "/home/mininet/src/PCC-RL/src/udt-plugins/testing/loaded_client.py", line 26, in <module>
    from common import sender_obs
  File "/home/mininet/src/PCC-RL/src/common/sender_obs.py", line 15, in <module>
    import numpy as np
ImportError: No module named 'numpy'

What I have tried

It seems that CPython can not find the correct Python interpreter.

In file PCC-Uspace/src/pcc/rate_control/pcc_python_rc.cpp line 8-12, I find:

void PccPythonRateController::InitializePython() {
    Py_Initialize();
    PyRun_SimpleString("import sys");

    std::stringstream set_argv_ss;

If I modify this file to

    Py_Initialize();
    PyRun_SimpleString("import sys");
    PyRun_SimpleString("print(sys.path)");  // +

In terminal B, I get

['/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-x86_64-linux-gnu', '/usr/lib/python3.5/lib-dynload', '/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages']

If I modify this file to

    Py_SetPythonHome(L"/home/mininet/miniconda3/envs/pcc/");  // +
    Py_Initialize();
    PyRun_SimpleString("import sys");

or

    Py_SetProgramName(L"/home/mininet/miniconda3/envs/pcc/bin/python");  // +
    Py_Initialize();
    PyRun_SimpleString("import sys");

B always prints:

Starting sending rate = 2.24e+06
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'

Current thread 0x00007fbc435c7780 (most recent call first):
Aborted
AirManH commented 5 years ago

Do not use conda env. Just /usr/bin/pip3 install tensorflow solved my problem