tkn-tub / ns3-gym

ns3-gym - The Playground for Reinforcement Learning in Networking Research
GNU General Public License v2.0
521 stars 197 forks source link

opengym module cannot be built while others are built successfully #32

Closed Forsworns closed 4 years ago

Forsworns commented 4 years ago

I built the repo on Ubuntu16.04 LTS and encountered the same error log as a closed issue:

fatal error: ns3/opengym-module.h: No such file or directory compilation terminated.

But I have upgrade to the latest protobuf manually and run ./waf clean each time before I run ./waf configure.

here is my ./waf configure output,

Setting top to                           : /mnt/sdb1/ns3-gym
Setting out to                           : /mnt/sdb1/ns3-gym/build
Checking for 'gcc' (C compiler)          : /usr/bin/gcc
Checking for cc version                  : 6.5.0
Checking for 'g++' (C++ compiler)        : /usr/bin/g++
Checking for compilation flag -Wl,--soname=foo support : ok
Checking for compilation flag -std=c++11 support       : ok
Checking for program 'python'                          : /usr/bin/python3
Checking for python version >= 2.3                     : 3.5.2
python-config                                          : /usr/bin/python3-config
Asking python-config for pyembed '--cflags --libs --ldflags' flags : yes
Testing pyembed configuration                                      : yes
Asking python-config for pyext '--cflags --libs --ldflags' flags   : yes
Testing pyext configuration                                        : yes
Checking for compilation flag -fvisibility=hidden support          : ok
Checking for compilation flag -Wno-array-bounds support            : ok
Checking for pybindgen location                                    : not found
Checking for python module 'pybindgen'                             : 0.21.0
Checking for pybindgen version                                     : 0.21.0
Checking for code snippet                                          : yes
Checking for types uint64_t and unsigned long equivalence          : no
Checking for code snippet                                          : no
Checking for types uint64_t and unsigned long long equivalence     : yes
Checking for the apidefs that can be used for Python bindings      : gcc-LP64
Checking for internal GCC cxxabi                                   : complete
Checking for python module 'pygccxml'                              : not found
Checking boost includes                                            : 1_58
Checking boost libs                                                : ok
Checking for boost linkage                                         : ok
Checking for click location                                        : not found
Checking for program 'pkg-config'                                  : /usr/bin/pkg-config
Checking for 'gtk+-3.0'                                            : not found
Checking for 'libxml-2.0'                                          : not found
checking for uint128_t                                             : not found
checking for __uint128_t                                           : yes
Checking high precision implementation                             : 128-bit integer (default)
Checking for header stdint.h                                       : yes
Checking for header inttypes.h                                     : yes
Checking for header sys/inttypes.h                                 : not found
Checking for header sys/types.h                                    : yes
Checking for header sys/stat.h                                     : yes
Checking for header dirent.h                                       : yes
Checking for header stdlib.h                                       : yes
Checking for header signal.h                                       : yes
Checking for header pthread.h                                      : yes
Checking for header stdint.h                                       : yes
Checking for header inttypes.h                                     : yes
Checking for header sys/inttypes.h                                 : not found
Checking for library rt                                            : yes
Checking for header sys/ioctl.h                                    : yes
Checking for header net/if.h                                       : yes
Checking for header net/ethernet.h                                 : yes
Checking for header linux/if_tun.h                                 : yes
Checking for header netpacket/packet.h                             : yes
Checking for NSC location                                          : not found
Checking for OpenFlow location                                     : not found
Checking for library zmq                                           : not found
Checking for library protobuf                                      : yes
Checking for program 'protoc'                                      : /usr/local/bin/protoc
Checking for protoc version                                        : 3.12.3
Checking for 'sqlite3'                                             : not found
Checking for header linux/if_tun.h                                 : yes
Checking for python module 'gi'                                    : 3.20.0
Checking for python module 'gi.repository.GObject'                 : ok
Checking for python module 'cairo'                                 : ok
Checking for python module 'pygraphviz'                            : not found
Checking for python module 'gi.repository.Gtk'                     : ok
Checking for python module 'gi.repository.Gdk'                     : ok
Checking for python module 'gi.repository.Pango'                   : ok
Checking for python module 'gi.repository.GooCanvas'               : not found
Checking for program 'sudo'                                        : /usr/bin/sudo
Checking for program 'valgrind'                                    : not found
Checking for 'gsl'                                                 : not found
python-config                                                      : not found
Checking for compilation flag -fstrict-aliasing support            : ok
Checking for compilation flag -fstrict-aliasing support            : ok
Checking for compilation flag -Wstrict-aliasing support            : ok
Checking for compilation flag -Wstrict-aliasing support            : ok
Checking for program 'doxygen'                                     : not found
---- Summary of optional NS-3 features:
Build profile                 : debug
Build directory               :
BRITE Integration             : not enabled (BRITE not enabled (see option --with-brite))
DES Metrics event collection  : not enabled (defaults to disabled)
Emulation FdNetDevice         : enabled
Examples                      : not enabled (defaults to disabled)
File descriptor NetDevice     : enabled
GNU Scientific Library (GSL)  : not enabled (GSL not found)
Gcrypt library                : not enabled (libgcrypt not found: you can use libgcrypt-config to find its location.)
GtkConfigStore                : not enabled (library 'gtk+-3.0 >= 3.0' not found)
MPI Support                   : not enabled (option --enable-mpi not selected)
NS-3 Click Integration        : not enabled (nsclick not enabled (see option --with-nsclick))
NS-3 OpenFlow Integration     : not enabled (OpenFlow not enabled (see option --with-openflow))
Network Simulation Cradle     : not enabled (NSC not found (see option --with-nsc))
PlanetLab FdNetDevice         : not enabled (PlanetLab operating system not detected (see option --force-planetlab))
PyViz visualizer              : not enabled (Missing python modules: pygraphviz, gi.repository.GooCanvas)
Python API Scanning Support   : not enabled (Missing 'pygccxml' Python module)
Python Bindings               : enabled
Real Time Simulator           : enabled
SQlite stats data output      : not enabled (library 'sqlite3' not found)
Tap Bridge                    : enabled
Tap FdNetDevice               : enabled
Tests                         : not enabled (defaults to disabled)
Threading Primitives          : enabled
Use sudo to set suid bit      : not enabled (option --enable-sudo not selected)
XmlIo                         : not enabled (library 'libxml-2.0 >= 2.7' not found)
'configure' finished successfully (5.253s)

And the /src/opengym/model/ns3gym/ns3gym/messages_pb2.py is not generated when I run ./waf configure before. After configuration, I run pip install ./src/opengym/model/ns3gym and get this error message "Protocol Buffer messages are missing. Please run ./waf configure to generate the file").

I removed the scratch dir and the other modules are built successfully, except opengym. Do you have any idea why it is not built? I have run ./waf clean before the cofiguration.

Modules built:
antenna                   aodv                      applications
bridge                    buildings                 config-store
core                      csma                      csma-layout
dsdv                      dsr                       energy
fd-net-device             flow-monitor              internet
internet-apps             lr-wpan                   lte
mesh                      mobility                  mpi
netanim (no Python)       network                   nix-vector-routing
olsr                      point-to-point            point-to-point-layout
propagation               sixlowpan                 spectrum
stats                     tap-bridge                test (no Python)
topology-read             traffic-control           uan
virtual-net-device        wave                      wifi
wimax

Modules not built (see ns-3 tutorial for explanation):
brite                     click                     openflow
opengym                   visualizer
pgawlowicz commented 4 years ago

seems that zmq library is not installed

Forsworns commented 4 years ago

seems that zmq library is not installed

Thanks. I directly used apt-get to install the zmq as the README. And it told me that the libzmq5 has been installed so I didn't notice the library zmq is not found during building.

Note, selecting 'libzmq3-dev' instead of 'libzmq5-dev'
libzmq5 is already the newest version (4.1.4-7ubuntu0.1).
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libzmq3-dev : Depends: libzmq5 (= 4.1.4-7) but 4.1.4-7ubuntu0.1 is to be installed
E: Unable to correct problems, you have held broken packages.

So I degrade the libzmq5 to 4.1.4-7 and runsudo apt install libzmq5-dev again. The apt-get told me that the libzmq3-dev is selected instead of libzmq5-dev. I check the packages in the ubuntu source. They didn't supply the libzmq5-dev package. It's kinda weird... Do you use another apt source? Like the PPA of protobuf.

And I also check the source codes of the team zeromq on the Github, I cannot find a repo labeled by version 5.


Building your source with libzmq3-dev throws the following errors.

The first one is that the ByteSize() is deprecated, so I replace them to ByteSizeLong() as recommended in the opengym_interface.cc. It works. I think maybe you should replace them, too, since it is an error from google::protobuf.

../src/opengym/model/opengym_interface.cc:258:69: error: ‘int google::protobuf::MessageLite::ByteSize() const’ is deprecated: Please use ByteSizeLong() instead [-Werror=deprecated-declarations]
   envStateMsg.SerializeToArray(request.data(), envStateMsg.ByteSize());

The second one maybe comes from the libzmq3-dev.

I just found this error is the same as another opening issue.

../src/opengym/model/opengym_interface.cc: In member function ‘void ns3::OpenGymInterface::Init()’:
../src/opengym/model/opengym_interface.cc:192:36: error: ‘zmq::send_flags’ has not been declared
   m_zmq_socket.send (request, zmq::send_flags::none);
                                    ^~~~~~~~~~
../src/opengym/model/opengym_interface.cc:197:41: error: ‘zmq::recv_flags’ has not been declared
   (void) m_zmq_socket.recv (reply, zmq::recv_flags::none);
                                         ^~~~~~~~~~
../src/opengym/model/opengym_interface.cc: In member function ‘void ns3::OpenGymInterface::NotifyCurrentState()’:
../src/opengym/model/opengym_interface.cc:259:36: error: ‘zmq::send_flags’ has not been declared
   m_zmq_socket.send (request, zmq::send_flags::none);
                                    ^~~~~~~~~~
../src/opengym/model/opengym_interface.cc:264:41: error: ‘zmq::recv_flags’ has not been declared
   (void) m_zmq_socket.recv (reply, zmq::recv_flags::none);

The zmq::recv_flags is not declared. Do you have any ideas about this?

Forsworns commented 4 years ago

In fact, I'm working on using DRL in ns3 these days, too. So I really appreciate your marvellous jobs on creating gym environments for ns3.

Since I cannot build your files these days, I found another interesting way to use the existing python DRL source codes in ns3. It origins from the document of the waf tool.

I simply modified the wscript of the ns3, such that the python scripts can be embedded in to the cpp files as described in Python document.

def options(opt):
    # options provided by the modules
    opt.load('compiler_c')
    opt.load('compiler_cxx')
    opt.load('cflags')
    opt.load('gnu_dirs')
    opt.load('python') # add this command
    # other commands

def create_ns3_program(bld, name, dependencies=('core',)):
    program = bld(features='cxx cxxprogram pyembed') # modify here
    program.is_ns3_program = True
    program.name = name

:) But sure, if I can use your framework, my code would be much more readable, considering that embedding python files into the cpp is too ugly, XD

pgawlowicz commented 4 years ago

I have recently applied a pull request that replaced the deprecated zmq functions with the new ones: https://github.com/tkn-tub/ns3-gym/commit/19bfe0a583e641142609939a090a09dfc63a095f

Maybe you can revert this commit and check.