ParthPratim commented 3 years ago

ns3 version : ns-3.32
DCE version : commit 38b624203ea2b888662c1f4edadbd2c214ecb3f5
Operating system/version : Ubuntu 20.04.1 LTS
Libc version : ldd (Ubuntu GLIBC 2.31-0ubuntu9) 2.31
compiler version : gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

Description of the problem

Branch to reproduce error: ubuntu-20.04-changes On running the dce-iperf script on Ubuntu-20.04:

./waf --run dce-iperf

It throws the following error with Glibc:

Waf: Entering directory `/home/kiit/ltest3/bake/source/ns-3-dce/build'
Building library netlink
Building library dce
Building library dce-test
Building library dce-test-netlink
[302/322] Creating build/lib/pkgconfig/libns3-dev-netlink-debug.pc
[303/322] Creating build/lib/pkgconfig/libns3-dev-dce-debug.pc
Waf: Leaving directory `/home/kiit/ltest3/bake/source/ns-3-dce/build'
Build commands will be stored in build/compile_commands.json
'build' finished successfully (1.470s)
Don't know how to configure dynamic library path for the platform 'linux'; assuming it's LD_LIBRARY_PATH.
Fatal error: glibc detected an invalid stdio handle
Command ['/home/kiit/ltest3/bake/source/ns-3-dce/build/bin/dce-iperf'] terminated with signal SIGIOT. Run it under a debugger to get more information (./waf --run <program> --command-template="gdb --args %s <args>").

output of ./waf configure

Setting top to                           : /home/kiit/ltest3/bake/source/ns-3-dce 
Setting out to                           : /home/kiit/ltest3/bake/source/ns-3-dce/build 
Checking for 'gcc' (C compiler)          : /usr/bin/gcc 
Checking for cc version                  : 9.3.0 
Checking for 'g++' (C++ compiler)        : /usr/bin/g++ 
Checking for program 'pkg-config'        : /usr/bin/pkg-config 
Checking for pkg-config version >= '0.0.0' : yes 
Checking for -Wl,--soname=foo              : yes 
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Checking for header stdint.h               : yes 
Checking for header inttypes.h             : yes 
Checking for header sys/inttypes.h         : not found 
Checking for header sys/types.h            : yes 
Checking for header sys/stat.h             : yes 
Checking for header dirent.h               : yes 
Checking for library dl                    : yes 
Checking for glibc get_cpu_features        : yes 
Checking for glibc secure_getenv           : yes 
Checking for glibc explicit_bzero          : yes 
Checking for header valgrind/valgrind.h    : not found 
Checking for header valgrind/memcheck.h    : not found 
Checking for 'gcc' (C compiler)            : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)          : /usr/bin/g++ 
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Checking for 'gcc' (C compiler)            : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)          : /usr/bin/g++ 
Could not find lib*-core-debug
Could not find lib*-core-debug
Checking for header hook-manager.h         : not found 
Checking for program 'valgrind'            : not found 
Checking for program 'doxygen'             : /usr/bin/doxygen 
Checking for header netinet/sctp.h         : not found 
Checking for 'gcc' (C compiler)            : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)          : /usr/bin/g++ 
Checking for program 'python'              : /usr/bin/python3 
Checking for python version >= 2.3         : 3.8.5 
python-config                              : /usr/bin/python3-config 
Asking python-config for pyembed '--cflags --libs --ldflags --embed' flags : yes 
Testing pyembed configuration                                              : yes 
Asking python-config for pyext '--cflags --libs --ldflags' flags           : yes 
Testing pyext configuration                                                : yes 
Checking for pybindgen location                                            : ../pybindgen (guessed) 
Checking for python module 'pybindgen'                                     : 0.21.0.post8+g3a8e313 
Checking for pybindgen version                                             : 0.21.0.post8+g3a8e313 
PyBindGen                                                                  : ok 
Checking for code snippet                                                  : yes 
Checking for types uint64_t and unsigned long equivalence                  : no 
Checking for code snippet                                                  : no 
Checking for types uint64_t and unsigned long long equivalence             : yes 
Checking for the apidefs that can be used for Python bindings              : gcc-LP64 
Checking for internal GCC cxxabi                                           : complete 
Checking for python module 'pygccxml'                                      : not found 
---- Summary of optional NS-3 features:
Static build                  : not enabled (option --enable-static not selected)
Logging                       : enabled
Assert checks                 : enabled
Code coverage                 : not enabled (option --enable-gcov not selected)
Example programs              : enabled
Test programs                 : enabled
Debug Symbols                 : enabled
ELF magic loader              : not enabled (elf-loader not found)
Aspect-based tracing          : not enabled (libaspect not found)
sctp-tools-dev                : not enabled (sctp-tools (netinet/sctp.h) not found)
Python Bindings               : enabled
Python API Scanning Support   : not enabled (Missing 'pygccxml' Python module)
'configure' finished successfully (2.696s)

Steps to reproduce

I have managed to build DCE on an Ubuntu-20.04 machine and have also mentioned the Github branch below to reproduce the same.

Clone into ubuntu-20.04-changes branch
bake.py build

Things I did to get the DCE code to build :

Cloned into ns-3.32-alignment branch
Aligned a myscripts/mt3/dce-mt3.cc to the ns-3.32 API
I identified that the libio.h and _G_config.h header files were missing from Ubuntu-20.04 systems which implemented the _IO_getc and _IO_putc functions as required in libc-ns3.h. Bug Report
I had to commend out the lines DCE(_IO_getc) and DCE(_IO_putc) from libc-ns3.h.
I had to include the header file cstdarg.h to replace _G_va_list with __gnuc_va_list.
Included the header file linux/sockios.h to enable support for SIOCGSTAMP in test/test-socket.c.
The issue with bake.py show showing libc6-dbg requirement not satisfied was thrown because it isn't looking for ld-2.31.so and can be resolved by applying the following patch. Just download this patch and run git apply bake.patch
After applying the patch run bake.py configure -e dce-linux-dev

After building DCE when I ran the dce-iperf example it throws the following error Fatal error: glibc detected an invalid stdio handle

I'm trying to work on this issue and would highly appreciate support from the DCE community.

Similar Issue : #57

mas2tg commented 3 years ago

I am actually facing the same issue with Ubuntu 18.04. Does anyone have any updates on this?

tomhenderson commented 3 years ago

@ParthPratim is looking into fixing this; he will update any progress here (collaborators would be welcome).

tomhenderson commented 3 years ago

Possibly relevant: https://sourceware.org/bugzilla/show_bug.cgi?id=20191

It seems like disabling the vtables hardening is a possible solution (if that can be done).

Another possibility is to maintain and build a patched glibc for use with DCE.

ParthPratim commented 3 years ago

I tried building glibc release 2.23 on an Ubuntu-20.04 machine but it fails with the following error:

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libselinux.so: undefined reference to `gettid@GLIBC_2.30'

I'm trying to find some way to get it to build.

I'll also try disabling vtables hardening, if that's possible.

teto commented 3 years ago

IIRC that's a documented error, as in you need to add gettid to DCE. Btw do you need selinux ? this seems like a painful dep to have with dce

ParthPratim commented 3 years ago

The error with selinux comes up when I tried to build the glibc release 2.23, not the DCE source.

I'm thinking of building the release/2.31/master branch of glibc and then commenting out the following line from vtables.c

__libc_fatal ("Fatal error: glibc detected an invalid stdio handle\n");

This line causes the DCE scripts to stop execution when it detects any tampering with the vtable.

I've generated the custom glibc as a proper system root against which DCE could be built. I'm trying to figure out which scripts and build files to change so that I could tweek the gcc build arguments to initiate build against the custom glibc and not the default glibc.

mas2tg commented 3 years ago

Thanks for looking into this. I am just wondering, does this issue prevent ns3-dce from working at all? I tried dce-udp-simple and dce-iperf, and I was wondering if all other binaries would fail as well (I believe the answer is yes).

ParthPratim commented 3 years ago

All the binaries are likely to fail because the issue arises due to stack smashing used in DCE which is why Glibc throws the _libc_fatal(...)

tomhenderson commented 3 years ago

Can someone try to add the flag '-fvtable-verify=none' to cxxflags and see if it helps? Perhaps search the ns-3-dce/wscript and add this flag wherever there is a setting of 'cxxflags' variable?

Perhaps other options are possible from here? https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html

mas2tg commented 3 years ago

Hey, I just followed your suggestion and ran into the same issue...

tomhenderson commented 3 years ago

After browsing the glibc source a bit today, I'm skeptical we can fix this with a linker option. Perhaps static linking would fix it, but I don't think we want to do that.

So it seems like the options are that we build a custom glibc for ourselves (downside-- a complicated build, reported to need 5.5GB disk space), or explore whether some other libc can be used instead: http://www.etalabs.net/compare_libcs.html

We could also ask glibc maintainers about a linker flag to disable this check (in future releases).

mas2tg commented 3 years ago

Mmm... I wonder how hard it would be to change libc to musl. Even just using a custom glibc seems hard enough for me. Thanks for your help nonetheless!

mas2tg commented 3 years ago

BTW, I am able to successfully run DCE with Ubuntu 16.04 (not sure if this was public knowledge).

tomhenderson commented 3 years ago

Yes, 16.04 is the distribution that I presently use for DCE.

ParthPratim commented 3 years ago

I just found a way to setup a custom Glibc 2.25 and make NS-3 and DCE build from it, on Ubuntu 20.04. Almost all examples work correctly(haven't tested all of them) and the error which used to pop up, doesn't show up now.

Steps to reproduce the same :

Bake Setup

git clone https://gitlab.com/ParthPratim1/bake.git cd bake git checkout glibc-build export PATH=`pwd`/build/bin${PATH:+:$PATH} export PYTHONPATH=`pwd`/build/lib${PYTHONPATH:+:$PYTHONPATH} export LD_LIBRARY_PATH=`pwd`/source/ns-3-dce/build/lib:`pwd`/source/ns-3-dev/build/lib:`pwd`/build/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH} sudo apt-get install gawk

Bake Build

python3 bake.py configure -e dce-ns3-dev python3 bake.py download -vvv python3 bake.py build -vvv

Testing DCE

cd source/ns-3-dce python3 waf --run dce-iperf It works !...(at least for now)

Source References :

https://gitlab.com/ParthPratim1/bake/-/tree/glibc-build https://github.com/ParthPratim/ns-3-dce/tree/glibc-build https://gitlab.com/ParthPratim1/ns-3-dev/-/tree/glibc-build

Successful Run :

Screenshot from 2021-02-09 11-42-59

dblessed commented 3 years ago

python3 bake.py build -vvv hi I am running dce on ubuntu 20 and I have encountered this error. please assist. the Glibc wont build

dblessed@ubuntu:~/bake$ python3 bake.py build -vvv

Building glibc-2.25 - > mkdir -p /home/dblessed/bake/build/glibc-build;cd /home/dblessed/bake/build/glibc-build;/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror mkdir -p /home/dblessed/bake/build/glibc dir=/home/dblessed/bake/source/glibc make dir=/home/dblessed/bake/build/glibc-build make: *** No targets specified and no makefile found. Stop.

Building glibc-2.25 - Problem Subprocess failed with error 2: ['make'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

ParthPratim commented 3 years ago

python3 bake.py build -vvv hi I am running dce on ubuntu 20 and I have encountered this error. please assist. the Glibc wont build

dblessed@ubuntu:~/bake$ python3 bake.py build -vvv

Building glibc-2.25 - > mkdir -p /home/dblessed/bake/build/glibc-build;cd /home/dblessed/bake/build/glibc-build;/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror mkdir -p /home/dblessed/bake/build/glibc dir=/home/dblessed/bake/source/glibc make dir=/home/dblessed/bake/build/glibc-build make: *** No targets specified and no makefile found. Stop.

Building glibc-2.25 - Problem Subprocess failed with error 2: ['make'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

The error you ran into might be a dependency problem. It turns out that your system lacks some of the dependencies used by the configure script, which is responsible for generating the Makefile.

Please let me know the output of these commands so I that can help you out better :

cd /home/dblessed/bake/build/glibc-build
/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror

dblessed commented 3 years ago

/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror

This is my reply... sorry for the long script dblessed@ubuntu:~$ cd /home/dblessed/bake/build/glibc-build dblessed@ubuntu:~/bake/build/glibc-build$ /home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking for gcc... gcc checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for readelf... readelf checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking whether g++ can link programs... yes checking for sysdeps preconfigure fragments... aarch64 alpha arm hppa i386 m68k microblaze mips nacl nios2 powerpc s390 sh sparc tile x86_64 checking whether gcc compiles in -mx32 mode by default... no

configure: running configure fragment for add-on libidn checking for -fstack-protector... yes checking for -fstack-protector-strong... yes checking for -fstack-protector-all... yes checking for assembler and linker STT_GNU_IFUNC support... yes checking for gcc attribute ifunc support... yes checking sysdep dirs... sysdeps/unix/sysv/linux/x86_64/64 sysdeps/unix/sysv/linux/x86_64 sysdeps/unix/sysv/linux/x86 sysdeps/unix/sysv/linux/wordsize-64 sysdeps/x86_64/nptl sysdeps/unix/sysv/linux sysdeps/nptl sysdeps/pthread sysdeps/gnu sysdeps/unix/inet sysdeps/unix/sysv sysdeps/unix/x86_64 sysdeps/unix sysdeps/posix sysdeps/x86_64/64 sysdeps/x86_64/fpu/multiarch sysdeps/x86_64/fpu sysdeps/x86/fpu sysdeps/x86_64/multiarch sysdeps/x86_64 sysdeps/x86 sysdeps/ieee754/ldbl-96 sysdeps/ieee754/dbl-64/wordsize-64 sysdeps/ieee754/dbl-64 sysdeps/ieee754/flt-32 sysdeps/wordsize-64 sysdeps/ieee754 sysdeps/generic checking for a BSD-compatible install... /usr/bin/install -c checking whether ln -s works... yes checking whether as is GNU as... yes checking whether ld is GNU ld... yes checking for as... as checking version of as... 2.34, ok checking for ld... ld checking version of ld... 2.34, ok checking for gnumake... no checking for gmake... no checking for make... make checking version of make... 4.2.1, ok checking for gnumsgfmt... no checking for gmsgfmt... no checking for msgfmt... no checking for makeinfo... makeinfo checking version of makeinfo... 6.7, ok checking for sed... sed checking version of sed... 4.7, ok checking for gawk... gawk checking version of gawk... 5.0.1, ok checking if gcc is sufficient to build libc... yes checking for nm... nm checking for python3... python3 configure: WARNING: These auxiliary programs are missing or incompatible versions: msgfmt some features or tests will be disabled. *** Check the INSTALL file for required versions. checking LD_LIBRARY_PATH variable... ok checking for bash... /usr/bin/bash checking for perl... /usr/bin/perl checking for install-info... /usr/bin/install-info checking for bison... bison checking version of bison... 3.5.1, ok checking for .set assembler directive... yes checking linker support for protected data symbol... yes checking for broken attribute((alias()))... no checking whether to put _rtld_local into .sdata section... no checking whether to use .ctors/.dtors header and trailer... no checking for libunwind-support in compiler... no checking whether --noexecstack is desirable for .S files... yes checking for -z combreloc... yes checking for linker that supports -z execstack... yes checking for -fpie... yes checking for --hash-style option... yes checking for sufficient default -shared layout... yes checking for GLOB_DAT reloc... yes checking linker output format... elf64-x86-64 checking for -fno-toplevel-reorder -fno-section-anchors... yes checking for -mtls-dialect=gnu2... yes checking whether cc puts quotes around section names... no checking for builtin_memset... yes checking for redirection of built-in functions... yes checking for compiler option to disable generation of FMA instructions... -ffp-contract=off checking if gcc accepts -fno-tree-loop-distribute-patterns with attribute ((optimize))... yes checking for libgd... no checking for is_selinux_enabled in -lselinux... yes checking for audit_log_user_avc_message in -laudit... no checking for cap_init in -lcap... no checking for _FORTIFY_SOURCE predefine... yes checking whether the linker provides working ehdr_start... yes checking for __builtin_trap with no external dependencies... yes checking whether the C++ compiler supports thread_local... yes running configure fragment for sysdeps/unix/sysv/linux/x86_64/64 running configure fragment for sysdeps/unix/sysv/linux/x86_64 running configure fragment for sysdeps/unix/sysv/linux checking installed Linux kernel header files... 3.2.0 or later checking for kernel header at least 2.6.32... ok checking for symlinks in /usr/include... ok running configure fragment for sysdeps/gnu running configure fragment for sysdeps/x86_64 checking for as... (cached) as checking version of as... 2.34, ok checking for AVX512DQ support in assembler... yes checking for AVX512 support... yes checking for Intel MPX support... yes checking whether -fPIC is default... yes checking whether -fPIE is default... yes configure: creating ./config.status config.status: creating config.make config.status: creating Makefile config.status: creating config.h config.status: executing default commands

ParthPratim commented 3 years ago

The configure script output seems fine to me. There ain't any serious dependency issues, just msgfmt is missing, which can be installed with sudo apt-get install gettext

This is pretty strange to me, as I did a fresh install a few days back, and it worked fine.

You might want to try running python3 bake.py build -vvv again to see if it works.

I'll try doing a fresh install again to see if I can reproduce the same issue.

dblessed commented 3 years ago

python3 bake.py build -vvv hi I am running dce on ubuntu 20 and I have encountered this error. please assist. the Glibc wont build

dblessed@ubuntu:~/bake$ python3 bake.py build -vvv

Building glibc-2.25 - > mkdir -p /home/dblessed/bake/build/glibc-build;cd /home/dblessed/bake/build/glibc-build;/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror mkdir -p /home/dblessed/bake/build/glibc dir=/home/dblessed/bake/source/glibc make dir=/home/dblessed/bake/build/glibc-build make: *** No targets specified and no makefile found. Stop.

Building glibc-2.25 - Problem Subprocess failed with error 2: ['make'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

The error you ran into might be a dependency problem. It turns out that your system lacks some of the dependencies used by the configure script, which is responsible for generating the Makefile.

Please let me know the output of these commands so I that can help you out better :
cd /home/dblessed/bake/build/glibc-build
/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror

Hi thank you for the reply. I did as you suggested and ran the above two lines. Glibc built but later terminated and failed to finish the build process. These were the last lines of the entire build

-Wl,--as-needed /home/dblessed/bake/build/glibc-build/elf/ld.so -Wl,--no-as-needed -lgcc gcc --print-file-name=crtendS.o /home/dblessed/bake/build/glibc-build/csu/crtn.o /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libselinux.so: undefined reference to `gettid@GLIBC_2.30' collect2: error: ld returned 1 exit status make[2]: [../Rules:192: /home/dblessed/bake/build/glibc-build/nss/makedb] Error 1 make[2]: Leaving directory '/home/dblessed/bake/source/glibc/nss' make[1]: [Makefile:215: nss/others] Error 2 make[1]: Leaving directory '/home/dblessed/bake/source/glibc' make: *** [Makefile:9: all] Error 2

Building glibc-2.25 - Problem Subprocess failed with error 2: ['make'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

ParthPratim commented 3 years ago

Are you on Ubuntu 20.04 ? Cause I'm using that, and it builds fine. As far as I see there's some issue with the linker on your system.

dblessed commented 3 years ago

Are you on Ubuntu 20.04 ? Cause I'm using that, and it builds fine. As far as I see there's some issue with the linker on your system.

Yes i am on ubuntu 20

i have built again and this is what i get still on glibc

Building glibc-2.25 - > mkdir -p /home/dblessed/bake/build/glibc-build;cd /home/dblessed/bake/build/glibc-build;/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror mkdir -p /home/dblessed/bake/build/glibc dir=/home/dblessed/bake/source/glibc make dir=/home/dblessed/bake/build/glibc-build make: libc.so.6: version `GLIBC_2.27' not found (required by make)

Building glibc-2.25 - Problem Subprocess failed with error 1: ['make'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

ParthPratim commented 3 years ago

Can you please send me the output of : ldd --version ldd and lsb_release -a

dblessed commented 3 years ago

ldd --version ldd

dblessed@ubuntu: /bake$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.2 LTS Release: 20.04 Codename: focal

dblessed@ubuntu: /bake$ ldd --version ldd ldd (Ubuntu GLIBC 2.31-0ubuntu9.2) 2.31 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper.

ParthPratim commented 3 years ago

It's pretty strange because it builds on Ubuntu 20.04.1 with the same Glibc version as yours.

Can you please attach the output of :

cd /home/dblessed/bake/source/glibc
git branch

dblessed commented 3 years ago

It's pretty strange because it builds on Ubuntu 20.04.1 with the same Glibc version as yours.

Can you please attach the output of :
cd /home/dblessed/bake/source/glibc
git branch

though I am also missing this

Dependency libc-debug - Problem Module: "libc-debug" is required by other modules but it is not available on your system. Ask your system admin or review your library database to add "libc-debug" More information from the module: "Didn't find: libc debug package; please install it." Problem: Optional dependency, module "libc-debug" not available This may reduce the functionality of the final build. However, bake will continue since "libc-debug" is not an essential dependency. For more information call bake with -v or -vvv, for full verbose mode.

ParthPratim commented 3 years ago

though I am also missing this

Dependency libc-debug - Problem Module: "libc-debug" is required by other modules but it is not available on your system. Ask your system admin or review your library database to add "libc-debug" More information from the module: "Didn't find: libc debug package; please install it." Problem: Optional dependency, module "libc-debug" not available This may reduce the functionality of the final build. However, bake will continue since "libc-debug" is not an essential dependency. For more information call bake with -v or -vvv, for full verbose mode.

You can fix that with :

sudo apt-get install libc6-dbg

dblessed commented 3 years ago

git branch dblessed@ubuntu: $ cd /home/dblessed/bake/source/glibc dblessed@ubuntu: /bake/source/glibc$ git branch

release/2.25/master

dblessed commented 3 years ago

sudo apt-get install libc6-dbg It is always telling me this

Reading package lists... Done Building dependency tree
Reading state information... Done libc6-dbg is already the newest version (2.31-0ubuntu9.2). 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.

ParthPratim commented 3 years ago

sudo apt-get install libc6-dbg It is always telling me this

Reading package lists... Done Building dependency tree Reading state information... Done libc6-dbg is already the newest version (2.31-0ubuntu9.2). 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.

libc-debug doesn't really matter for now. I'm worried about the reason why glibc build looks for Glibc 2.27, while it doesn't look for it on my system.

You might try the following two things :

Maybe try doing a fresh install. I know it's pretty expensive on data usage as it downloads the entire linux source tree.
If you can get access to an Ubuntu 20.04.1 machine and maybe try to build it there.

I'll meanwhile check why isn't it building on the latest ubuntu release.

dblessed commented 3 years ago

okay

sudo apt-get install libc6-dbg It is always telling me this

Reading package lists... Done Building dependency tree Reading state information... Done libc6-dbg is already the newest version (2.31-0ubuntu9.2). 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.

libc-debug doesn't really matter for now. I'm worried about the reason why glibc build looks for Glibc 2.27, while it doesn't look for it on my system.

You might try the following two things :

Maybe try doing a fresh install. I know it's pretty expensive on data usage as it downloads the entire linux source tree.

If you can get access to an Ubuntu 20.04.1 machine and maybe try to build it there.

I'll meanwhile check why isn't it building on the latest ubuntu release.

okay thanks I will keep you updated

ParthPratim commented 3 years ago

Just a random search on stackoverflow, lead me to this.

sudo apt-get update
sudo apt-get install libc6 libc6-dev

I'm not sure if this works, but please do check if it builds after executing these.

dblessed commented 3 years ago

sudo apt-get install libc6-dbg It is always telling me this

Reading package lists... Done Building dependency tree Reading state information... Done libc6-dbg is already the newest version (2.31-0ubuntu9.2). 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.

libc-debug doesn't really matter for now. I'm worried about the reason why glibc build looks for Glibc 2.27, while it doesn't look for it on my system.

You might try the following two things :

Maybe try doing a fresh install. I know it's pretty expensive on data usage as it downloads the entire linux source tree.

If you can get access to an Ubuntu 20.04.1 machine and maybe try to build it there.

I'll meanwhile check why isn't it building on the latest ubuntu release.

egmentation fault (core dumped) Segmentation fault (core dumped) make[2]: [../o-iterator.mk:9: /home/dblessed/bake/build/glibc-build/libc_pic.a] Error 139 make[2]: Leaving directory '/home/dblessed/bake/source/glibc/locale' make[1]: [Makefile:215: locale/others] Error 2 make[1]: Leaving directory '/home/dblessed/bake/source/glibc' make: *** [Makefile:9: all] Error 2

Building glibc-2.25 - Problem Subprocess failed with error 2: ['make'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

ParthPratim commented 3 years ago

Hi, I just got access to an Ubuntu 20.04.2 machine and could get Glibc to build without making any extra tweaks.

Here is the glibc build output part of the entire build process : Screenshot from 2021-03-04 07-23-09

Do you recall manually replacing the libc.so.6 ? You might try updating the build-essentials, in case you are using a stale make binary.

dblessed commented 3 years ago

sudo apt-get install libc6-dbg It is always telling me this

Reading package lists... Done Building dependency tree Reading state information... Done libc6-dbg is already the newest version (2.31-0ubuntu9.2). 0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.

libc-debug doesn't really matter for now. I'm worried about the reason why glibc build looks for Glibc 2.27, while it doesn't look for it on my system.

You might try the following two things :

Maybe try doing a fresh install. I know it's pretty expensive on data usage as it downloads the entire linux source tree.

If you can get access to an Ubuntu 20.04.1 machine and maybe try to build it there.

I'll meanwhile check why isn't it building on the latest ubuntu release.

Hi I tried building it a second time now in ubuntu 20.04.1 and still got this

Building glibc-2.25 - > mkdir -p /home/dblessed/bake/build/glibc-build;cd /home/dblessed/bake/build/glibc-build;/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror mkdir -p /home/dblessed/bake/build/glibc dir=/home/dblessed/bake/source/glibc make dir=/home/dblessed/bake/build/glibc-build make: libc.so.6: version `GLIBC_2.27' not found (required by make)

Building glibc-2.25 - Problem Subprocess failed with error 1: ['make'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

ParthPratim commented 3 years ago

It's pretty strange, because I've tested the setup on Ubuntu 16.04, 20.04, 20.04.1 and 20.04.2 and nowhere does it create this issue.

Can you tell me more about your Ubuntu setup, like is it setup from the official ISO or you're running linux in some other way ?

dblessed commented 3 years ago

Hi, I just got access to an Ubuntu 20.04.2 machine and could get Glibc to build without making any extra tweaks.

Here is the glibc build output part of the entire build process :

Do you recall manually replacing the libc.so.6 ? You might try updating the build-essentials, in case you are using a stale make binary.

could it be because I am using a virtual machine?

ParthPratim commented 3 years ago

could it be because I am using a virtual machine?

I don't think so because even I'm running Ubuntu 20.04.2 on VirtualBox and it builds fine.

dblessed commented 3 years ago

It's pretty strange, because I've tested the setup on Ubuntu 16.04, 20.04, 20.04.1 and 20.04.2 and nowhere does it create this issue.

Can you tell me more about your Ubuntu setup, like is it setup from the official ISO or you're running linux in some other way ?

i have it running in vmware

dblessed commented 3 years ago

It's pretty strange, because I've tested the setup on Ubuntu 16.04, 20.04, 20.04.1 and 20.04.2 and nowhere does it create this issue.

Can you tell me more about your Ubuntu setup, like is it setup from the official ISO or you're running linux in some other way ?

okay I will have to try it in ubuntu 16 then... you said that it works fine in ubuntu 16?

ParthPratim commented 3 years ago

i have it running in vmware

I don't think so that should be a problem.

Because as far as I see, this error usually occurs when you're using a binary of make which has been built on GLIBC 2.27, and it looks for it now but couldn't find it.

Does upgrading make, be of any help ?

ParthPratim commented 3 years ago

okay I will have to try it in ubuntu 16 then... you said that it works fine in ubuntu 16?

Yes it does work on Ubuntu 16, but you could get it to work on Ubuntu 16 without this patch by just following the official docs, the whole motive of this Issue was to get DCE working on Ubuntu 20.04 LTS.

You system behaves pretty weird and I'm also not able to reproduce the error even while using your base system as it works fine for me still.

dblessed commented 3 years ago

Hi in ubuntu 16 i have gotten this error.

make[2]: Leaving directory '/home/dblessed/bake/source/glibc/elf' make[1]: Leaving directory '/home/dblessed/bake/source/glibc' make install DESTDIR=/home/dblessed/bake/build/glibc dir=/home/dblessed/bake/build/glibc-build make: �v�S�: ��k: Error 1789877066

Building glibc-2.25 - Problem Subprocess failed with error 127: ['make', 'install', 'DESTDIR=/home/dblessed/bake/build/glibc'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

ParthPratim commented 3 years ago

make: �v�S�: ��k�: Error 1789877066

I don't quite understand this error message because I could get it to build on a clean Ubuntu 16 installation.

I wonder what these strange characters could mean.

dblessed commented 3 years ago

make: �v�S�: ��k�: Error 1789877066

I don't quite understand this error message because I could get it to build on a clean Ubuntu 16 installation.

I wonder what these strange characters could mean.

Hi, when building in your system are you installing any dependencies? and if yes which ones. coz I have been using this tutorial page in all my systems. https://www.nsnam.org/wiki/Installation

also must you install ns3 3.32 before doing your build?

ParthPratim commented 3 years ago

Hi, when building in your system are you installing any dependencies? and if yes which ones. coz I have been using this tutorial page in all my systems. https://www.nsnam.org/wiki/Installation

The only external dependency I had to install was gawk, which can be installed by sudo apt-get install gawk Nothing other than this was required.

Are you using the build instructions mentioned here https://github.com/direct-code-execution/ns-3-dce/issues/110#issuecomment-775698732 ?

also must you install ns3 3.32 before doing your build?

The Bake build system must do that for you.

dblessed commented 3 years ago

hi Parthpratim, there should be something that you are doing differently while installing dce in these ubuntu versions that you have not told us because sincerely I have tried your set up now 8 times and I am getting the same error. both in ubuntu 16 and ubuntu 20. i have tried new installations on different machines but still getting the same error.

Building glibc-2.25 - > mkdir -p /home/dblessed/bake/build/glibc-build;cd /home/dblessed/bake/build/glibc-build;/home/dblessed/bake/source/glibc/configure --prefix=/usr --disable-werror mkdir -p /home/dblessed/bake/build/glibc dir=/home/dblessed/bake/source/glibc make dir=/home/dblessed/bake/build/glibc-build make: libc.so.6: version `GLIBC_2.27' not found (required by make)

Building glibc-2.25 - Problem Subprocess failed with error 1: ['make'] Error: Critical dependency, module "glibc-2.25" failed For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

ParthPratim commented 3 years ago

hi Parthpratim, there should be something that you are doing differently while installing dce in these ubuntu versions that you have not told us because sincerely I have tried your set up now 8 times and I am getting the same error. both in ubuntu 16 and ubuntu 20. i have tried new installations on different machines but still getting the same error.

All the builds I've tested were made on clean installations. I downloaded a fresh ISO from the official website, installed it on VirtualBox and setup the entire project. I'm also sure that I made no other external installations other than build-essential(already there on most systems but can be installed if not there) and gawk . Nothing other than this was required to setup the project.

The only think I could do now is create a video of the installation process and post it on YouTube if that helps.

dblessed commented 3 years ago

hi Parthpratim, there should be something that you are doing differently while installing dce in these ubuntu versions that you have not told us because sincerely I have tried your set up now 8 times and I am getting the same error. both in ubuntu 16 and ubuntu 20. i have tried new installations on different machines but still getting the same error.

All the builds I've tested were made on clean installations. I downloaded a fresh ISO from the official website, installed it on VirtualBox and setup the entire project. I'm also sure that I made no other external installations other than build-essential(already there on most systems but can be installed if not there) and gawk . Nothing other than this was required to setup the project.

The only think I could do now is create a video of the installation process and post it on YouTube if that helps.

okay let us work on it

ParthPratim commented 3 years ago

Can you please send me the output of these commands from an Ubuntu 20.04 machine ?

readelf -Ws /bin/make | grep GLIBC*

make --version

direct-code-execution / ns-3-dce

Ubuntu 20.04 support #110

Description of the problem

output of ./waf configure

Steps to reproduce

Things I did to get the DCE code to build :

Bake Setup

Bake Build

Testing DCE

Source References :

Successful Run :