Closed RasMH closed 2 years ago
peadar
commented
3 years ago Hm. I was moving both projects a little independently, and broke the interface between them when pushing some cleanups to pstack. I've pushed bc2938f to ctypegen which should fix the build issue. There may be some platform dependency issues with the tests, depending on the version of libc that you have locally - let me know how you get on.
RasMH
commented
3 years ago It compiled nicely now. Thanks for the quick answer and fix! I will now investigate further the functionality.
There was however some issue with the test. Not sure if it matters.
make PYTHON=python3 test
echo "/home/myuser/git/ctypegen:/home/myuser/git/ctypegen/build/lib.linux-x86_64-3.8"
/home/myuser/git/ctypegen:/home/myuser/git/ctypegen/build/lib.linux-x86_64-3.8
PYTHONPATH=/home/myuser/git/ctypegen:/home/myuser/git/ctypegen/build/lib.linux-x86_64-3.8 make -C test
make[1]: Entering directory '/home/myuser/git/ctypegen/test'
...
python3 ./ChainTest.py ./libChainTest.so
generated and tested chaintest
I mock you: 1 2 3
mockme(3, 2, 1)
python3 ./PointersTest.py ./libFOpenTest.so
generated and tested ptrgen
no argument type information provided for function fopen. Provide 'argtypes' manually, or generate with CTypeGen
Traceback (most recent call last):
File "./PointersTest.py", line 39, in
peadar
commented
3 years ago CTypeGen tries to generate bindings for as much of libc as it can. It seems that the tests rely on the generated libc content more than I was aware - can you try installing the debug symbols from glibc, and rebuilding? For RPM-based systems, the package is glibc-debuginfo, and for .deb based systems, I think its libc6-dbg
Once you've installed those, you should remove CMock/libc.py so it's regenerated - the regenerated output should have the line "lib.fopen.restype = POINTER( FILE )" somewhere in it if it sucessfully built.
The build process for CTypeGen appears rough around the edges outside our internal environment, it seems. I'll endeavor to improve it based on your experience.
RasMH
commented
3 years ago I installed libc6-dbg and compiled from a fresh git clone. Compilation now errors out in the CMock part.
make PYTHON=python3
env CFLAGS="-g --std=c++14" PYTHONPATH=/home/myuser/git/ctypegen python3 ./setup.py build
running build
running build_py
running build_ext
building 'libCTypeGen' extension
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g --std=c++14 -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/include -I/usr/include/python3.8 -c CTypeGen.cpp -o build/temp.linux-x86_64-3.8/CTypeGen.o
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g --std=c++14 -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/CTypeGen.o -L/usr/local/lib -Wl,--enable-new-dtags,-R/usr/local/lib -ldwelf -o build/lib.linux-x86_64-3.8/libCTypeGen.cpython-38-x86_64-linux-gnu.so
building 'libCTypeMock' extension
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g --std=c++14 -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/include -I/usr/include/python3.8 -c cmock.cpp -o build/temp.linux-x86_64-3.8/cmock.o
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g --std=c++14 -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/include -I/usr/include/python3.8 -c thunk-x86_64.s -o build/temp.linux-x86_64-3.8/thunk-x86_64.o
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g --std=c++14 -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/cmock.o build/temp.linux-x86_64-3.8/thunk-x86_64.o -L/usr/local/lib -Wl,--enable-new-dtags,-R/usr/local/lib -ldwelf -o build/lib.linux-x86_64-3.8/libCTypeMock.cpython-38-x86_64-linux-gnu.so
running build_scripts
creating build/scripts-3.8
copying and adjusting ctypegen -> build/scripts-3.8
changing mode of build/scripts-3.8/ctypegen from 644 to 755
cc -O0 -g -fPIC -fno-eliminate-unused-debug-types -g3 -c -o dbghelper.o dbghelper.c
cc -g --shared -o libdbghelper.so dbghelper.o
PYTHONPATH=/home/myuser/git/ctypegen:/home/a335103/git/ctypegen/build/lib.linux-x86_64-3.8 python3 ./generateLibc.py libc.so.6 CMock/libc.py
Traceback (most recent call last):
File "./generateLibc.py", line 66, in
RasMH
commented
3 years ago Blah. Did not mean to close the issue...
peadar
commented
3 years ago I've cleaned up the build process and dependencies, which should have fixed this issue, as of 4269bc6. Can you confirm?
On Thu, 7 Oct 2021 at 06:55, RasMH @.***> wrote:
Reopened #9 https://github.com/aristanetworks/ctypegen/issues/9.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/aristanetworks/ctypegen/issues/9#event-5426113710, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAE6NDNG3IBAVFJMJBZIGB3UFUY6PANCNFSM5FOR7WFA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
RasMH
commented
3 years ago It improved things. Compilation is clean but some tests have issues. Showing relevant test output:
...
recursion depth: 56, 55, 54, 53, 52, 51, 50, 49, 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, done
python3 ./CTypeGenSanity.py ./CTypeSanity
warning: CTypeGen.generate requires a list of ELF images as its first argument
warning: no type found for NoSuchType
warning: variable NoSuchGlobal not found
warning: function nosuch_func not found
generated and tested CTypeSanity
compare anInt: 3/3
compare aChar: a/b'a'
...
python3 ./PointersTest.py ./libFOpenTest.so
generated and tested ptrgen
python3 ./GreedyTestGen.py ./libGreedyTest.so
error: failed to find definition for std::basic_istream<char, std::char_traits
peadar
commented
3 years ago Those are actually fine. I can suppress the warnings, but everything should be working
On Fri 8 Oct 2021, 08:19 RasMH, @.***> wrote:
It improved things. Compilation is clean but some tests still have some issues. Showing relevant test output: ... recursion depth: 56, 55, 54, 53, 52, 51, 50, 49, 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, done python3 ./CTypeGenSanity.py ./CTypeSanity warning: CTypeGen.generate requires a list of ELF images as its first argument warning: no type found for NoSuchType warning: variable NoSuchGlobal not found warning: function nosuch_func not found generated and tested CTypeSanity compare anInt: 3/3 compare aChar: a/b'a' ... python3 ./PointersTest.py ./libFOpenTest.so generated and tested ptrgen python3 ./GreedyTestGen.py ./libGreedyTest.so error: failed to find definition for std::basic_istream<char, std::char_traits > - will fall back to using declaration error: failed to find definition for std::basic_ostream<char, std::char_traits > - will fall back to using declaration error: failed to find definition for std::basic_istream<wchar_t, std::char_traits
> - will fall back to using declaration error: failed to find definition for std::basic_ostream<wchar_t, std::char_traits > - will fall back to using declaration error: _IO_lock_t is 'void' - cannot output definition generated and tested GreedyTest python3 ./EnumTest.py ./libEnumTest.so a: start=-128, end=127 h: start=0, end=255 s: start=-32768, end=32767 t: start=0, end=65535 i: start=-2147483648, end=2147483647 j: start=0, end=4294967295 l: start=-9223372036854775808, end=9223372036854775807 m: start=0, end=18446744073709551615 error: _IO_lock_t is 'void' - cannot output definition error: failed to find definition for std::basic_istream<char, std::char_traits > - will fall back to using declaration error: failed to find definition for std::basic_ostream<char, std::char_traits > - will fall back to using declaration error: failed to find definition for std::basic_istream<wchar_t, std::char_traits > - will fall back to using declaration error: failed to find definition for std::basic_ostream<wchar_t, std::char_traits > - will fall back to using declaration generated and tested EnumGenerated python3 ./SupplyDemandGen.py generated and tested Supply generated and tested Demand make[1]: Leaving directory '/home/myuser/git/ctypegen/test' echo "Tested for python3" Tested for python3 — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/aristanetworks/ctypegen/issues/9#issuecomment-938405715, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAE6NDLTEW5QD474CWZ4WFDUF2LPVANCNFSM5FOR7WFA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
Hi, I saw the nice presentation on youtube on ctypegen and wanted to try it out. I could compile pstack without issues, but for ctypes compilation errors out with an error concerning a private function.
me@mycomp:~/git/ctypegen$ make PYTHON=python3 env CFLAGS="-g --std=c++14" PYTHONPATH=/home/a335103/git/ctypegen python3 ./setup.py build running build running build_py creating build creating build/lib.linux-x86_64-3.8 copying CTypeGenRun.py -> build/lib.linux-x86_64-3.8 creating build/lib.linux-x86_64-3.8/CMock copying CMock/helpers.py -> build/lib.linux-x86_64-3.8/CMock copying CMock/init.py -> build/lib.linux-x86_64-3.8/CMock creating build/lib.linux-x86_64-3.8/CTypeGen copying CTypeGen/expression.py -> build/lib.linux-x86_64-3.8/CTypeGen copying CTypeGen/init.py -> build/lib.linux-x86_64-3.8/CTypeGen running build_ext building 'libCTypeGen' extension creating build/temp.linux-x86_64-3.8 x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g --std=c++14 -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/include -I/usr/include/python3.8 -c CTypeGen.cpp -o build/temp.linux-x86_64-3.8/CTypeGen.o CTypeGen.cpp: In function ‘PyObject pyAttr(PyDwarfEntry, Dwarf::AttrName, const Dwarf::Attribute&)’: CTypeGen.cpp:834:37: error: ‘std::shared_ptr Dwarf::DIE::unit’ is private within this context
834 | auto lines = entry->die.unit->getLines();
| ^~~~
In file included from CTypeGen.cpp:36:
/usr/local/include/libpstack/dwarf.h:213:27: note: declared private here
213 | std::shared_ptr unit;
| ^~~~
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
make: *** [Makefile:24: build-all] Error 1
Am I doing something wrong? I work in Ubuntu 20.4 in windows subsystem from linux. Should have the latest gcc, g++ etc. Python version is 3.8. CXX compiler is GNU 9.3.0.