search

theopolis / build-anywhere

Scripts for building compilers that run anywhere, which build things that run anywhere
86 stars 9 forks source link

Would a cython python app benefit from this/how do i do this? #3

Open hippylover opened 4 years ago

hippylover commented 4 years ago

Hi. I downloaded the precompiled version of build-anywhere earlier today. Then: source /home/kris/x86_64-anywhere-linux-gnu

Then i made a .c file with cython(embedding the python interpreter): cython --embed -3 program.py this depends upon pyqt5 and enum(that again depends on some other standard libs in /usr/lib/python3.7 im not sure how to include).

Then how shall i compile this app making sure i include the right dependencies? I don't care if i have to copy the whole /usr/lib/python3.7 folder and set the sys.path inside the python program to look inside customdir/python3.7 if this is the only option, but first and foremost i just want to compile the app.

Then i will try to copy it to centos6 VM just to see if it runs.

hippylover commented 4 years ago

I tried this:

$ gcc -O0 -fPIC -I /usr/include/python3.7m  -L/usr/lib -lpython3.7m -lcrypt -lpthread -ldl  -lutil -lm  -o program program.c
ld.lld: error: unknown -format value: elf64-x86-64 (supported formats: elf, default, binary)
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)

but that is what i got.

Flags gotten through:

$ python3.7-config --ldflags
 -L/usr/lib -lpython3.7m -lcrypt -lpthread -ldl  -lutil -lm
theopolis commented 4 years ago

Hey @hippylover, I think you'd have to build each of those dependencies with this toolchain.

It you look at how ssdeep's build is configured, it installs itself into the toolchain: https://github.com/theopolis/build-anywhere/blob/master/tests/basic.sh#L49

Then other libraries/projects that depend on ssdeep can use the portable version.

hippylover commented 4 years ago

I think this is going a bit above my paygrade... im primarily a python dev. I don't speak C and all i know of C++ is half of the tutorial on learncpp.com...

Iv'e already spent two months on this madness(should have just learned golang).

If you could give it a shot i would be much obliged, for where you spend 2 hours i spend 2 months. I'm sure a lot of python devs would appreciate it too, because binary distribution is currently a nightmare with python and cython.

theopolis commented 4 years ago

Sure I can try later today. :)

hippylover commented 4 years ago

Found some stuff that might be relevant: https://web.archive.org/web/20180926104719/http://general-purpose.io/2015/12/06/compiling-python-and-libpython-statically-using-musl-libc/ https://gist.github.com/maddouri/c4b97474f21fabc9ef61

These two show some configure flags etc to do it that should be relevant even if one doesn't use musl i think(!) https://stackoverflow.com/questions/1150373/compile-the-python-interpreter-statically/1155092#1155092

I don't think i can use musl though. Because i need to use the pyqt libs which are built on glibc. To save download time i plan on letting the user install needed deps through a static mythical pip that i may or may not be able to build.

hippylover commented 4 years ago

Btw, if i want to use pyqt does that also have to be compiled with build-anywhere or can i avoid it? Maybe alpine linux is a help, they only build musl binaries?

edit: i also need PyQtWebEngine from pip.

hippylover commented 4 years ago

Well, seems like this will help: https://wiki.python.org/moin/BuildStatically. I will try it and report back. How's your progress? :D

hippylover commented 4 years ago

I actually managed to make a portable no-dynamic libs python binary with included standard libs... I will outline the steps i took tomorrow when im not so tired.

I have two issues left though(no doubt before i get more issues xD). What on god's green earth shall i put in Setup/Setup.dist/Setup.local to get _decimal module to build?? This must be portable of course. I don't know if i need to build libmpdec portable too, if so i probably need help with that too.

That is one problem. Another problem is which line do i need for _sqlite3 module... I haven't tried that one so far. Will attempt now.

theopolis commented 4 years ago

Hey @hippylover, my approach was to build python using the build-anywhere toolchain then try to perform the small test you mentioned at the top.

It looks like the scope of what you are trying to accomplish keeps increasing. I do not know python well or what you are trying to ultimately achieve so I'm not sure I can be very helpful. I do recommend that if you want to ship something portable, it, and all of it's dependencies need to be built with build-anywhere.