search

benfogle / crossenv

Cross-compiling virtualenv for Python
MIT License
108 stars 22 forks source link

Cross compile for my target board #18

Open shubham101096 opened 4 years ago

shubham101096 commented 4 years ago

Hi

I need the pycryptodome/lxml package for the following architecture of my target board. The target board does not have internet access so I cannot directly install pycryptodome/lxml on it. I need a method to compile it on my ubuntu 64 bit pc and then transfer it on my board. Following describes the toolchain I am using and the architecture of the target board. I cant find a .whl file or anything else for this architecture. Can I solve this problem using crossenv and how?

PPC_e5500_CC_BASE_DIR := /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux ifeq (${TARGET_CPU}, PPC_e5500) ifeq (${TARGET_CPU_BIT},) $(info TARGET_CPU_BIT must be 32BIT/64BIT) $(warning TARGET_CPU_BIT not specified, assuming 32BIT) TARGET_CPU_BIT = 32BIT endif TARGET_ARCH = powerpc COMPILER_TYPE = gcc ifeq (${TARGET_CPU_BIT}, 32BIT)

TARGET_CPU_BIT = 32BIT

CROSS_COMPILE_PATH = ${PPC_e5500_CC_BASE_DIR}/usr/bin/powerpc-fsl-linux:${PPC_e5500_CC_BASE_DIR}/usr/bin CROSS_COMPILE = powerpc-fsl-linux- HOST = powerpc-fsl-linux CC_SYSROOT = ${PPC_e5500_CC_BASE_DIR}/../ppce5500-fsl-linux COMMON_CFLAGS += -m32 -mhard-float -mcpu=e5500 COMMON_CFLAGS += --sysroot=${CC_SYSROOT}

Regards Shubham Mishra

benfogle commented 4 years ago

Building a wheel for an embedded platform is an intended use case for crossenv. While I haven't tested this on a powerpc target yet, if you have successfully compiled Python for this board, then crossenv should work.

shubham101096 commented 4 years ago

Screenshot from 2020-02-19 13-37-50

I installed the python-docx package using crossenv. The names of the .so files have "cpython-36m-powerpc-linux-gnu" written in them because I cross compiled them. For eg: the name of _raw_ecb.so is written as _raw_ecb.cpython-36m-powerpc-linux-gnu.so . So will it cause problem because I think code would try to find _raw_ecb.so and not "_raw_ecb.cpython-36m-powerpc-linux-gnu.so". Will I have to remove "cpython-36m-powerpc-linux-gnu" from every .so file name ?

benfogle commented 4 years ago

No, I don't think you'll need to rename anything, and those file names are a good sign that everything is working.

What your seeing is a PEP3149 ABI-tagged .so file, which was introduced in Python 3.2. Python will produce the long name by default and prefers that on import, but it will still fall back to the shorter version (_raw_ecb.so) if it can't find the long one.

shubham101096 commented 4 years ago

I am trying to cross compile lxml package for my board which has following coniguration:

PPC_e5500_CC_BASE_DIR := /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux TARGET_CPU_BIT = 32BIT TARGET_ARCH = powerpc COMPILER_TYPE = gcc CROSS_COMPILE_PATH = ${PPC_e5500_CC_BASE_DIR}/usr/bin/powerpc-fsl-linux:${PPC_e5500_CC_BASE_DIR}/usr/bin CROSS_COMPILE = powerpc-fsl-linux- HOST = powerpc-fsl-linux CC_SYSROOT = ${PPC_e5500_CC_BASE_DIR}/../ppce5500-fsl-linux COMMON_CFLAGS += -m32 -mhard-float -mcpu=e5500 COMMON_CFLAGS += --sysroot=${CC_SYSROOT}

My pc has ubuntu 16.04 64bit OS. I am using the crossenv package for this purpose (https://pypi.org/project/crossenv/). But I am getting the following error when installing lxml.

(cross) p@OptiPlex-5050:~/Desktop$ pip install lxml Collecting lxml Using cached https://files.pythonhosted.org/packages/39/2b/0a66d5436f237aff76b91e68b4d8c041d145ad0a2cdeefe2c42f76ba2857/lxml-4.5.0.tar.gz Installing collected packages: lxml Running setup.py install for lxml ... error Complete output from command /home/prateek/Desktop/v/cross/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-nr9hop30/lxml/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-qq5739wi-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/prateek/Desktop/v/cross/include/site/python3.6/lxml:

Command "/home/prateek/Desktop/v/cross/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-nr9hop30/lxml/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-qq5739wi-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/prateek/Desktop/v/cross/include/site/python3.6/lxml" failed with error code 1 in /tmp/pip-build-nr9hop30/lxml/ You are using pip version 9.0.1, however version 20.0.2 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

Then I tried installing libxml2 and got the following error:

(cross) p@OptiPlex-5050:~/Desktop$ pip install libxml2 Collecting libxml2 Could not find a version that satisfies the requirement libxml2 (from versions: ) No matching distribution found for libxml2 You are using pip version 9.0.1, however version 20.0.2 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

If possible, please help me in resolving this issue. Thank you.

benfogle commented 4 years ago

Libxml2 is a C library that the lxml uses, along with libxslt. Lxml requires both to build and to run. You will need to do the following:

  1. Download libxml2 and libxslt from ftp://xmlsoft.org/libxml2/
  2. Cross compile both for your system and install them as appropriate. I'm afraid I can't provide much guidance here except to say that you will need the header files to build, and the resulting .so files will need to be somewhere on your firmware where the linker can find them.
  3. Let lxml know where to find them. If you installed them to sysroot, maybe you don't have to do anything. Otherwise set the environment variables CFLAGS and LDFLAGS as needed. You can do this by passing --env when building crossenv, or by simply running export CFLAGS=... after activating the environment.
  4. Pip install should work. If it doesn't you'll need to examine the output (that will look much like what you posted above) to figure out the error. The important line is the one that begins with powerpc-fsl-linux-gcc -m32 -mhard-float .... Make sure that it has the right -I and -L flags to find libxml2 and libxslt.