kraj
commented
4 years ago @MastaG what is PYTHONHOME set to ? perhaps bitbake -e firefox might tell us
Open MastaG opened 4 years ago
kraj
commented
4 years ago @MastaG what is PYTHONHOME set to ? perhaps bitbake -e firefox might tell us
MastaG
commented
4 years ago @kraj unfortunately bitbake -e firefox doesn't expose PYTHONPATH or PYTHONHOME. However it does say:
# $PYTHON
# set /home/mastag/src/odroid-oe-core/openembedded-core/meta/conf/bitbake.conf:533
# "${@sys.executable}"
PYTHON="/usr/bin/python3"So I can assume it's using python3 from my host? That one is capable of importing "site":
Python 3.7.5 (default, Oct 17 2019, 12:16:48)
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import siteDe compile log does say: 0:11.08 Consider setting $PYTHONHOME to
Could it be that it's unset by default?
kraj
commented
4 years ago interesting, I wonder if its defaulting to modules from python that yocto builds and python binary from host then it might be an issue where yocto python3-native might not be building site module.
dswalla
commented
4 years ago I'm seeing a similar error in building Firefox:
| 15:36.62 error: build failed
| 15:36.63 make[4]: *** [/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/config/makefiles/rust.mk:240: force-cargo-library-build] Error 101
| 15:36.63 make[3]: *** [/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/config/recurse.mk:74: toolkit/library/rust/target] Error 2
| 15:36.63 make[3]: *** Waiting for unfinished jobs....
| 16:22.83 make[2]: *** [/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/config/recurse.mk:34: compile] Error 2
| 16:22.83 make[1]: *** [/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/config/rules.mk:413: default] Error 2
| 16:22.83 make: *** [client.mk:125: build] Error 2And prior to that in the logs, the Python error:
| 13:45.45 --- stderr
| 13:45.45 Traceback (most recent call last):
| 13:45.45 File "/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/servo/components/style/properties/build.py", line 5, in <module>
| 13:45.45 import json
| 13:45.45 File "/usr/lib/python2.7/json/__init__.py", line 108, in <module>
| 13:45.45 from .decoder import JSONDecoder
| 13:45.45 File "/usr/lib/python2.7/json/decoder.py", line 5, in <module>
| 13:45.45 import struct
| 13:45.45 File "/usr/lib/python2.7/struct.py", line 1, in <module>
| 13:45.45 from _struct import *
| 13:45.45 ImportError: /usr/lib/python2.7/lib-dynload/_struct.so: undefined symbol: PyUnicodeUCS4_AsEncodedString
| 13:45.45 warning: build failed, waiting for other jobs to finish...
ashie
commented
4 years ago I can't reproduce it. Could you show me your build environment's information?
In addition, please show me PYTHON in your config.status (tmp/work/<target-arch>/firefox/68.0esr-r0/firefox-68.0/firefox-build-dir/config.status)
dswalla
commented
4 years ago @ashie Build is on Manjaro - kernel 4.19.88-1. All my recipes are from the Yocto zeus branch.
From config.status:
'PYTHON': '/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/firefox-build-dir/_virtualenvs/init/bin/python',
'PYTHON3': '/var/tmp/yocto/hosttools/python3',
'PYTHON3_VERSION': '3.7.4',And bblayers.conf:
POKY_BBLAYERS_CONF_VERSION = "2"
BBPATH = "${TOPDIR}"
BBFILES ?= ""
BBLAYERS ?= " \
${TOPDIR}/../meta \
${TOPDIR}/../meta-poky \
${TOPDIR}/../meta-yocto-bsp \
${TOPDIR}/../meta-openembedded/meta-filesystems \
${TOPDIR}/../meta-openembedded/meta-gnome \
${TOPDIR}/../meta-openembedded/meta-initramfs \
${TOPDIR}/../meta-openembedded/meta-multimedia \
${TOPDIR}/../meta-openembedded/meta-networking \
${TOPDIR}/../meta-openembedded/meta-oe \
${TOPDIR}/../meta-openembedded/meta-perl \
${TOPDIR}/../meta-openembedded/meta-python \
${TOPDIR}/../meta-openembedded/meta-webserver \
${TOPDIR}/../meta-openembedded/meta-xfce \
${TOPDIR}/../meta-browser \
${TOPDIR}/../meta-rust \
${TOPDIR}/../meta-intel \
${TOPDIR}/../meta-clang \
"'PYTHON': '/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/firefox-build-dir/_virtualenvs/init/bin/python',
Hmm, it's almost same with mine. Is it capable of importing "site"? (It's capable on my environment)
ashie
commented
4 years ago Manjaro
It's interesting, I've heard for the first time at this time. I don't have knowledge about Arch Linux based distributions, I'll try it when I have spare time...
dswalla
commented
4 years ago @ashie No issues importing "site" with the Python version in the Firefox build directory or with my host python.
Another interesting note, when I run bitbake -e firefox and search for PYTHON I see this:
# $PYTHON
PYTHON="/usr/bin/python3"
dburnsii
commented
4 years ago Confirmed the issue in Manjaro and in Ubuntu:16.04 Docker image with x86 Target.
| 0:03.28 error: failed to run custom build command for `style v0.0.1 (/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/servo/components/style)`
| 0:03.28 Caused by:
| 0:03.28 process didn't exit successfully: `/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/firefox-build-dir/release/build/style-01cf4f6b28ee30c3/build-script-build` (exit code: 1)
| 0:03.28 --- stdout
| 0:03.28 cargo:rerun-if-changed=build.rs
| 0:03.28 cargo:out_dir=/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/firefox-build-dir/x86_64-poky-linux/release/build/style-209e7c87851a4395/out
| 0:03.29 cargo:rerun-if-changed=properties/helpers/animated_properties.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/gecko.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/properties.html.mako
| 0:03.29 cargo:rerun-if-changed=properties/computed_value_flags.rs
| 0:03.29 cargo:rerun-if-changed=properties/Mako-0.9.1.zip
| 0:03.29 cargo:rerun-if-changed=properties/longhands/padding.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/inherited_box.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/outline.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/effects.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/inherited_table.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/list.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/svg.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/xul.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/ui.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/inherited_text.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/background.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/text.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/margin.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/table.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/border.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/font.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/column.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/color.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/position.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/inherited_svg.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/box.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/counters.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/longhands/inherited_ui.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/build.py
| 0:03.29 cargo:rerun-if-changed=properties/declaration_block.rs
| 0:03.29 cargo:rerun-if-changed=properties/properties.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/data.py
| 0:03.29 cargo:rerun-if-changed=properties/helpers.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/padding.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/outline.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/list.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/svg.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/inherited_text.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/background.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/text.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/margin.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/border.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/font.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/column.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/position.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/inherited_svg.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/shorthands/box.mako.rs
| 0:03.29 cargo:rerun-if-changed=properties/cascade.rs
| 0:03.29 --- stderr
| 0:03.29 Traceback (most recent call last):
| 0:03.29 File "/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/firefox-68.0/servo/components/style/properties/build.py", line 5, in <module>
| 0:03.29 import json
| 0:03.29 File "/usr/lib/python2.7/json/__init__.py", line 108, in <module>
| 0:03.29 from .decoder import JSONDecoder
| 0:03.29 File "/usr/lib/python2.7/json/decoder.py", line 5, in <module>
| 0:03.29 import struct
| 0:03.29 File "/usr/lib/python2.7/struct.py", line 1, in <module>
| 0:03.29 from _struct import *
| 0:03.29 ImportError: /usr/lib/python2.7/lib-dynload/_struct.so: undefined symbol: PyUnicodeUCS4_AsEncodedString
| 0:03.29 warning: build failed, waiting for other jobs to finish...One aspect to explore would be to see how python2 is built on host as well as python-native on yocto perhaps try to run following program with python2 from yocto build and the native one
import sys
print(sys.maxunicode)From within the docker container:
yocto@138ad52962b6:/yocto$ python2
Python 2.7.12 (default, Oct 8 2019, 14:14:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.maxunicode)
1114111
>>> exit()
yocto@138ad52962b6:/yocto$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenialPython seems to have no problem importing json from within the docker shell?
kraj
commented
4 years ago and what do you get when you run python2 from sysroot-native that yocto built ?
dswalla
commented
4 years ago @kraj If you're referring to the python2 at /var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/bin/python-native, I get this:
/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/bin/python-native ᐅ ./python2
Python 2.7.16 (default, Dec 10 2019, 23:39:41)
[GCC 9.1.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.maxunicode)
65535@dswalla yes thats the problem. Yocto compiles its own version of python2 and it seems to be using modules from host, and they wont work reliably if the pythons are not compiled with same settings/features in this case we can see that your host python2 is compiled with 32bit encoding support but yocto version is compiled with 16-bit encoding support for unicode.
dburnsii
commented
4 years ago Interesting. Is there a portion of the Firefox build script that overrides the python modules path? Running the python2-native executable and checking the path produces the following:
~ ᐅ /var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/bin/python-native/python2
Python 2.7.16 (default, Dec 10 2019, 23:39:41)
[GCC 9.1.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print '\n'.join(sys.path)
/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/lib/python27.zip
/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/lib/python2.7
/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/lib/python2.7/plat-linux2
/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/lib/python2.7/lib-tk
/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/lib/python2.7/lib-old
/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/lib/python2.7/lib-dynload
/var/tmp/yocto/work/corei7-64-poky-linux/firefox/68.0esr-r0/recipe-sysroot-native/usr/lib/python2.7/site-packages@kraj with the latest openembedded-core, meta-openembedded and meta-python2 it seems to build now (still building so I'm not sure whether it will finish successfully), However I've had to create a .bbappend for firefox with: DEPENDS += "python-native" in order for it to detect python2.
EDIT: Well it failed again but not related to this issue.
4:06.22 Compiling audioipc v0.2.4 (/home/mastag/src/odroid-oe-core/build/tmp/work/cortexa15t2hf-neon-vfpv4-oe-linux-gnueabi/firefox/68.0esr-r0/firefox-68.0/media/audioipc/audioipc)
4:06.24 error[E0506]: cannot assign to self.input.cached_token because it is borrowed
4:06.24 --> /usr/src/debug/firefox/68.0esr-r0/firefox-68.0/third_party/rust/cssparser/src/parser.rs:584:17
4:06.24 |
4:06.24 559 | pub fn next_including_whitespace_and_comments(&mut self) -> Result<&Token<'i>, BasicParseError<'i>> {
4:06.24 | - let's call the lifetime of this reference '1
4:06.24 ...
4:06.24 572 | Some(ref cached_token)
4:06.24 | ---------------- borrow of self.input.cached_token occurs here
4:06.24 ...
4:06.24 584 | self.input.cached_token = Some(CachedToken {
4:06.24 | ^^^^^^^^^^^^^^^^^^^^^^^ assignment to borrowed self.input.cached_token occurs here
4:06.24 ...
4:06.24 596 | Ok(token)
4:06.24 | --------- returning this value requires that self.input.cached_token.0 is borrowed for '1
4:06.26 8 warnings generated.
4:06.27 Compiling rkv v0.9.4
So I guess the python problems are solved when using upstream version of bitbake, openembedded-core, meta-openembedded and meta-python2. All it needs is a: DEPENDS += "python-native"
MastaG
commented
4 years ago Small update, the python problem seems to persist. Firefox can still be built using:
PREFERRED_VERSION_rust-native ?= "1.37.0" PREFERRED_VERSION_rust-cross-aarch64 ?= "1.37.0" PREFERRED_VERSION_rust-llvm-native ?= "1.37.0" PREFERRED_VERSION_libstd-rs ?= "1.37.0" PREFERRED_VERSION_cargo-native ?= "1.37.0"
Even when adding DEPENDS += "python-native" it seems to error out with the same message as my opening post:
2:02.38 Could not find platform independent libraries
When I run the native python I can import site without problems: ./tmp/work/cortexa15t2hf-neon-vfpv4-oe-linux-gnueabi/firefox/68.0esr-r0/recipe-sysroot-native/usr/bin/python-native/python Python 2.7.17 (default, Mar 10 2020, 14:40:54) [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import site
shr-project
commented
4 years ago error[E0506]: cannot assign to self.input.cached_token because it is borrowed
The fix for this is available here: https://hg.mozilla.org/releases/mozilla-esr68/rev/f0020f517832
I've already backported it to meta-browser in: https://github.com/shr-project/meta-browser/commits/jansa/rust but then it panics elsewhere in rust and I haven't found a fix for that yet (log.do_compile shown in the same branch).
For python2 issue in dunfell, even using pythonnative didn't work for me, only adding python to HOSTTOOLS worked for me on Ubuntu 20.04, I've added my WIP changes to: https://github.com/shr-project/meta-browser/commits/jansa/python
lmajewski
commented
4 years ago @shr-project I do guess that you are using zeus YPRR ? If yes, you would use gcc 9.x ?
shr-project
commented
4 years ago @lmajewski no zeus, I'm using dunfell as mentioned with python2. So I'm using gcc 10.1.
For zeus it builds fine (with older meta-rust 5d1ada0c97723e1526bf5599b2fa2cbb56c2c0dc)
ashie
commented
4 years ago For python2 issue in dunfell, even using pythonnative didn't work for me, only adding python to HOSTTOOLS worked for me on Ubuntu 20.04,
I've confirmed it with dunfell & meta-python2. The python2.7 command is placed under:
and PATH is not set for it while most other commands are placed under:
kraj
commented
4 years ago arent we using inherit pythonnative
shr-project
commented
4 years ago arent we using inherit pythonnative
We currently aren't and adding it doesn't work.
ashie
commented
4 years ago As @shr-project mentioned adding python-native to DEPENDS takes no effect.
ashie
commented
4 years ago arent we using inherit pythonnative
Ah, I got it. meta-python2/classes/pythonnative.bbclass provides it.
EXTRANATIVEPATH += "python-native"
DEPENDS_append = " python-native "arent we using inherit pythonnative
It's still failing with the following error:
| 0:00.34 File "/build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/recipe-sysroot-native/usr/lib/python2.7/distutils/sysconfig.py", line 424, in _init_posix
| 0:00.34 raise DistutilsPlatformError(my_msg)
| 0:00.34 distutils.errors.DistutilsPlatformError: invalid Python installation: unable to open /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/recipe-sysroot/usr/lib/python2.7/config/Makefile (No such file or directory)It should be recipe-sysroot-native?
shr-project
commented
4 years ago It probably should be recipe-sysroot-native, because even if you add target python to resolve this issue it will try to use the target library from there and fail on host
0:01.82 /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/firefox/68.0esr-r0/firefox-68.0/firefox-build-dir/_virtualenvs/init/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directoryas shown in top 2 commits of https://github.com/shr-project/meta-browser/commits/jansa/python
ashie
commented
4 years ago meta-python2/classes/pythonnative.bbclass:
# python-config and other scripts are using distutils modules
# which we patch to access these variables
export STAGING_INCDIR
export STAGING_LIBDIRProbably it should be like this
export STAGING_INCDIR="${STAGING_INCDIR_NATIVE}"
export STAGING_LIBDIR="${STAGING_LIBDIR_NATIVE}"The former one cannot run python-config but the later one can do it.
ashie
commented
4 years ago But I still got another error:
| 0:00.34 IOError: [Errno 13] Permission denied: '/build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/firefox-build-dir/
_virtualenvs/init/lib/python2.7/lib-dynload/readline.so' It points system's one
$ LANG=C ls -l /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/firefox-build-dir/_virtualenvs/init/lib/python2.7/lib-dynload
lrwxrwxrwx 1 aho aho 30 Jun 4 08:28 /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/firefox-build-dir/_virtualenvs/init/lib/python2.7/lib-dynload -> /usr/lib/python2.7/lib-dynload@shr-project @ashie Do you have any idea how this issue could be fixed? With newest -master for firefox and rust updated to 1.39 I still experience this error:
9:07.43 cargo:rerun-if-changed=properties/data.py
9:07.43 cargo:rerun-if-changed=properties/gecko.mako.rs
9:07.43 --- stderr
9:07.43 Could not find platform independent libraries <prefix>
9:07.43 Could not find platform dependent libraries <exec_prefix>
9:07.43 Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
9:07.43 ImportError: No module named site
9:07.43 warning: build failed, waiting for other jobs to finish...
9:22.98 config/external/nspr/pr
I've tried to:
sysroot-native for firefox a link to /usr/bin/python2Unfortunately, I still see this error (on Centos 7/Fedora 31). What is strange it works (and compiles on Ubuntu 16.04 and Debian).
A side question:
Is there any way to increase the debug output from mach script?
The modification of ./meta-browser/class/mozilla.bblass to have mach -v build -v doesn't bring any more useful debug output.
lmajewski
commented
4 years ago Workaround for mentioned in this thread problem with firefox_68esr build:
The culprit is: firefox/68.0esr-r0/firefox-68.0/servo/components/style
I'm using thud YPRR on Centos 7.6.1804, but the code works with Debian 9/10, Ubuntu 18.04
The style component 68.0esr-r0/firefox-68.0/servo/components/style of
firefox requires python 2.7 to be correctly build
In my setup the python is really old 2.7.5 (OE/Yocto's host python - recipe-sysroot-native is 2.7.16).
Python relevant directories in the PATH for running cargo to build servo: ./firefox/68.0esr-r0/recipe-sysroot-native/usr/bin ./build/tmp/hosttools
And servo in its build.rs searches for python2.7, so it finds it in hosttools,
which in my case is the python 2.7.5 (old one) - then it complains about missing
site.py module.
On 'thud' YPRR (2.6) - the PATH passed to cargo, doesn't have path to python2.7, which is installed in recipe-sysroot-native/usr/bin/python-native and fails with python from hostools.
Adding 'include pythonnative' causes errors related to virtualenv setup with -c configure of firefox_68esr.
The workaround for the firefox build system oddities, which worked on thud (2.6) and Centos/Fedora: https://github.com/lmajewski/meta-browser/commit/7f2d1766df83d73b57356e96c47c8d953ac9066c
This workaround works for centos 7.6 and Ubuntu 18.04.
otavio
commented
4 years ago Could you prepare a PR fixing this so we can go over a review?
ashie
commented
4 years ago I've retried this issue with these patches (I'm trying to fix this issue without HOSTTOOLS hack):
for meta-python2:
diff --git a/classes/pythonnative.bbclass b/classes/pythonnative.bbclass
index 0e9019d..ce30636 100644
--- a/classes/pythonnative.bbclass
+++ b/classes/pythonnative.bbclass
@@ -9,8 +9,8 @@ DEPENDS_append = " python-native "
# python-config and other scripts are using distutils modules
# which we patch to access these variables
-export STAGING_INCDIR
-export STAGING_LIBDIR
+export STAGING_INCDIR="${STAGING_INCDIR_NATIVE}"
+export STAGING_LIBDIR="${STAGING_LIBDIR_NATIVE}"
# Packages can use
# find_package(PythonInterp REQUIRED)for meta-browser:
iff --git a/classes/mozilla.bbclass b/classes/mozilla.bbclass
index 4fcb61d..e9ad612 100644
--- a/classes/mozilla.bbclass
+++ b/classes/mozilla.bbclass
@@ -1,9 +1,9 @@
SECTION = "x11/utils"
-DEPENDS += "gnu-config-native virtual/libintl libxt libxi zip-native gtk+"
+DEPENDS += "gnu-config-native virtual/libintl libxt libxi zip-native gtk+ python-native"
SRC_URI += "file://mozconfig"
-inherit pkgconfig
+inherit pkgconfig pythonnative
EXTRA_OECONF = "--target=${TARGET_SYS} --host=${BUILD_SYS} \
--with-toolchain-prefix=${TARGET_SYS}- \Now I've got the following error:
| 0:00.29 Creating Python environment | 0:00.41 New python executable in /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/firefox-build-dir/_virtualenvs/init/bin/python
| 0:00.41 Installing setuptools, pip, wheel...
| 0:00.41 Complete output from command /build/master/build/...envs/init/bin/python - setuptools pip wheel:
| 0:00.41 Traceback (most recent call last):
| 0:00.41 File "<stdin>", line 8, in <module>
| 0:00.41 File "/build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/third_party/python/virtualenv/virtualenv_support/pip-9.0.3-py2.py3-none-any.whl/pip/__init__.py", line 43, in <module>
| 0:00.41 File "/build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/third_party/python/virtualenv/virtualenv_support/pip-9.0.3-py2.py3-none-any.whl/pip/utils/__init__.py", line 23, in <module>
| 0:00.41 File "/build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/third_party/python/virtualenv/virtualenv_support/pip-9.0.3-py2.py3-none-any.whl/pip/locations.py", line 10, in <module>
| 0:00.42 File "/build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/recipe-sysroot-native/usr/lib/python2.7/distutils/command/install.py", line 25, in <module>
| 0:00.42 libname = sys.lib
| 0:00.42 AttributeError: 'module' object has no attribute 'lib' Original Python doesn't have lib attribute in sys module, it's OE specific feature.
https://git.openembedded.org/meta-python2/tree/recipes-devtools/python/python-native/multilib.patch#n215
It means that system's python is used instead of meta-python2's one.
But when I compared both meta-python2's binary and virutualenv's one, their checksum is same (and different with the system's one).
$ sha256sum /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/recipe-sysroot-native/usr/bin/python-native/python2.7
eb4dee7e4a4c6ea6404bfd2e7ce25943bab500224c4b092d2ab1d1b77039cc0a /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/recipe-sysroot-native/usr/bin/python-native/python2.7
$ sha256sum /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/firefox-build-dir/_virtualenvs/init/bin/python2.7
eb4dee7e4a4c6ea6404bfd2e7ce25943bab500224c4b092d2ab1d1b77039cc0a /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/firefox-build-dir/_virtualenvs/init/bin/python2.7On the other hand, virtualenv's one is linked with system's libpython
$ ldd /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/recipe-sysroot-native/usr/bin/python-native/python2.7
....
libpython2.7.so.1.0 => /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/recipe-sysroot-native/usr/bin/python-native/../../lib/libpython2.7.so.1.0 (0x00007ffff7dab000)
....
$ ldd /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/firefox-68.9.0/firefox-build-dir/_virtualenvs/init/bin/python2.7
....
libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007ffff7c39000)
....So I doubted & checked rpath in the python binary:
$ readelf -d -w /build/master/build/tmp/work/core2-64-poky-linux/firefox/68.9.0esr-r0/recipe-sysroot-native/usr/bin/python-native/python2.7
...
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../../lib:$ORIGIN/../../../lib]
...Probably this is the cause of this issue. virutualenv copies the python binary to the new directory but the relative path to the libpython isn't preserved. As a result rpath isn't resolved then system's libpython is used as a fallback.
But currently I have no I idea how to fix it.
ashie
commented
4 years ago virutualenv copies the python binary to the new directory but the relative path to the libpython isn't preserved. As a result rpath isn't resolved then system's libpython is used as a fallback.
Even I passed the issue by modifying rpath (or adding LD_LIBRARY_PATH or doing any other equivalents), I got another error:
| 0:33.28 error[E0463]: can't find crate for `core`
| 0:33.28 |
| 0:33.28 = note: the `x86_64-poky-linux` target may not be installed
| 0:33.28 error: aborting due to previous error
| 0:33.28 For more information about this error, try `rustc --explain E0463`.
| 0:33.28 error: Could not compile `cfg-if`. It doesn't occur when I use HOSTTOOLS=python2.7 on Ubuntu 20.20.
Hmm, using pythonnative is hard ;-(
lmajewski
commented
4 years ago This is strange for me - HOSTTOOLS=python2.7 doesn't work for me on Centos/Fedora and Thud. However, I can build firefox 68esr on Thud (2.6) without setting anything (HOSTTOOLS and other hacks) on Debian10 and Ubuntu 18.04 (which are recommended for this YPRR).
Hi I'm a bit new to firefox/rust, but it fails to build for armv7.
It seems to be missing some python module "site" ? Any ideas?