kraj
commented
5 years ago you need to add
IMAGE_INSTALL_append = " python3-fcntl"Closed miikeat closed 5 years ago
kraj
commented
5 years ago you need to add
IMAGE_INSTALL_append = " python3-fcntl"
miikeat
commented
5 years ago Wow ... now I'm impressed... I seriously have no idea how that works!
I also saw that hint somewhere earlier but as there is no recipe and bitbake python3-fcntl throws out an error:
ERROR: Nothing PROVIDES 'python3-fcntl'. Close matches:
python3-prctl
python3-can
python3-ntplib
python3 RPROVIDES python3-fcntl I thought someone didn't care to check if that recipe even exists and just wrote that...
But now i noticed the last line of the error...
Where is this RPROVIDES coming from in the python3 recipe?
Also, shouldn't that module be included by default, same as pydoc and probably some other default modules? In Python 2 they are included and on x86 machines as well! Whats the reason that they are missing on ARM?
kraj
commented
5 years ago a recipe can provide many packages, and when you install into image you install packages not recipes. Why many packages, because in some cases only portions are needed so we can keep the size in check. bitbake
miikeat
commented
5 years ago yeah .. i kind of figured that out in the meantime, that via the manifest.json these packages are generated dynamically...
But shouldn't the behaviour of the python3 recipe be the same on x86 and arm? why do i need to manually pick those packages on arm but on x86 they are included when only installing python3!
But thanks for the explanation! :+1:
agherzan
commented
5 years ago Because you are using a RPi specific image which packages a set for packages. What image are you comparing with X86?
miikeat
commented
5 years ago I hope if it's ok if i continue asking here out of pure curiosity:
I have one qemux86 image with just poky and i build core-image-base there.
And one raspberrypi3 image with poky, meta-openembedded and meta-raspberrypi where i build as well core-image-base. In both images i add python3 and only in the qemux86 image i have all modules.
Neither meta-raspberrypi nor meta-openembedded have a bbappend for python3 so i just wonder how it comes, that for raspberrypi i'm missing those modules?
I also looked through the machine configurations and could not find any clue what is causing these packages to be missing on raspberrypi.
Whats also kind of irritating, python 2.x contains all those modules by default!
So I don't really see where this behaviour comes from and if that is really intended that way...
agherzan
commented
5 years ago Can you try using the same build configuration to build core-image-base for both machines? Basically, run twice bitbake core-image-base with a different machine in local.conf
miikeat
commented
5 years ago I had the same idea and I'm already building it... =)
In a couple of hours I should know more!
And if the modules are also missing in that image I'll remove the meta-raspberrypi layer and rebuild it again. That way it should be clear which layer is responsible for the reduced number of modules.
miikeat
commented
5 years ago The results:
Build Configuration:
BB_VERSION = "1.40.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "neon-18.04"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.6.1"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "thud:f5a57e939e626a5b7c6de5b51799ca602ed355ed"
meta-oe
meta-multimedia
meta-python
meta-networking = "thud:6ef9657068492d4644079c88f2adee9c3cac9543"
meta-raspberrypi = "thud:c71d79efc5a06a0c896c278c94f5b14413bb4d69"
root@qemux86:~# pip3 install gunicorn
Collecting gunicorn
Downloading https://files.pythonhosted.org/packages/8c/da/b8dd8deb741bff556db53902d4706774c8e1e67265f69528c14c003644e6/gunicorn-19.9.0-py2.py3-none-any.whl (112kB)
100% |################################| 122kB 557kB/s
Installing collected packages: gunicorn
Successfully installed gunicorn-19.9.0
root@qemux86:~# gunicorn
usage: gunicorn [OPTIONS] [APP_MODULE]
gunicorn: error: No application module specified.
root@qemux86:~# So here fcntl is available again even though the build configuration is exactly the same except the machine.
miikeat
commented
5 years ago As help() -> modules in python console lists the importable modules here a comparison between the two images:
raspberrypi3:
root@raspberrypi3:~# python3
Python 3.5.6 (default, Mar 6 2019, 15:43:23)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help()
Welcome to Python 3.5's help utility!
If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/3.5/tutorial/.
Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".
To get a list of available modules, keywords, symbols, or topics, type
"modules", "keywords", "symbols", or "topics". Each module also comes
with a one-line summary of what it does; to list the modules whose name
or summary contain a given string such as "spam", type "modules spam".
help> modules
Please wait a moment while I gather a list of all available modules...
__future__ _weakrefset gunicorn reprlib
_ast abc gzip rlcompleter
_bisect argparse hashlib select
_bootlocale array heapq selectors
_bz2 ast hmac setuptools
_codecs asyncio html shelve
_collections atexit http shlex
_collections_abc base64 imaplib shutil
_compat_pickle binascii imp signal
_compression bisect importlib site
_crypt builtins inspect sitecustomize
_csv bz2 io smtplib
_ctypes calendar ipaddress socket
_ctypes_test cgi itertools socketserver
_datetime cmath json sre_compile
_decimal cmd keyword sre_constants
_dummy_thread code linecache sre_parse
_elementtree codecs locale ssl
_functools codeop logging stat
_hashlib collections lzma string
_heapq compileall marshal stringprep
_imp concurrent math struct
_io configparser mimetypes subprocess
_json contextlib multiprocessing symbol
_locale copy nntplib sys
_lzma copyreg numbers sysconfig
_markupbase crypt opcode tarfile
_multiprocessing csv operator telnetlib
_opcode ctypes optparse tempfile
_operator datetime os termios
_pickle decimal parser textwrap
_posixsubprocess difflib pickle threading
_pydecimal dis pickletools time
_pyio distutils pip token
_random dummy_threading pipes tokenize
_sha256 easy_install pkg_resources traceback
_sha512 email pkgutil types
_signal encodings platform unicodedata
_sitebuiltins enum plistlib unittest
_socket errno poplib urllib
_sre faulthandler posix uu
_ssl fnmatch posixpath uuid
_stat formatter pprint warnings
_string fractions pwd weakref
_strptime ftplib py_compile xml
_struct functools pydoc xmlrpc
_symtable gc pydoc_data xxsubtype
_sysconfigdata genericpath pyexpat zipfile
_thread getopt queue zipimport
_threading_local getpass quopri zlib
_tracemalloc gettext random
_warnings glob re
_weakref grp readline
Enter any module name to get more help. Or, type "modules spam" to search
for modules whose name or summary contain the string "spam".
help>qemux86:
root@qemux86:~# python3
Python 3.5.6 (default, Mar 7 2019, 21:11:37)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help()
Welcome to Python 3.5's help utility!
If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/3.5/tutorial/.
Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".
To get a list of available modules, keywords, symbols, or topics, type
"modules", "keywords", "symbols", or "topics". Each module also comes
with a one-line summary of what it does; to list the modules whose name
or summary contain a given string such as "spam", type "modules spam".
help> modules
Please wait a moment while I gather a list of all available modules...
CDROM _weakref heapq selectors
DLFCN _weakrefset hmac setuptools
IN abc html shelve
TYPES aifc http shlex
__future__ antigravity idlelib shutil
_ast argparse imaplib signal
_bisect array imghdr site
_bootlocale ast imp sitecustomize
_bz2 asynchat importlib smtpd
_codecs asyncio inspect smtplib
_codecs_cn asyncore io sndhdr
_codecs_hk atexit ipaddress socket
_codecs_iso2022 audioop itertools socketserver
_codecs_jp base64 json spwd
_codecs_kr bdb keyword sqlite3
_codecs_tw binascii lib2to3 sre_compile
_collections binhex linecache sre_constants
_collections_abc bisect locale sre_parse
_compat_pickle builtins logging ssl
_compression bz2 lzma stat
_crypt cProfile macpath statistics
_csv calendar macurl2path string
_ctypes cgi mailbox stringprep
_ctypes_test cgitb mailcap struct
_curses chunk marshal subprocess
_curses_panel cmath math sunau
_datetime cmd mimetypes symbol
_dbm code mmap symtable
_decimal codecs modulefinder sys
_dummy_thread codeop multiprocessing sysconfig
_elementtree collections netrc syslog
_functools colorsys nntplib tabnanny
_hashlib compileall ntpath tarfile
_heapq concurrent nturl2path telnetlib
_imp configparser numbers tempfile
_io contextlib opcode termios
_json copy operator textwrap
_locale copyreg optparse this
_lsprof crypt os threading
_lzma csv ossaudiodev time
_markupbase ctypes parser timeit
_md5 curses pathlib tkinter
_multibytecodec datetime pdb token
_multiprocessing dbm pickle tokenize
_opcode decimal pickletools trace
_operator difflib pip traceback
_osx_support dis pipes tracemalloc
_pickle distutils pkg_resources tty
_posixsubprocess doctest pkgutil turtle
_pydecimal dummy_threading platform turtledemo
_pyio easy_install plistlib types
_random email poplib typing
_sha1 encodings posix unicodedata
_sha256 ensurepip posixpath unittest
_sha512 enum pprint urllib
_signal errno profile uu
_sitebuiltins faulthandler pstats uuid
_socket fcntl pty venv
_sqlite3 filecmp pwd warnings
_sre fileinput py_compile wave
_ssl fnmatch pyclbr weakref
_stat formatter pydoc webbrowser
_string fractions pydoc_data wsgiref
_strptime ftplib pyexpat xdrlib
_struct functools queue xml
_symtable gc quopri xmlrpc
_sysconfigdata genericpath random xxlimited
_testbuffer getopt re xxsubtype
_testcapi getpass readline zipapp
_testimportmultiple gettext reprlib zipfile
_testmultiphase glob resource zipimport
_thread grp rlcompleter zlib
_threading_local gunicorn runpy
_tracemalloc gzip sched
_warnings hashlib select
Enter any module name to get more help. Or, type "modules spam" to search
for modules whose name or summary contain the string "spam".
help> This is a OE-Core bug please open a bugzilla entry https://bugzilla.yoctoproject.org/
I'm trying to host a rest api on my raspberrypi3. I wrote everything in python and used falcon, so i would like to host it using gunicorn which I installed using python3-pip. This works fine on my desktop, but not on my raspberrypi.
I am running a basic Yocto image where i just added
IMAGE_INSTALL_append = " python3 python3-pip"tolocal.conf. Everything else is as described in the README.The Errormessage states that the module 'fcntl' is missing which should be a builtin module. I also tried to figure out if this is a problem with the python recipe in general, but if i build a qemux86 image and run in qemu gunicorn works and python has the module 'fcntl' included.
I later stumbled across this issue: https://stackoverflow.com/questions/40047363/how-to-compile-missing-extensions-modules-when-cross-compiling-python-3-5-2-for which led me to this bugreport https://bugs.python.org/issue28444 which fits this issue pretty good but should be fixed since 3.5.2. Maybe for some reason that problem is back again...
Steps to reproduce the issue:
IMAGE_INSTALL_append = " python3 python3-pip"tolocal.conf)pip3 install gunicorngunicornDescribe the results you received:
Describe the results you expected: Gunicorn should print the usage message.
Additional information you deem important (e.g. issue happens only occasionally): According to a comment in the bugreport
sys.builtin_module_namesshould contain fcntl but on neither system that was the case. Builtin Modules in my raspberrypi image:I then found out
help('modules')lists all importable modules, which also works and contains 'fcntl' on the qemux86 machine. But on my raspberrypi i just get another error:Additional details (revisions used, host distro, etc.):
Additions to default
local.conf: