Open Artim96 opened 10 months ago
Yes, since v0.10.1, CoBang has been broken in production (after releasing to FlatHub), while is still Ok if run in development. I'm still trying to fix. It is difficult though, because it can only reproducible after releasing to FlatHub.
Could you try the v0.10.3? It is not listed on FlatHub yet, but can still try the test build as in https://github.com/flathub/vn.hoabinh.quan.CoBang/pull/9#issuecomment-1869690339
From https://github.com/hongquan/CoBang/issues/76#issuecomment-1870230462, this still exists, though I cannot reproduce it.
From #76 (comment), this still exists, though I cannot reproduce it.
I have recorded an strace log if that helps, after completely uninstalling CoBang and deleting its data and reinstalling it from flathub stable. cobang_strace.txt
@Artim96 Looking into your log, I find something weird. Your log said that CoBang is running with Python 3.10, but the base Flatpak image of the v0.10.3 comes with Python 3.11. Maybe your system still install the old version of CoBang?
Ah, the strace.txt log is newer, it is Python 3.11.
@Artim96 CoBang v0.10.4 has been released, with Python dependencies being upgraded. Could you test if the issue of missing brotli still happen?
It doesn't appear on FlatHub yet, you can install by the command in https://github.com/flathub/vn.hoabinh.quan.CoBang/pull/10#issuecomment-1876477033
Again, same error.
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/app/lib/python3.11/site-packages/cobang/__main__.py", line 21, in <module>
from .app import CoBangApplication
File "/app/lib/python3.11/site-packages/cobang/app.py", line 33, in <module>
from . import ui
File "/app/lib/python3.11/site-packages/cobang/ui.py", line 16, in <module>
from .resources import get_ui_filepath
File "/app/lib/python3.11/site-packages/cobang/resources.py", line 8, in <module>
import requests
File "/app/lib/python3.11/site-packages/requests/__init__.py", line 43, in <module>
import urllib3
File "/app/lib/python3.11/site-packages/urllib3/__init__.py", line 17, in <module>
from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool, connection_from_url
File "/app/lib/python3.11/site-packages/urllib3/connectionpool.py", line 15, in <module>
from ._request_methods import RequestMethods
File "/app/lib/python3.11/site-packages/urllib3/_request_methods.py", line 10, in <module>
from .response import BaseHTTPResponse
File "/app/lib/python3.11/site-packages/urllib3/response.py", line 284, in <module>
class BaseHTTPResponse(io.IOBase):
File "/app/lib/python3.11/site-packages/urllib3/response.py", line 294, in BaseHTTPResponse
DECODER_ERROR_CLASSES += (brotli.error,)
^^^^^^^^^^^^
AttributeError: module 'brotli' has no attribute 'error'
The Flatpak package of CoBang is supposed to not include Brotli, its sandbox environment should not have Brotli Python library. But somehow your sandbox environment has an old version of Brotli. I wonder where it is from, maybe the host system leaks to Flatpak sandbox?
Could you help me debug your sandbox to see where brotli
comes from:
Access CoBang sandbox shell:
$ flatpak run --command=sh vn.hoabinh.quan.CoBang
Inside the sandbox, cd
to /app/
then run python3
.
In Python REPL, do this:
>>> import brotli
>>> brotli.__file__
and
>>> import brotlicffi
>>> brotlicffi.__file__
python3
Python 3.11.7 (main, Nov 10 2011, 15:00:00) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import brotli
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'brotli'
>>> import brotlicffi
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'brotlicffi'
I also just went through a few flatpak directories with ripgrep-all to find all mentions of brotli. Here's what I found, maybe that might help.
rga "brotli" .local/share/flatpak/app/vn.hoabinh.quan.CoBang/
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/pkgconfig/libsoup-2.4.pc
Requires.private: sysprof-capture-4, libxml-2.0, sqlite3, libpsl >= 0.20, libbrotlidec, zlib
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/pkgconfig/libsoup-gnome-2.4.pc
Requires.private: sysprof-capture-4, glib-2.0 >= 2.58, gmodule-2.0 >= 2.58, gobject-2.0 >= 2.58, gio-2.0 >= 2.58, libxml-2.0, sqlite3, libpsl >= 0.20, libbrotlidec, zlib
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/Logbook-1.7.0.post0.dist-info/METADATA
Requires-Dist: brotli ; extra == 'compression'
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/Pillow-10.1.0.dist-info/LICENSE
BROTLI
Copyright (c) 2009, 2010, 2013-2016 by the Brotli Authors.
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/Pillow-10.1.0.dist-info/RECORD
Pillow.libs/libbrotlicommon-3ecfe81c.so.1,sha256=AkvHYFUCz_1Fs_fD83_gQ2lMw7S0y385rzuacnk-TC4,144425
Pillow.libs/libbrotlidec-922c819b.so.1,sha256=CGnqJ6LQOPrJ57Pf2TUjXYf7UdIu2DbE5s4ZY-CXXWQ,58225
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/urllib3/util/request.py
import brotlicffi as _unused_module_brotli # type: ignore[import] # noqa: F401
import brotli as _unused_module_brotli # type: ignore[import] # noqa: F401
``True`` translates to 'gzip,deflate'. If either the ``brotli`` or
``brotlicffi`` package is installed 'gzip,deflate,br' is used instead.
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/urllib3-2.1.0.dist-info/METADATA
Provides-Extra: brotli
Requires-Dist: brotli>=1.0.9; platform_python_implementation == 'CPython' and extra == 'brotli'
Requires-Dist: brotlicffi>=0.8.0; platform_python_implementation != 'CPython' and extra == 'brotli'
- Support for gzip, deflate, brotli, and zstd encoding.
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/urllib3/response.py
import brotlicffi as brotli # type: ignore[import]
import brotli # type: ignore[import]
brotli = None
if brotli is not None:
class BrotliDecoder(ContentDecoder):
# Supports both 'brotlipy' and 'Brotli' packages
# are for 'brotlipy' and bottom branches for 'Brotli'
self._obj = brotli.Decompressor()
if brotli is not None and mode == "br":
return BrotliDecoder()
if brotli is not None:
if brotli is not None:
DECODER_ERROR_CLASSES += (brotli.error,)
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/logbook/handlers.py
class BrotliCompressionHandler(FileHandler):
from brotli import Compressor
"The brotli library is required for the BrotliCompressionHandler."
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/logbook/__init__.py
BrotliCompressionHandler,
And looking for files or directories containing brotli in their name, I found this:
find .local/share/flatpak/app/vn.hoabinh.quan.CoBang/ -iname "*brotli*"
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/Pillow.libs/libbrotlidec-922c819b.so.1
.local/share/flatpak/app/vn.hoabinh.quan.CoBang/x86_64/test/57aabcce73d34c75b287a06a52673f0f7b691f35074575534a04ec5c33f9c56f/files/lib/python3.11/site-packages/Pillow.libs/libbrotlicommon-3ecfe81c.so.1
Your Python shell shows that brotli
Python module does not exist in the sandbox. That is expected. But somehow this code goes through:
if brotli is not None:
DECODER_ERROR_CLASSES += (brotli.error,)
So weird.
Could you run Python REPL in the sandbox again and try to run parts of urllib3's code, to see how the brotli
variable is.
First, run all the import
lines, and until this line https://github.com/urllib3/urllib3/blob/69be2992f8a25a1f27e49f339e4d5b98dec07462/src/urllib3/response.py#L39
After this step, please check how brotli
is.
Second, run these lines.
python3
Python 3.11.7 (main, Nov 10 2011, 15:00:00) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import annotations
>>> import collections
>>> import io
>>> import json as _json
>>> import logging
>>> import re
>>> import sys
>>> import typing
>>> import warnings
>>> import zlib
>>> from contextlib import contextmanager
>>> from http.client import HTTPMessage as _HttplibHTTPMessage
>>> from http.client import HTTPResponse as _HttplibHTTPResponse
>>> from socket import timeout as SocketTimeout
>>> try:
... import zstandard as zstd
... _zstd_version = _zstd_version = tuple(
... map(int, re.search(r"^([0-9]+)\.([0-9]+)", zstd.__version__).groups())
... )
... if _zstd_version < (0, 18):
... zstd = None
... except (AttributeError, ImportError, ValueError): # Defensive:
... zstd = None
...
>>> import brotli
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'brotli'
>>> import brotlicffi
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'brotlicffi'
>>> class BaseHTTPResponse(io.IOBase):
... CONTENT_DECODERS = ["gzip", "x-gzip", "deflate"]
... if brotli is not None:
... CONTENT_DECODERS += ["br"]
... if zstd is not None:
... CONTENT_DECODERS += ["zstd"]
... REDIRECT_STATUSES = [301, 302, 303, 307, 308]
... DECODER_ERROR_CLASSES: tuple[type[Exception], ...] = (IOError, zlib.error)
... if brotli is not None:
... DECODER_ERROR_CLASSES += (brotli.error,)
... if zstd is not None:
... DECODER_ERROR_CLASSES += (zstd.ZstdError,)
...
>>> import brotli
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'brotli'
>>> brotli.__file__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute '__file__'. Did you mean: '__le__'?
Thank you, looking into this
>>> import brotli
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'brotli'
>>> brotli.__file__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute '__file__'. Did you mean: '__le__'?
I want to cry.
Right previously, Python said that ModuleNotFoundError: No module named 'brotli'
, but then in next line of code, it considers brotli
to be NoneType
object!
-- Update --
Ah, the result:
>>> brotli.__file__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute '__file__'. Did you mean: '__le__'?
is because brotli
variable had been assigned None
at https://github.com/urllib3/urllib3/blob/69be2992f8a25a1f27e49f339e4d5b98dec07462/src/urllib3/response.py#L23
I notice that when you run the definition code of BaseHTTPResponse(io.IOBase)
manually in Python shell, Python doesn't raise error at
DECODER_ERROR_CLASSES += (brotli.error,)
So in this shell test, the code behaves as expected.
Sadly, I also can't test if there may be leakage from the OS into the Flatpak container. I have python3-brotli installed (on Debian) because it's required by python3-urllib3 which itself is required e.g. by gnome-music and gnome-browser-connector. And both are required by gnome itself. So Brotli as a Python package may be present on any system with Gnome installed.
I use Ubuntu and also have python3-brotli
installed, but it is not leaked to Flatpak sandbox. So weird. I will try changing the Flatpak build script to include Brotli
, to see how things will be.
Hi. I has released v0.10.5, where I include Python Brotli lib. Could you help test?
When I'm able to update, sure. Right now I only get the error
error: Can't load uri https://dl.flathub.org/build-repo/74100/vn.hoabinh.quan.CoBang.flatpakref: Server returned status 404
so it seems the server is down right now.
`
0.10.5 was now released. It works now with the only error message being
** (gst-plugin-scanner:7): CRITICAL **: 10:01:35.246: Couldn't g_module_open libpython. Reason: /usr/lib/x86_64-linux-gnu/libpython3.11.so: cannot open shared object file: No such file or directory
Though the UI seems a bit wonky. In maximized screen you have the results bar in the right, but when scanning e.g. a WiFi QR Code generated by Android (by the settings app, at least on Pixel devices, no idea if that's a Pixel or an Android feature), the results bar moves to the bottom edge to very litlle height with the content showing up in raw results underneath it. The scaling of the raw results bar is off though. It's barely high enough for the text, but the copy button is being cut off. In not maximized screen, there's also only the narrow results bar at the bottom with the raw results underneath, but here the text box is even narrower, also cutting off the top of the text.
But that would be something for a different issue.
Thank you. The UI issue is known. It is the result of my attempt to make the UI responsive, working both on desktop screen and mobile (Linux phone) screen. To workaround it, I will work on a feature of remember window size.
The message of "missing libpython3.11.so" is undesired, though.
While I can't tell if my last issue was fixed with this release, there's now another issue with 0.10.2 preventing the app from even opening: