zlobober
commented
9 months ago @denvr, @savnadya This issue is quite annoying, might having a look?
Open zlobober opened 9 months ago
zlobober
commented
9 months ago @denvr, @savnadya This issue is quite annoying, might having a look?
thenno
commented
4 months ago @zlobober
It seems the issue was fixed in https://github.com/ytsaurus/ytsaurus/commit/f7db734cacc99fc679b824f7b7ce52cc785145b6
According to the first message there are "urllib3 (1.26.9) or chardet (None)/charset_normalizer (3.3.2)" ->
And right now I can't reproduce the problem:
docker run ubuntu:mantic bash -c "apt update; apt install -y python3 python3-pip; pip3 install ytsaurus-client --break-system-packages; yt"
.....
Downloading tqdm-4.66.4-py3-none-any.whl (78 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.3/78.3 kB 8.7 MB/s eta 0:00:00
Downloading typing_extensions-4.11.0-py3-none-any.whl (34 kB)
Installing collected packages: typing-extensions, tqdm, six, simplejson, distro, decorator, charset-normalizer, argcomplete, ytsaurus-client
Successfully installed argcomplete-3.3.0 charset-normalizer-3.3.2 decorator-4.4.2 distro-1.9.0 simplejson-3.19.2 six-1.16.0 tqdm-4.66.4 typing-extensions-4.11.0 ytsaurus-client-0.13.14
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
usage: yt [-h] [--proxy PROXY] [--prefix PREFIX] [--config CONFIG] [--tx TX]
[--master-cell-id MASTER_CELL_ID] [--ping-ancestor-txs] [--trace]
[--version]
command ...
yt: error: the following arguments are required: commandThere is another warning WARNING: Running pip as the 'root', but it only appears when you call pip with --break-system-packages.
# chiffa @ chiffa-osx in ~/Code/yt-client-chardet-warning-222 [20:32:49]
$ cat Dockerfile
FROM ubuntu:mantic
RUN apt update && apt install -y python3 python3-pip
RUN pip3 install ytsaurus-client --break-system-packages
CMD ["/bin/bash"]
# chiffa @ chiffa-osx in ~/Code/yt-client-chardet-warning-222 [20:32:53]
$ docker build -t yt-client-chardet-warning-222 .
[+] Building 0.0s (7/7) FINISHED docker:orbstack
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 225B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:mantic 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/3] FROM docker.io/library/ubuntu:mantic 0.0s
=> CACHED [2/3] RUN apt update && apt install -y python3 python3-pip 0.0s
=> CACHED [3/3] RUN pip3 install ytsaurus-client --break-system-packages 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:ebc2a5f3198f126340912f47a8cd901283fef51a4a40a59f1ced6354d42f9b45 0.0s
=> => naming to docker.io/library/yt-client-chardet-warning-222 0.0s
# chiffa @ chiffa-osx in ~/Code/yt-client-chardet-warning-222 [20:33:19]
$ docker run -it yt-client-chardet-warning-222
root@509f745d8c8e:/# yt
usage: yt [-h] [--proxy PROXY] [--prefix PREFIX] [--config CONFIG] [--tx TX] [--master-cell-id MASTER_CELL_ID] [--ping-ancestor-txs] [--trace] [--version] command ...
yt: error: the following arguments are required: commandProblem has been reproduced when a user has fresh chardet in a python environment.
https://github.com/ytsaurus/ytsaurus/commit/6f6b729862cc45b867134960e7e138543789c810 should fix this issue.
On Ubuntu 23.10 ytsaurus-client reports the followin warning whenever yt.wrapper is imported:
I suspect the following logic to be broken (or at least not be ready for fresh library versions): https://github.com/ytsaurus/ytsaurus/blob/main/yt/python/contrib/python-requests/requests/__init__.py#L51-L98
ytsaurus-client does not vendor charset_normalizer nor chardet, so they are taken from the host environment. At the same time, ytsaurus-client fixes the charset-normalizer version >=3.3.0, which contradicts the assertion from the check above.
https://github.com/ytsaurus/ytsaurus/blob/e11de159f4964a42fd33380a0a718ac9a2bafb72/yt/python/packages/ytsaurus-client/setup.py#L49
Finally, ytsaurus-client does not have a dependency on chardet, but it somehow appeared in my system (possibly, as a dependency of some other package), and it is also of much fresher version (5.1.0), which also does not fit into the requirements of the code above.
I am not sure, which one of these two libraries is actually used, but my YT client works perfectly fine for a couple of months of active use, so I'd suggest relaxing these checks one way or another. At the very least, two statements for charset_normalizer should be contradicting.
Reproduce this issue by running the following command: