Closed jwaltrip4 closed 9 months ago
What is the command you are using to attempt to run the script?
I suspect you are running something like:
grab-all-posters.py
rather than:
python grab-all-posters.py
as that's the only way I can reproduce this.
I think that was it. Cant test further until tomorrow
Gettingthis error at the top:
/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (2.0.7) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
Also... the run of the script did not produce any results. assets folder ie empty.
what version of python is this? Are you running this in a venv? Can you provide the output of:
python --version
and
python -m pip freeze
I've never seen that error.
I suspect the requirements are not installed or are not current.
On my test machine:
python --version
Python 3.10.12
python -m pip freeze
about-time==3.1.1
alive-progress==2.4.1
certifi==2023.7.22
cffi==1.15.1
cfgv==3.4.0
charset-normalizer==3.2.0
cinemagoer==2023.5.1
cryptography==41.0.3
distlib==0.3.7
filelock==3.12.3
filetype==1.2.0
grapheme==0.6.0
greenlet==2.0.2
identify==2.5.27
idna==3.4
IMDbPY==2022.7.9
lxml==4.9.3
nodeenv==1.8.0
pathvalidate==3.1.0
piexif==1.1.3
Pillow==10.1.0
platformdirs==3.10.0
PlexAPI==4.15.0
pre-commit==3.3.3
pycparser==2.21
pyOpenSSL==23.2.0
python-dotenv==1.0.0
PyYAML==6.0.1
requests==2.31.0
ruamel.yaml==0.17.32
ruamel.yaml.clib==0.2.7
SQLAlchemy==1.4.49
tabulate==0.9.0
tmdbapis==1.2.0
tmdbsimple==2.9.1
tmdbv3api==1.9.0
tvdb-v4-official==1.1.0
typing_extensions==4.7.1
urllib3==2.0.4
validators==0.22.0
virtualenv==20.24.3
Not running VENV on this. should I?
Python 3.8.10
python -m pip freeze
about-time==4.2.1
acme==1.1.0
alive-progress==3.1.5
anyio==4.0.0
async-timeout==4.0.3
asyncclick==8.1.3.4
asyncio==3.4.3
attrs==19.3.0
Automat==0.8.0
bernhard==0.2.6
blinker==1.4
bottle==0.12.25
cassandra-driver==3.28.0
certbot==0.40.0
certbot-apache==0.39.0
certifi==2019.11.28
cfgv==3.4.0
chardet==3.0.4
chevron==0.14.0
cinemagoer==2023.5.1
Click==7.0
cloud-init==23.3.3
colorama==0.4.3
command-not-found==0.3
ConfigArgParse==0.13.0
configobj==5.0.6
configshell-fb==1.1.25
constantly==15.1.0
CouchDB==1.2
cryptography==2.8
cupshelpers==1.0
dbus-python==1.2.16
defer==1.0.6
defusedxml==0.7.1
distlib==0.3.8
distro==1.4.0
distro-info==0.23+ubuntu1.1
dnspython==1.16.0
docker==4.1.0
elastic-transport==8.4.1
elasticsearch==8.10.1
entrypoints==0.3
exceptiongroup==1.1.3
filelock==3.13.1
filetype==1.2.0
future==0.18.2
geomet==0.2.1.post1
Glances==3.4.0.3
gpg==1.13.1
grapheme==0.6.0
graphitesender==0.11.2
greenlet==3.0.3
httplib2==0.14.0
hyperlink==19.0.0
identify==2.5.33
idna==2.8
ifaddr==0.2.0
IMDbPY==2022.7.9
importlib-metadata==1.5.0
incremental==16.10.1
influxdb==5.3.1
influxdb-client==1.38.0
Jinja2==2.10.1
josepy==1.2.0
jsonpatch==1.22
jsonpointer==2.0
jsonschema==3.2.0
kafka-python==2.0.2
keyring==18.0.1
language-selector==0.1
launchpadlib==1.10.13
lazr.restfulclient==0.14.2
lazr.uri==1.0.3
libvirt-python==6.1.0
linecache2==1.0.0
lxml==4.9.4
macaroonbakery==1.3.1
Markdown==3.1.1
MarkupSafe==1.1.0
meson==0.53.2
mock==3.0.5
more-itertools==4.2.0
msgpack==1.0.7
netifaces==0.10.4
nodeenv==1.8.0
oauthlib==3.1.0
packaging==20.3
paho-mqtt==1.6.1
parsedatetime==2.4
pathvalidate==3.2.0
pbkdf2==1.3
pbr==5.4.5
pexpect==4.6.0
piexif==1.1.3
pika==1.3.2
Pillow==10.1.0
platformdirs==4.1.0
PlexAPI==4.15.7
ply==3.11
potsdb==1.0.3
pre-commit==3.5.0
prometheus-client==0.17.1
protobuf==3.6.1
psutil==5.9.6
py3nvml==0.2.7
pyasn1==0.4.2
pyasn1-modules==0.2.1
pycairo==1.16.2
pycrypto==2.6.1
pycryptodomex==3.19.0
pycups==1.9.73
pydantic==1.10.13
pygal==3.0.0
Pygments==2.3.1
PyGObject==3.36.0
PyHamcrest==1.9.0
PyICU==2.4.2
PyJWT==1.7.1
pymacaroons==0.13.0
pymdstat==0.4.3
pymongo==4.5.0
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyparsing==2.4.6
PyQt5==5.14.1
pyRFC3339==1.1
pyrsistent==0.15.5
pyserial==3.4
pysmi==0.3.4
pysnmp==4.4.12
python-apt==2.0.1+ubuntu0.20.4.1
python-augeas==0.5.0
python-dateutil==2.8.2
python-debian==0.1.36+ubuntu1.1
python-dotenv==1.0.0
python-kasa==0.5.3
pytz==2019.3
pyudev==0.21.0
PyYAML==5.3.1
pyzmq==25.1.1
reactivex==4.0.4
requests==2.22.0
requests-toolbelt==0.8.0
requests-unixsocket==0.2.0
rtslib-fb==2.1.71
ruamel.yaml==0.18.5
ruamel.yaml.clib==0.2.8
scandir==1.10.0
SecretStorage==2.3.1
service-identity==18.1.0
simplejson==3.16.0
sip==4.19.21
six==1.14.0
sniffio==1.3.0
sos==4.5.6
SQLAlchemy==2.0.24
ssh-import-id==5.10
statsd==4.0.1
systemd-python==234
tabulate==0.9.0
targetcli-fb==2.1.51
testresources==2.0.0
tmdbapis==1.2.2
tmdbsimple==2.9.1
tmdbv3api==1.9.0
traceback2==1.4.0
tvdb-v4-official==1.1.0
Twisted==18.9.0
typing-extensions==4.8.0
ubuntu-advantage-tools==8001
ubuntu-drivers-common==0.0.0
ujson==5.8.0
unattended-upgrades==0.1
unittest2==1.1.0
urllib3==2.0.7
urwid==2.0.1
validators==0.22.0
virtualenv==20.25.0
wadllib==1.3.3
websocket-client==0.53.0
wifi==0.3.8
xkit==0.0.0
xmltodict==0.13.0
zeroconf==0.119.0
zipp==1.0.0
zope.component==4.3.0
zope.event==4.4
zope.hookable==5.0.0
zope.interface==4.7.1
I would generally recommend running python scripts in venvs to keep the setup clean. You're using an older version of requests than I am, which seems odd since other things are newer.
I haven't tried this with 3.8 in some time. I'll give it a try to see if there's some issue there.
I will switch to a VENV
Also... is this error why the assets folder is not being populated? Or have I left something out of the .env file?
What version of Python should I be using? I have a multi version set up.
I just created a new venv using Python 3.8.18, installed requirements, script ran fine.
I will switch to a VENV
Also... is this error why the assets folder is not being populated? Or have I left something out of the .env file?
The script doesn't appear to be running successfully, so I wouldn't expect anything to get downloaded.
OK.... Still not working, still same error at top.
Here is script
sudo update-alternatives --set python3 /usr/bin/python3.8
cd /mnt/Media01B/Media-Scripts
sudo python3 -m venv pmm-venv
source pmm-venv/bin/activate
cd Plex
sudo python3 grab-all-posters.py
deactivate
sudo update-alternatives --set python3 /usr/bin/python3.8
I have run requirements.txt in both the Media-Scripts folder, and the Plex subfolder
Here is the screen output
/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (2.0.7) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
Starting grab-all-posters 0.8.1 at 2023-12-29 11:12:43
================== ATTENTION ==================
Downloading disabled; file identification not possible
Script will default to .jpg extension on all images
================== ATTENTION ==================
Threaded downloads: False
connecting to http://192.168.30.170:32400...
Loading All Movies ...
Resetting rundate for All Movies to 2023-12-29 11:12:43.617273...
getting collections from [All Movies]...
209 collection(s) retrieved...
Grab Collection Posters |████████████████████████████████████████| 209/209 [100%] in 22.4s (9.35/s)
Loading movies ...
Completed loading 2500 of 2258 movie(s) from All Movies
Grab all posters All Movies |███▍ | ▂▂▄ 208/2500 [8%] in 3:21 (1.0/s, eta: 36:53)
assets folder is creaated under config, but nothing is in it.
Hmmm... changed
POSTER_DOWNLOAD=0
to
POSTER_DOWNLOAD=1
and now it is downloading.
That seems to be behaving as expected.
Did you install the requirements into the venv? I cannot reproduce this with python 3.8.18 if I start with a new clone of the repo, create a new venv, and install requirements:
/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (2.0.7) or chardet (3.0.4) doesn't match a supported version!
Yes I did... ran from both Media-Scripts folder and Media-Scripts/Plex folder
Will send output from both as soon as run completes.
Output 1:
(pmm-venv) jwaltrip@plex-media-svr2:/mnt/Media01B/Media-Scripts$ sudo pip install -r requirements.txt
/usr/bin/pip:6: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
from pkg_resources import load_entry_point
Requirement already satisfied: alive-progress==2.4.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 1)) (2.4.1)
Requirement already satisfied: imdbpy in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 2)) (2022.7.9)
Requirement already satisfied: pathvalidate in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 3)) (3.2.0)
Requirement already satisfied: PlexAPI in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 4)) (4.15.7)
Requirement already satisfied: pre-commit in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 5)) (3.5.0)
Requirement already satisfied: pyopenssl in /usr/lib/python3/dist-packages (from -r requirements.txt (line 6)) (19.0.0)
Requirement already satisfied: python-dotenv in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 7)) (1.0.0)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from -r requirements.txt (line 8)) (2.22.0)
Requirement already satisfied: ruamel.yaml in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 9)) (0.18.5)
Requirement already satisfied: tmdbapis in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 10)) (1.2.2)
Requirement already satisfied: tmdbsimple in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 11)) (2.9.1)
Requirement already satisfied: tmdbv3api in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 12)) (1.9.0)
Requirement already satisfied: tvdb_v4_official in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 13)) (1.1.0)
Requirement already satisfied: piexif in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 14)) (1.1.3)
Requirement already satisfied: filetype in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 15)) (1.2.0)
Requirement already satisfied: sqlalchemy<2.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 16)) (1.4.50)
Requirement already satisfied: tabulate in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 17)) (0.9.0)
Requirement already satisfied: validators in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 18)) (0.22.0)
Requirement already satisfied: about-time==3.1.1 in /usr/local/lib/python3.8/dist-packages (from alive-progress==2.4.1->-r requirements.txt (line 1)) (3.1.1)
Requirement already satisfied: grapheme==0.6.0 in /usr/local/lib/python3.8/dist-packages (from alive-progress==2.4.1->-r requirements.txt (line 1)) (0.6.0)
Requirement already satisfied: cinemagoer in /usr/local/lib/python3.8/dist-packages (from imdbpy->-r requirements.txt (line 2)) (2023.5.1)
Requirement already satisfied: cfgv>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from pre-commit->-r requirements.txt (line 5)) (3.4.0)
Requirement already satisfied: identify>=1.0.0 in /usr/local/lib/python3.8/dist-packages (from pre-commit->-r requirements.txt (line 5)) (2.5.33)
Requirement already satisfied: nodeenv>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from pre-commit->-r requirements.txt (line 5)) (1.8.0)
Requirement already satisfied: pyyaml>=5.1 in /usr/lib/python3/dist-packages (from pre-commit->-r requirements.txt (line 5)) (5.3.1)
Requirement already satisfied: virtualenv>=20.10.0 in /usr/local/lib/python3.8/dist-packages (from pre-commit->-r requirements.txt (line 5)) (20.25.0)
Requirement already satisfied: ruamel.yaml.clib>=0.2.7; platform_python_implementation == "CPython" and python_version < "3.13" in /usr/local/lib/python3.8/dist-packages (from ruamel.yaml->-r requirements.txt (line 9)) (0.2.8)
Requirement already satisfied: greenlet!=0.4.17; python_version >= "3" and (platform_machine == "aarch64" or (platform_machine == "ppc64le" or (platform_machine == "x86_64" or (platform_machine == "amd64" or (platform_machine == "AMD64" or (platform_machine == "win32" or platform_machine == "WIN32")))))) in /usr/local/lib/python3.8/dist-packages (from sqlalchemy<2.0->-r requirements.txt (line 16)) (3.0.3)
Requirement already satisfied: lxml in /usr/local/lib/python3.8/dist-packages (from cinemagoer->imdbpy->-r requirements.txt (line 2)) (4.9.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from nodeenv>=0.11.1->pre-commit->-r requirements.txt (line 5)) (68.2.2)
Requirement already satisfied: distlib<1,>=0.3.7 in /usr/local/lib/python3.8/dist-packages (from virtualenv>=20.10.0->pre-commit->-r requirements.txt (line 5)) (0.3.8)
Requirement already satisfied: filelock<4,>=3.12.2 in /usr/local/lib/python3.8/dist-packages (from virtualenv>=20.10.0->pre-commit->-r requirements.txt (line 5)) (3.13.1)
Requirement already satisfied: platformdirs<5,>=3.9.1 in /usr/local/lib/python3.8/dist-packages (from virtualenv>=20.10.0->pre-commit->-r requirements.txt (line 5)) (4.1.0)
Output 2:
(pmm-venv) jwaltrip@plex-media-svr2:/mnt/Media01B/Media-Scripts/Plex$ sudo pip install -r requirements.txt
/usr/bin/pip:6: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
from pkg_resources import load_entry_point
Requirement already satisfied: PlexAPI in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 1)) (4.15.7)
Requirement already satisfied: tmdbapis in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 2)) (1.2.2)
Requirement already satisfied: tmdbv3api in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 3)) (1.9.0)
Requirement already satisfied: python-dotenv in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 4)) (1.0.0)
Requirement already satisfied: tvdb_v4_official in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 5)) (1.1.0)
Requirement already satisfied: ruamel.yaml in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 6)) (0.18.5)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from -r requirements.txt (line 7)) (2.22.0)
Requirement already satisfied: pathvalidate in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 8)) (3.2.0)
Requirement already satisfied: imdbpy in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 9)) (2022.7.9)
Requirement already satisfied: alive_progress in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 10)) (2.4.1)
Requirement already satisfied: piexif in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 11)) (1.1.3)
Requirement already satisfied: filetype in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 12)) (1.2.0)
Requirement already satisfied: pillow in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 13)) (10.1.0)
Requirement already satisfied: ruamel.yaml.clib>=0.2.7; platform_python_implementation == "CPython" and python_version < "3.13" in /usr/local/lib/python3.8/dist-packages (from ruamel.yaml->-r requirements.txt (line 6)) (0.2.8)
Requirement already satisfied: cinemagoer in /usr/local/lib/python3.8/dist-packages (from imdbpy->-r requirements.txt (line 9)) (2023.5.1)
Requirement already satisfied: about-time==3.1.1 in /usr/local/lib/python3.8/dist-packages (from alive_progress->-r requirements.txt (line 10)) (3.1.1)
Requirement already satisfied: grapheme==0.6.0 in /usr/local/lib/python3.8/dist-packages (from alive_progress->-r requirements.txt (line 10)) (0.6.0)
Requirement already satisfied: SQLAlchemy in /usr/local/lib/python3.8/dist-packages (from cinemagoer->imdbpy->-r requirements.txt (line 9)) (1.4.50)
Requirement already satisfied: lxml in /usr/local/lib/python3.8/dist-packages (from cinemagoer->imdbpy->-r requirements.txt (line 9)) (4.9.4)
Requirement already satisfied: greenlet!=0.4.17; python_version >= "3" and (platform_machine == "aarch64" or (platform_machine == "ppc64le" or (platform_machine == "x86_64" or (platform_machine == "amd64" or (platform_machine == "AMD64" or (platform_machine == "win32" or platform_machine == "WIN32")))))) in /usr/local/lib/python3.8/dist-packages (from SQLAlchemy->cinemagoer->imdbpy->-r requirements.txt (line 9)) (3.0.3)
It's apparently seeing that those are already installed globally, so creating the venv after the initial install of those requirements isn't doing anything.
Were it happening to me I would delete the venv, then remove those packages from the system, then create the venv, then install requirements into the venv.
OK.... My ignorance is exposed :)
Deleting the VENV is as simple as deleting that folder, correct?
As to removing the packages from the system.... How do I do that?
https://stackoverflow.com/questions/11248073/how-do-i-remove-all-packages-installed-by-pip <== ??
Also.. I seem to have a bunch of requirements installed globally.... I assume maybe I dont want to remove all? How do I know I wont be removing something outside of the realm of the scripts like this?
Doing a little more research I'm not sure why it's seeing those since the default behavior these days should be to hide the system packages. ¯\_(ツ)_/¯
As far as uninstalling, yes, that's how you'd do that.
As far as uninstalling things that are used by other scripts, that's really down to what other python scripts you're using. The simplest way to find out would be to uninstall all packages and see what breaks. when you run the other python scripts you are using, then put those scripts into venvs of their own.
I run all python in venvs and install nothing on the host, so haven't needed to deal with that in a very long time.
I'm also using pyenv and direnv to manage this stuff, so somewhat different to your setup.
thanks for the reply.... one of the problems I see is that some python stuff is not obvious...... Some of the applications will use it, but unless you look under the hood, you dont see it.
I will try removing all of the requirements, and see what explodes outside of the obvious stuff I i know about.
Heh, I just reinstalled the server from scratch due to Python issues.... On the other hand, I have at this point a well documented install so recovery only takes a day end to end.
I am also shifting scripts and the like to a VM to keep bare metal as simple as possible.
Closing due to no activity. Reopen if this continues to be an issue.
Output:
ENV: