thefcraft / nsfw-prompt-detection-sd

NSFW Prompt Detection for Stable Diffusion
22 stars 1 forks source link

Can't load pickle file #1

Closed himmetozcan closed 1 year ago

himmetozcan commented 1 year ago

It can load this one:

with open('nsfw_classifier_tokenizer.pickle', 'rb') as f:
    tokenizer = pickle.load(f)

However, It gives an error for the one below :

with open('nsfw_classifier.pickle', 'rb') as f:
    model = pickle.load(f)

ValueError: Expected a model.weights.h5 or model.weights.npz file.

thefcraft commented 1 year ago

bro but its working fine on my device i just git clone it and it working fine see Python version = 3.9.13 cuda

import json
import tensorflow as tf
import numpy as np
import random
import pickle
with open('nsfw_classifier_tokenizer.pickle', 'rb') as f:
    tokenizer = pickle.load(f)
with open('nsfw_classifier.pickle', 'rb') as f:
    model = pickle.load(f)

here is output in ipynb file

Keras model archive loading:
File Name                                             Modified             Size
config.json                                    2023-04-24 20:17:46         3194
metadata.json                                  2023-04-24 20:17:46           64
variables.h5                                   2023-04-24 20:17:46      8107504
Keras weights file (<HDF5 file "variables.h5" (mode r)>) loading:
...layers\concatenate
......vars
...layers\dense
......vars
.........0
.........1
...layers\embedding
......vars
.........0
...layers\input_layer
......vars
...layers\input_layer_1
......vars
...layers\lstm
......vars
...layers\lstm\cell
......vars
.........0
.........1
.........2
...metrics\mean
......vars
.........0
.........1
...metrics\mean_metric_wrapper
......vars
.........0
.........1
...optimizer
......vars
.........0
.........1
.........10
.........11
.........12
.........2
.........3
.........4
.........5
.........6
.........7
.........8
.........9
...vars

!python -m pip list

Package                       Version
----------------------------- --------------------
absl-py                       1.3.0
accelerate                    0.16.0
addict                        2.4.0
aenum                         3.1.11
aiofiles                      23.1.0
aiohttp                       3.8.3
aiosignal                     1.3.1
aitextgen                     0.6.0
alabaster                     0.7.12
altair                        4.2.2
anaconda-client               1.11.0
anaconda-navigator            2.3.1
anaconda-project              0.11.1
antlr4-python3-runtime        4.9.3
anyio                         3.5.0
appdirs                       1.4.4
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arrow                         1.2.2
astroid                       2.11.7
astropy                       5.1
astunparse                    1.6.3
async-generator               1.10
async-timeout                 4.0.2
asyncer                       0.0.2
atomicwrites                  1.4.0
attrs                         21.4.0
audioread                     3.0.0
Automat                       20.2.0
autopep8                      1.6.0
Babel                         2.9.1
backcall                      0.2.0
backoff                       2.2.1
backports.functools-lru-cache 1.6.4
backports.tempfile            1.0
backports.weakref             1.0.post1
basicsr                       1.4.2
bcrypt                        3.2.0
beartype                      0.12.0
beautifulsoup4                4.11.1
bidict                        0.22.1
binaryornot                   0.4.4
bitarray                      2.5.1
bkcharts                      0.2
black                         22.6.0
bleach                        4.1.0
blendmodes                    2022
blis                          0.7.9
bokeh                         2.4.3
boltons                       23.0.0
boto3                         1.24.28
botocore                      1.27.28
Bottleneck                    1.3.5
brotlipy                      0.7.0
cachetools                    5.2.0
catalogue                     2.0.8
certifi                       2022.9.14
cffi                          1.15.1
chardet                       3.0.4
charset-normalizer            2.0.4
ChatterBot                    1.0.8
click                         8.1.3
clip                          0.2.0
cloudpickle                   2.0.0
clyent                        1.2.2
cohere                        4.1.3
colorama                      0.4.6
colorcet                      3.0.0
coloredlogs                   15.0.1
comtypes                      1.1.10
conda                         23.1.0
conda-build                   3.22.0
conda-content-trust           0.1.3
conda-pack                    0.6.0
conda-package-handling        1.9.0
conda-repo-cli                1.0.20
conda-token                   0.4.0
conda-verify                  3.4.2
confection                    0.0.4
constantly                    15.1.0
cookiecutter                  1.7.3
COVID19Py                     0.3.0
crypto                        1.4.1
cryptography                  37.0.1
cssselect                     1.1.0
cycler                        0.11.0
cymem                         2.0.7
Cython                        0.29.32
cytoolz                       0.11.0
daal4py                       2021.6.0
dask                          2022.7.0
datasets                      2.9.0
datashader                    0.14.1
datashape                     0.5.4
debugpy                       1.5.1
decorator                     5.1.1
defusedxml                    0.7.1
deprecation                   2.1.0
diff-match-patch              20200713
diffusers                     0.14.0
dill                          0.3.6
distlib                       0.3.6
distributed                   2022.7.0
docker-pycreds                0.4.0
docutils                      0.18.1
efficientnet                  1.0.0
einops                        0.6.0
einops-exts                   0.0.4
ema-pytorch                   0.1.4
en-core-web-md                3.3.0
en-core-web-sm                2.2.0
entrypoints                   0.4
et-xmlfile                    1.1.0
evaluate                      0.4.0
exceptiongroup                1.1.1
facexlib                      0.2.5
fastapi                       0.87.0
fastjsonschema                2.16.2
ffmpy                         0.3.0
filelock                      3.6.0
filetype                      1.2.0
filterpy                      1.4.5
fire                          0.5.0
flake8                        4.0.1
Flask                         1.1.2
Flask-Cors                    3.0.10
Flask-SocketIO                5.3.3
flatbuffers                   22.12.6
fonts                         0.0.3
fonttools                     4.25.0
frozenlist                    1.3.3
fsspec                        2022.7.1
ftfy                          6.1.1
future                        0.18.2
gast                          0.4.0
gensim                        4.1.2
gfpgan                        1.3.8
gitdb                         4.0.10
GitPython                     3.1.31
glide-text2im                 0.0.0
glob2                         0.7
google-auth                   2.15.0
google-auth-oauthlib          0.4.6
google-pasta                  0.2.0
googletrans                   3.0.0
gradio                        3.20.1
greenlet                      1.1.1
grpcio                        1.51.1
gTTS                          2.3.1
h11                           0.9.0
h2                            3.2.0
h5py                          3.7.0
HeapDict                      1.0.1
holoviews                     1.15.0
hpack                         3.0.0
hstspreload                   2023.1.1
httpcore                      0.9.1
httpx                         0.13.3
huggingface-hub               0.12.0
humanfriendly                 10.0
hvplot                        0.8.0
hyperframe                    5.2.0
hyperlink                     21.0.0
idna                          2.10
image-classifiers             1.0.0
imagecodecs                   2021.8.26
ImageHash                     4.3.1
imageio                       2.19.3
imagesize                     1.4.1
importlib-metadata            4.11.3
incremental                   21.3.0
inflect                       5.3.0
inflection                    0.5.1
iniconfig                     1.1.1
intake                        0.6.5
intervaltree                  3.1.0
ipykernel                     6.15.2
ipython                       7.31.1
ipython-genutils              0.2.0
ipywidgets                    7.6.5
isort                         5.9.3
itemadapter                   0.3.0
itemloaders                   1.0.4
itsdangerous                  2.0.1
jdcal                         1.4.1
jedi                          0.18.1
jellyfish                     0.9.0
Jinja2                        2.11.3
jinja2-time                   0.2.0
jmespath                      0.10.0
joblib                        1.1.0
json5                         0.9.6
jsonlines                     3.1.0
jsonmerge                     1.9.0
jsonpatch                     1.32
jsonpointer                   2.3
jsonschema                    4.16.0
jupyter                       1.0.0
jupyter_client                7.3.4
jupyter-console               6.4.3
jupyter_core                  4.11.1
jupyter-server                1.18.1
jupyterlab                    3.4.4
jupyterlab-pygments           0.1.2
jupyterlab-server             2.10.3
jupyterlab-widgets            1.0.0
keras                         2.11.0
Keras-Applications            1.0.8
Keras-Preprocessing           1.1.2
keyring                       23.4.0
kiwisolver                    1.4.2
kornia                        0.6.9
langcodes                     3.3.0
language-tool-python          2.7.1
lark                          1.1.5
lazy-object-proxy             1.6.0
libarchive-c                  2.9
libclang                      14.0.6
librosa                       0.8.1
lightning-utilities           0.6.0.post0
linkify-it-py                 2.0.0
llvmlite                      0.39.1
lmdb                          1.4.0
locket                        1.0.0
loguru                        0.6.0
lxml                          4.9.1
lz4                           3.1.3
Markdown                      3.3.4
markdown-it-py                2.2.0
MarkupSafe                    2.0.1
mathparse                     0.1.2
matplotlib                    3.5.1
matplotlib-inline             0.1.6
mccabe                        0.6.1
mdit-py-plugins               0.3.3
mdurl                         0.1.2
mediapipe                     0.9.1.0
menuinst                      1.4.19
mistune                       0.8.4
mkl-fft                       1.3.1
mkl-random                    1.2.2
mkl-service                   2.4.0
mock                          4.0.3
mpmath                        1.2.1
msgpack                       1.0.3
multidict                     6.0.4
multipledispatch              0.6.0
multiprocess                  0.70.14
munkres                       1.1.4
murmurhash                    1.0.9
mutagen                       1.46.0
mypy-extensions               0.4.3
Naked                         0.1.32
navigator-updater             0.3.0
nbclassic                     0.3.5
nbclient                      0.5.13
nbconvert                     6.4.4
nbformat                      5.5.0
nest-asyncio                  1.5.5
networkx                      2.8.4
neuralintents                 0.0.4
nltk                          3.7
nodelist-inflator             0.2.0
nomic                         1.1.4
nose                          1.3.7
nostril                       0.1.1
notebook                      6.4.12
numba                         0.56.4
numexpr                       2.8.3
numpy                         1.22.4
numpydoc                      1.4.0
oauthlib                      3.2.2
olefile                       0.46
omegaconf                     2.3.0
onnxruntime                   1.13.1
open-clip-torch               2.16.0
opencv-contrib-python         4.7.0.68
opencv-python                 4.7.0.68
opencv-python-headless        4.6.0.66
openpyxl                      3.0.10
opt-einsum                    3.3.0
orjson                        3.8.7
outcome                       1.2.0
packaging                     21.3
pandas                        1.4.4
pandocfilters                 1.5.0
panel                         0.13.1
param                         1.12.0
paramiko                      2.8.1
parsel                        1.6.0
parso                         0.8.3
partd                         1.2.0
pathlib                       1.0.1
pathspec                      0.9.0
pathtools                     0.1.2
pathy                         0.10.1
patsy                         0.5.2
peft                          0.2.0
pep8                          1.7.1
pexpect                       4.8.0
pickleshare                   0.7.5
piexif                        1.1.3
Pillow                        9.4.0
pip                           22.2.2
pkginfo                       1.8.2
platformdirs                  2.5.2
plotly                        5.9.0
pluggy                        1.0.0
pooch                         1.6.0
poyo                          0.5.0
preshed                       3.0.8
prometheus-client             0.14.1
prompt-toolkit                3.0.20
Protego                       0.1.16
protobuf                      3.19.6
psutil                        5.9.0
ptyprocess                    0.7.0
py                            1.11.0
pyarrow                       11.0.0
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycodestyle                   2.8.0
pycosat                       0.6.3
pycparser                     2.21
pycricbuzz                    2.4
pycryptodome                  3.17
pyct                          0.4.8
pycurl                        7.45.1
pydantic                      1.10.4
PyDispatcher                  2.0.5
pydocstyle                    6.1.1
pydub                         0.25.1
pyerfa                        2.0.0
pyflakes                      2.4.0
pygame                        2.1.2
Pygments                      2.14.0
PyHamcrest                    2.0.2
PyJWT                         2.4.0
pylint                        2.14.5
pyls-spyder                   0.4.0
PyMatting                     1.1.8
PyNaCl                        1.5.0
pynndescent                   0.5.8
pynvml                        11.5.0
pyodbc                        4.0.34
pyOpenSSL                     22.0.0
pyparsing                     3.0.9
pypng                         0.20220715.0
PyQt5                         5.15.6
PyQt5-Qt5                     5.15.2
PyQt5-sip                     12.11.1
pyreadline3                   3.4.1
pyrsistent                    0.18.0
PySocks                       1.7.1
pytest                        7.1.2
python-dateutil               2.8.2
python-engineio               4.4.0
python-lsp-black              1.0.0
python-lsp-jsonrpc            1.0.0
python-lsp-server             1.3.3
python-multipart              0.0.5
python-slugify                5.0.2
python-snappy                 0.6.0
python-socketio               5.8.0
pytorch-lightning             1.9.1
pytorch-warmup                0.1.1
pytz                          2022.1
pyviz-comms                   2.0.2
PyWavelets                    1.3.0
pywin32                       302
pywin32-ctypes                0.2.0
pywinpty                      2.0.2
PyYAML                        6.0
pyzmq                         23.2.0
QDarkStyle                    3.0.2
qrcode                        7.4.2
qstylizer                     0.1.10
QtAwesome                     1.0.3
qtconsole                     5.2.2
QtPy                          2.2.0
queuelib                      1.5.0
realesrgan                    0.3.0
regex                         2022.7.9
rembg                         2.0.30
requests                      2.28.1
requests-file                 1.5.1
requests-oauthlib             1.3.1
resampy                       0.4.2
resize-right                  0.0.2
responses                     0.18.0
rfc3986                       1.5.0
rich                          13.3.3
rope                          0.22.0
rsa                           4.9
Rtree                         0.9.7
ruamel.yaml                   0.17.21
ruamel.yaml.clib              0.2.6
ruamel-yaml-conda             0.15.100
s3transfer                    0.6.0
safetensors                   0.3.0
scikit-image                  0.19.3
scikit-learn                  1.0.2
scikit-learn-intelex          2021.20221004.171935
scipy                         1.7.3
Scrapy                        2.6.2
seaborn                       0.11.2
segmentation-models           1.0.1
selenium                      4.8.2
Send2Trash                    1.8.0
sentencepiece                 0.1.97
sentry-sdk                    1.18.0
service-identity              18.1.0
setproctitle                  1.3.2
setuptools                    63.4.1
shellescape                   3.8.1
shellingham                   1.5.0.post1
sip                           4.19.13
six                           1.16.0
smart-open                    5.2.1
smmap                         5.0.0
sniffio                       1.2.0
snowballstemmer               2.2.0
sortedcollections             2.1.0
sortedcontainers              2.4.0
sounddevice                   0.4.3
SoundFile                     0.10.3.post1
soupsieve                     2.3.1
spacy                         3.5.0
spacy-legacy                  3.0.12
spacy-loggers                 1.0.4
Sphinx                        5.0.2
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
spyder                        5.2.2
spyder-kernels                2.2.1
SQLAlchemy                    1.4.39
srsly                         2.4.5
starlette                     0.21.0
statsmodels                   0.13.2
sympy                         1.10.1
tables                        3.6.1
tabulate                      0.8.10
tb-nightly                    2.13.0a20230306
TBB                           0.2
tblib                         1.7.0
tenacity                      8.0.1
tensorboard                   2.11.0
tensorboard-data-server       0.7.0
tensorboard-plugin-wit        1.8.1
tensorflow                    2.11.0
tensorflow-estimator          2.11.0
tensorflow-gpu                2.10.1
tensorflow-hub                0.12.0
tensorflow-intel              2.11.0
tensorflow-io-gcs-filesystem  0.28.0
termcolor                     2.1.1
terminado                     0.13.1
testpath                      0.6.0
text-unidecode                1.3
textdistance                  4.2.1
tflearn                       0.5.0
Theano                        1.0.5
thinc                         8.1.7
threadpoolctl                 2.2.0
three-merge                   0.1.1
tifffile                      2021.7.2
timm                          0.6.12
tinycss                       0.4
tldextract                    3.2.0
tokenizers                    0.13.2
toml                          0.10.2
tomli                         2.0.1
tomlkit                       0.11.1
toolz                         0.11.2
torch                         2.0.0
torchaudio                    0.10.1+cu113
torchdiffeq                   0.2.3
torchfile                     0.1.0
torchmetrics                  0.11.1
torchsde                      0.2.5
torchvision                   0.11.2+cu113
tornado                       6.1
tqdm                          4.62.3
traitlets                     5.1.1
trampoline                    0.1.2
transformers                  4.26.1
trio                          0.22.0
trio-websocket                0.10.2
Twisted                       22.2.0
twisted-iocpsupport           1.0.2
typer                         0.7.0
typing_extensions             4.3.0
uc-micro-py                   1.0.1
ujson                         5.4.0
umap-learn                    0.5.2
Unidecode                     1.3.2
urllib3                       1.26.15
uvicorn                       0.20.0
verboselogs                   1.7
virtualenv                    20.21.0
visdom                        0.1.8.9
w3lib                         1.21.0
wandb                         0.14.0
wasabi                        1.1.1
watchdog                      2.1.9
wcwidth                       0.2.5
webencodings                  0.5.1
webrtcvad                     2.0.10
websocket-client              0.58.0
websockets                    10.4
Werkzeug                      2.0.3
wheel                         0.37.1
widgetsnbextension            3.5.2
win-inet-pton                 1.1.0
win-unicode-console           0.5
win32-setctime                1.1.0
wincertstore                  0.2
wonderwords                   2.2.0
wrapt                         1.14.1
wsproto                       1.2.0
xarray                        0.20.1
xlrd                          2.0.1
XlsxWriter                    3.0.3
xlwings                       0.27.15
xxhash                        3.2.0
yapf                          0.31.0
yarl                          1.8.2
zict                          2.1.0
zipp                          3.8.0
zope.interface                5.4.0
himmetozcan commented 1 year ago

I have python 3.8.10 Anyways, I have loaded h5 file like this, then it worked. Thanks.

from tensorflow.keras.models import load_model

model = load_model('nsfw_classifier.h5')