The current version of package is unstable and exceptions occur

[espdev]

Hello,

I try to use the package with ONNX runtime, but the current version works very unstable and doesn't work in most cases. Also it seems fast-sentence-transformers package is outdated.

I try to run examples from Readme and I get errors.

OS: Ubuntu 20.04

Installing:

pip install classy-classification[onnx]

SpaCy embeddings. It does not work.

Error:

...
File ~/tmp/classy/lib/python3.11/site-packages/classy_classification/classifiers/classy_spacy.py:101, in ClassySpacyInternal.get_embeddings(self, docs)
     99 elif doc.has_extension("trf_data"):
    100     print(doc)
--> 101     embeddings.append(doc._.trf_data.model_output.pooler_output[0])
    102 else:
    103     warnings.warn(
    104         f"None of the words in the text `{str(doc)}` have vectors. Returning zeros.", stacklevel=1
    105     )

AttributeError: 'DocTransformerOutput' object has no attribute 'model_output'

Sentence-transfomer embeddings. It works.

Hugginface zero-shot classifiers. It does not work.

Error:

...
File ~/tmp/classy/lib/python3.11/site-packages/optimum/exporters/onnx/__main__.py:356, in main_export(model_name_or_path, output, task, opset, device, fp16, optimize, monolith, no_post_process, framework, atol, cache_dir, trust_remote_code, pad_token_id, subfolder, revision, force_download, local_files_only, use_auth_token, for_ort, do_validation, model_kwargs, custom_onnx_configs, fn_get_submodels, use_subprocess, _variant, library_name, legacy, **kwargs_shapes)
    354         autodetected_message = ""
    355     model_tasks = TasksManager.get_supported_tasks_for_model_type(model_type, exporter="onnx")
--> 356     raise ValueError(
    357         f"Asked to export a {model_type} model for the task {task}{autodetected_message}, but the Optimum ONNX exporter only supports the tasks {', '.join(model_tasks.keys())} for {model_type}. Please use a supported task. Please open an issue at https://github.com/huggingface/optimum/issues if you would like the task {task} to be supported in the ONNX export for {model_type}."
    358     )
    360 # TODO: Fix in Transformers so that SdpaAttention class can be exported to ONNX. `attn_implementation` is introduced in Transformers 4.36.
    361 if model_type in SDPA_ARCHS_ONNX_EXPORT_NOT_SUPPORTED and _transformers_version >= version.parse("4.35.99"):

ValueError: Asked to export a bart model for the task text-classification, but the Optimum ONNX exporter only supports the tasks feature-extraction, feature-extraction-with-past, text-generation, text-generation-with-past, text2text-generation, text2text-generation-with-past for bart. Please use a supported task. Please open an issue at https://github.com/huggingface/optimum/issues if you would like the task text-classification to be supported in the ONNX export for bart.

Standalone usage without spaCy

It works with the default model: torch/sentence_transformers/sentence-transformers_paraphrase-multilingual-MiniLM-L12-v2/quantized_true.onnx

The following code does not work if the model is not yet downloaded:

# overwrite [embedding model](https://www.sbert.net/docs/pretrained_models.html)
classifier.set_embedding_model(model="paraphrase-MiniLM-L3-v2")

Error:

...
File ~/tmp/classy/lib/python3.11/site-packages/fast_sentence_transformers/FastSentenceTransformer.py:203, in FastSentenceTransformer.model2onnx(self)
    201     print(f"Model found at: {self.export_model_name}")
    202 else:
--> 203     self.onnx(self.model_path, "default", self.export_model_name, quantize=self.quantize)
    204     print(f"Model exported at: {self.export_model_name}")

File ~/tmp/classy/lib/python3.11/site-packages/fast_sentence_transformers/txtai/pipeline/train/hfonnx.py:124, in HFOnnx.__call__(self, path, task, output, quantize, opset)
    121     if not ONNX_RUNTIME:
    122         raise ImportError('onnxruntime is not available - install "pipeline" extra to enable')
--> 124     output = self.quantization(output)
    126 if isinstance(output, BytesIO):
    127     # Reset stream and return bytes
    128     output.seek(0)

File ~/tmp/classy/lib/python3.11/site-packages/fast_sentence_transformers/txtai/pipeline/train/hfonnx.py:162, in HFOnnx.quantization(self, output)
    160     output = Path(output)
    161 # Quantize optimized model
--> 162 quantize_dynamic(output, output, optimize_model=False)
    164 # Read file back to bytes if temp file was created
    165 if temp:

TypeError: quantize_dynamic() got an unexpected keyword argument 'optimize_model'

The second run works without errors.

pip list:

Package                    Version
-------------------------- ------------
accelerate                 0.26.1
aiohttp                    3.9.1
aiosignal                  1.3.1
annotated-types            0.6.0
asttokens                  2.4.1
attrs                      23.2.0
blis                       0.7.11
catalogue                  2.0.10
certifi                    2023.11.17
charset-normalizer         3.3.2
classy-classification      0.6.7
click                      8.1.7
cloudpathlib               0.16.0
coloredlogs                15.0.1
confection                 0.1.4
curated-tokenizers         0.0.9
curated-transformers       0.1.1
cymem                      2.0.8
datasets                   2.16.1
decorator                  5.1.1
dill                       0.3.7
en-core-web-trf            3.7.3
evaluate                   0.4.1
executing                  2.0.1
fast-sentence-transformers 0.4.1
filelock                   3.13.1
flatbuffers                23.5.26
frozenlist                 1.4.1
fsspec                     2023.10.0
huggingface-hub            0.20.3
humanfriendly              10.0
idna                       3.6
ipython                    8.20.0
jedi                       0.19.1
Jinja2                     3.1.3
joblib                     1.3.2
langcodes                  3.3.0
MarkupSafe                 2.1.4
matplotlib-inline          0.1.6
mpmath                     1.3.0
multidict                  6.0.4
multiprocess               0.70.15
murmurhash                 1.0.10
networkx                   3.2.1
nltk                       3.8.1
numpy                      1.26.3
nvidia-cublas-cu12         12.1.3.1
nvidia-cuda-cupti-cu12     12.1.105
nvidia-cuda-nvrtc-cu12     12.1.105
nvidia-cuda-runtime-cu12   12.1.105
nvidia-cudnn-cu12          8.9.2.26
nvidia-cufft-cu12          11.0.2.54
nvidia-curand-cu12         10.3.2.106
nvidia-cusolver-cu12       11.4.5.107
nvidia-cusparse-cu12       12.1.0.106
nvidia-nccl-cu12           2.18.1
nvidia-nvjitlink-cu12      12.3.101
nvidia-nvtx-cu12           12.1.105
onnx                       1.15.0
onnxruntime                1.16.3
optimum                    1.16.2
packaging                  23.2
pandas                     1.5.3
parso                      0.8.3
pexpect                    4.9.0
pillow                     10.2.0
pip                        23.3.2
preshed                    3.0.9
prompt-toolkit             3.0.43
protobuf                   4.25.2
psutil                     5.9.8
ptyprocess                 0.7.0
pure-eval                  0.2.2
pyarrow                    15.0.0
pyarrow-hotfix             0.6
pydantic                   2.5.3
pydantic_core              2.14.6
Pygments                   2.17.2
python-dateutil            2.8.2
pytz                       2023.3.post1
PyYAML                     6.0.1
regex                      2023.12.25
requests                   2.31.0
responses                  0.18.0
safetensors                0.4.2
scikit-learn               1.4.0
scipy                      1.12.0
sentence-transformers      2.2.2
sentencepiece              0.1.99
setuptools                 69.0.3
six                        1.16.0
smart-open                 6.4.0
spacy                      3.7.2
spacy-alignments           0.9.1
spacy-curated-transformers 0.2.1
spacy-legacy               3.0.12
spacy-loggers              1.0.5
spacy-transformers         1.3.4
srsly                      2.4.8
stack-data                 0.6.3
sympy                      1.12
thinc                      8.2.2
threadpoolctl              3.2.0
tokenizers                 0.15.1
torch                      2.1.2
torchvision                0.16.2
tqdm                       4.66.1
traitlets                  5.14.1
transformers               4.36.2
triton                     2.1.0
typer                      0.9.0
typing_extensions          4.9.0
urllib3                    2.1.0
wasabi                     1.1.2
wcwidth                    0.2.13
weasel                     0.3.4
wheel                      0.42.0
xxhash                     3.4.1
yarl                       1.9.4

Will the package run more stable without ONNX? Maybe we need to specify the critical dependency versions more strictly? Right now it's working very erratically and it's not clear at what point things might break down.

[davidberenstein1957]

Hi @espdev , what do you propose for specifying versioning ? PRs are welcome :)

[espdev]

Hi @davidberenstein1957,

It seems the dependency versions currently specified in pyproject.toml are not fully compatible with the package code. I think we need to specify the version ranges of the major dependencies with which the package will definitely work without errors (I hope semver will work). spaCy, sentence-transformers, transformers, onnx/onnxruntime. Also it seems fast-sentence-transformers package is outdated for the current version of sentence-transformers package.