search

IntelLabs / fastRAG

Efficient Retrieval Augmentation and Generation Framework
Apache License 2.0
1.32k stars 119 forks source link

Issues serializing a model #65

Open dnoliver opened 1 week ago

dnoliver commented 1 week ago

Hi,

I am having the following issue serializing the pipeline when I use the OpenVINOGenerator. The following code reproduce the issue:

from fastrag.generators.openvino import OpenVINOGenerator
from haystack import Pipeline

# Create OpenVINO generator
generator = OpenVINOGenerator(
    model="microsoft/Phi-3-mini-4k-instruct",
    compressed_model_dir="OpenVINO/Phi-3-mini-4k-instruct-int4-ov",
    device_openvino="GPU",
    task="text-generation",
    generation_kwargs={
        "max_new_tokens": 100,
    }
)

# Create a pipeline
pipeline = Pipeline()

# Add the generator to the pipeline
pipeline.add_component("generator", generator)

# Run the pipeline
results = pipeline.run({"generator": {"prompt": "Who is the best American actor?"}})

# Print the results
print(results)

# Serialize the pipeline into a file
with open("pipeline.yml", "w") as file:
    pipeline.dump(file)

The model runs fine, but the serialization part fails (pipeline.dump(file)). This is the error message:

(haystack) PS C:\Users\Nicolas Oliver\Downloads\haystack-101> python .\main.py
{'generator': {'replies': ['\n\n# Answer\nDetermining the "best" American actor is subjective and can vary based on personal preferences, acting styles, and the genres of movies or plays one enjoys. However, some actors are widely recognized for their exceptional talent and have received critical acclaim and numerous awards throughout their careers. Here are a few actors who are often celebrated for their acting prowess:\n\n1. **Meryl Streep** - Known for']}}

Traceback (most recent call last):
  File "C:\Users\Nicolas Oliver\Downloads\haystack-101\main.py", line 29, in <module>
    pipeline.dump(file)
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\haystack\core\pipeline\pipeline.py", line 213, in dump
    fp.write(marshaller.marshal(self.to_dict()))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\haystack\marshal\yaml.py", line 9, in marshal
    return yaml.dump(dict_)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\__init__.py", line 253, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\__init__.py", line 241, in dump_all
    dumper.represent(data)
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 27, in represent
    node = self.represent_data(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 52, in represent_data
    node = self.yaml_multi_representers[data_type](self, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 342, in represent_object
    return self.represent_mapping(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 52, in represent_data
    node = self.yaml_multi_representers[data_type](self, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Nicolas Oliver\.conda\envs\haystack\Lib\site-packages\yaml\representer.py", line 317, in represent_object
    reduce = data.__reduce_ex__(2)
             ^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot pickle 'Model' object

The intention of serializing the pipeline is to get it deployed with HayHooks. Any hints why this might be happening? is it an issue with this library or with the HayStack framework?

Thanks!

danielfleischer commented 3 days ago

We'll look into it.