Resolve serialization issues

[karbasia]

Currently, the embedding retriever component cannot be serialized. This is causing issues with Hayhooks deployments which leads to the following error:

2024-06-06 09:52:15 INFO:     Deployed pipeline: ingest_pipeline
2024-06-06 09:52:15 Traceback (most recent call last):
2024-06-06 09:52:15   File "/opt/venv/bin/hayhooks", line 8, in <module>
2024-06-06 09:52:15     sys.exit(hayhooks())
2024-06-06 09:52:15              ^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
2024-06-06 09:52:15     return self.main(*args, **kwargs)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/click/core.py", line 1078, in main
2024-06-06 09:52:15     rv = self.invoke(ctx)
2024-06-06 09:52:15          ^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
2024-06-06 09:52:15     return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-06-06 09:52:15                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
2024-06-06 09:52:15     return ctx.invoke(self.callback, **ctx.params)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
2024-06-06 09:52:15     return __callback(*args, **kwargs)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/hayhooks/cli/run/__init__.py", line 20, in run
2024-06-06 09:52:15     uvicorn.run("hayhooks.server:app", host=host, port=port)
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/uvicorn/main.py", line 575, in run
2024-06-06 09:52:15     server.run()
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/uvicorn/server.py", line 65, in run
2024-06-06 09:52:15     return asyncio.run(self.serve(sockets=sockets))
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
2024-06-06 09:52:15     return runner.run(main)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
2024-06-06 09:52:15     return self._loop.run_until_complete(task)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/uvicorn/server.py", line 69, in serve
2024-06-06 09:52:15     await self._serve(sockets)
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/uvicorn/server.py", line 76, in _serve
2024-06-06 09:52:15     config.load()
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/uvicorn/config.py", line 433, in load
2024-06-06 09:52:15     self.loaded_app = import_from_string(self.app)
2024-06-06 09:52:15                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/uvicorn/importer.py", line 19, in import_from_string
2024-06-06 09:52:15     module = importlib.import_module(module_str)
2024-06-06 09:52:15              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
2024-06-06 09:52:15     return _bootstrap._gcd_import(name[level:], package, level)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
2024-06-06 09:52:15   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
2024-06-06 09:52:15   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
2024-06-06 09:52:15   File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
2024-06-06 09:52:15   File "<frozen importlib._bootstrap_external>", line 995, in exec_module
2024-06-06 09:52:15   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/hayhooks/server/__init__.py", line 1, in <module>
2024-06-06 09:52:15     from hayhooks.server.app import app
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/hayhooks/server/app.py", line 32, in <module>
2024-06-06 09:52:15     app = create_app()
2024-06-06 09:52:15           ^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/hayhooks/server/app.py", line 27, in create_app
2024-06-06 09:52:15     deployed_pipeline = deploy_pipeline_def(app, pipeline_defintion)
2024-06-06 09:52:15                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/hayhooks/server/utils/deploy_utils.py", line 16, in deploy_pipeline_def
2024-06-06 09:52:15     pipe = registry.add(pipeline_def.name, pipeline_def.source_code)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/hayhooks/server/pipelines/registry.py", line 17, in add
2024-06-06 09:52:15     self._pipelines[name] = Pipeline.loads(source)
2024-06-06 09:52:15                             ^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/haystack/core/pipeline/pipeline.py", line 234, in loads
2024-06-06 09:52:15     return cls.from_dict(marshaller.unmarshal(data), callbacks)
2024-06-06 09:52:15                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/haystack/marshal/yaml.py", line 13, in unmarshal
2024-06-06 09:52:15     return yaml.safe_load(data_)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/__init__.py", line 125, in safe_load
2024-06-06 09:52:15     return load(stream, SafeLoader)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/__init__.py", line 81, in load
2024-06-06 09:52:15     return loader.get_single_data()
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/constructor.py", line 51, in get_single_data
2024-06-06 09:52:15     return self.construct_document(node)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/constructor.py", line 60, in construct_document
2024-06-06 09:52:15     for dummy in generator:
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
2024-06-06 09:52:15     value = self.construct_mapping(node)
2024-06-06 09:52:15             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/constructor.py", line 218, in construct_mapping
2024-06-06 09:52:15     return super().construct_mapping(node, deep=deep)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/constructor.py", line 143, in construct_mapping
2024-06-06 09:52:15     value = self.construct_object(value_node, deep=deep)
2024-06-06 09:52:15             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/constructor.py", line 100, in construct_object
2024-06-06 09:52:15     data = constructor(self, node)
2024-06-06 09:52:15            ^^^^^^^^^^^^^^^^^^^^^^^
2024-06-06 09:52:15   File "/opt/venv/lib/python3.12/site-packages/yaml/constructor.py", line 427, in construct_undefined
2024-06-06 09:52:15     raise ConstructorError(None, None,
2024-06-06 09:52:15 yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object:milvus_haystack.document_store.MilvusDocumentStore'
2024-06-06 09:52:15   in "<unicode string>", line 44, column 23:
2024-06-06 09:52:15           document_store: !!python/object:milvus_haystack. ... 
2024-06-06 09:52:15     

[zc277584121]

@karbasia Great work, thanks a lot. But the lint seems failed, can you fix it , and can you add some pytest to coverage the to_dict() and from_dict() methods?

[karbasia]

@karbasia Great work, thanks a lot. But the lint seems failed, can you fix it , and can you add some pytest to coverage the to_dict() and from_dict() methods?

Thank you for the feedback @zc277584121! I've addressed your items and pushed updates.

[zc277584121]

@karbasia ,released, https://pypi.org/project/milvus-haystack/