search

dgarnitz / vectorflow

VectorFlow is a high volume vector embedding pipeline that ingests raw data, transforms it into vectors and writes it to a vector DB of your choice.
https://www.getvectorflow.com/
Apache License 2.0
676 stars 49 forks source link

Debt #80 fix duplicated code #83

Closed Aleksandir closed 1 year ago

Aleksandir commented 1 year ago

Hi @dgarnitz

Addresses issue#80.

Please let me know if you have any feedback or suggested changes 😊

Aleksandir commented 1 year ago

Hey @dgarnitz Apologies for all the commits, transparently I'm rather new to contributing to OS projects and have had a hard time getting VS code settings in place to not affect existing code, leading to unintended merge conflicts/changes.

I have now addressed the recent code base changes and fixed unintended auto-refactor changes I was having.

If you have any feedback, it would be greatly appreciated 😊

dgarnitz commented 1 year ago

@Aleksandir This looks great! Thanks for completing this. Before it can merge tho, you need to test it using the Docker compose to ensure that the end to end flow still works. This can be done using the setup.sh script and the standard_upload_client.py in clients

Also please re-run all unit tests in API and worker to show they still pass.

Please post screen shots to prove everything works as anticipated.

Aleksandir commented 1 year ago

Closing PR due to various errors and missing internal modules. have confirmed all required external modules have been installed. i have worked for some time to resolve these however, unfortunately it seems its outside of my ability to resolve these errors.

Edit: Being rather new to open source contributions, i believe its a lack of knowledge more than anything.

dgarnitz commented 1 year ago

@Aleksandir can you post the errors you were getting please? I will try to take a look at what is going wrong.