[BUG] Incompatibility with promptflow_vectordb

[lsegdaniel]

Describe the bug Flow created through Azure Machine Learning Studio Prompt Flow. Runs fine in the cloud. When downloaded and run locally, fails with the following:

Flow test failed with ResolveToolError: Tool load failed in 'query': (ToolLoadError) Failed to load package tool 'Vector Index Lookup': (ModuleNotFoundError) No module named 'promptflow.runtime'

How To Reproduce the bug

1. Create a flow that uses tool: promptflow_vectordb.tool.vector_index_lookup.VectorIndexLookup.search
2. Run it

Expected behavior The flow should run successfully, just as it runs successfully in the Azure Machine Learning Studio.

Running Information(please complete the following information):

• Promptflow Package Version using pf -v: 0.1.0b7.post1
• Operating System: macOS Ventura 13.5.2
• Python Version using python --version: 3.9.18
• promptflow_vectordb Version: 0.1.2

[jimmy-nhk]

You @lsegdaniel can have this package installed (hope that you can solve the problem) pip install promptflow-vectordb. I passed this problem but encounter another problem related to az-login

[lsegdaniel]

You @lsegdaniel can have this package installed (hope that you can solve the problem) pip install promptflow-vectordb. I passed this problem but encounter another problem related to az-login

Sorry @jimmy-nhk I don't follow. I already have promptflow_vectordb installed, and don't get a login issue (though I do need to use az login).

[dans-msft]

I am able to repro this entirely with Azure ML Studio as follows:

1. Create a chat-with-index flow by selecting Vector Index and then Create (under promptflow)
2. One the ingestion pipeline finishes, go to the Notebooks view and locate the directory that was created for your flow.
3. Open up a shell and active a python 3.10 environment. Install promptflow following the instructions at aka.ms/promptflow.
4. Execute the flow via pf flow test. The error will pop up after you type your query.

[dans-msft]

pip install promptflow_vectordb does not fix the problem, although it does change the error.

Without pip install promptflow_vectordb: Flow test failed with PackageToolNotFoundError: Package tool 'promptflow_vectordb.tool.vector_index_lookup.VectorIndexLookup.search' is not found in the current environment. All available package tools are: ['promptflow.tools.embedding.embedding'].

With pip install promptflow_vectordb: Flow test failed with ResolveToolError: Tool load failed in 'search_question_from_indexed_docs': (ToolLoadError) Failed to load package tool 'Vector Index Lookup': (ModuleNotFoundError) No module named 'promptflow.runtime'

[dans-msft]

I found the root cause, though the fix is fairly involved.

This tool uses Python code that is part of the Promptflow runtime installed when running in Azure ML but that is not part of Promptflow itself. Specifically, this code is used to access vector index connection information directly from the user's Azure ML workspace. The fix is to create a first-class Promptflow connection type for these connections which would then automatically read from the workspace when running in Azure ML and use Yaml files when running locally.

[jakeatmsft]

I found the root cause, though the fix is fairly involved.

This tool uses Python code that is part of the Promptflow runtime installed when running in Azure ML but that is not part of Promptflow itself. Specifically, this code is used to access vector index connection information directly from the user's Azure ML workspace. The fix is to create a first-class Promptflow connection type for these connections which would then automatically read from the workspace when running in Azure ML and use Yaml files when running locally.

Are there any references for this workaround, I'm not sure how to "Yaml files when running locally."

[dans-msft]

Update: we have an updated index lookup tool with a lot more features that will replace the tool discussed here and also fixes this issue. Still waiting on a precise ETA for the release but it will be soon.

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!

[lsegdaniel]

I don't think you are actually waiting for my input on this but, if you are, can you please clarify what additional input you need?

[juliapoloSC]

Are there any updates on this?

[EdwardPrentice]

I am hitting this in ML Studio:

Tool load failed in 'search_question_from_indexed_docs': (ToolLoadError) Failed to load package tool 'Vector Index Lookup': (FileNotFoundError) [Errno 2] No such file or directory: '/tmp/tmpi9orug93/MLIndex'

[Adarsh-Ramanathan]

@juliapoloSC, the replacement for Vector Index Lookup is already available: https://learn.microsoft.com/en-us/azure/ai-studio/how-to/prompt-flow-tools/index-lookup-tool. To get up and running in vs code, you'll need to install promptflow-vectordb[azure] and pymongo in your python environment.

[Adarsh-Ramanathan]

@EdwardPrentice, this seems like an unrelated issue - could you open a new issue for this, please?

[dans-msft]

This particular issue is solved by the new index lookup tool described above.