[BUG] Unable to load model NV-EMBED-V2

[icon2341]

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. Follow commands from marqo quick setup on docker
2. run with gpu all
3. specify settings:

# print(reviews) settings = { "model": "hf-hub:nvidia/NV-Embed-v2", "modelProperties": { "name": "hf-hub:nvidia/NV-Embed-v2", "dimensions": 4096, "type": "hf", }, }

4. run create index with settings

Expected behavior Model loads and begins embedding results

Screenshots raise MarqoWebError(message=response_msg, code=code, error_type=error_type, marqo.errors.MarqoWebError: MarqoWebError: MarqoWebError Error message: {'message': "Problem vectorising query. Reason: Unable to load model=hf-hub:nvidia/NV-Embed-v2 on device=cuda with normalization=True. If you are trying to load a custom model, please check that model_properties={'name': 'hf-hub:nvidia/NV-Embed-v2', 'dimensions': 4096, 'type': 'hf'} is correct and Marqo has access to the weights file.", 'code': 'bad_request', 'type': 'invalid_request', 'link': 'https://docs.marqo.ai/2.12/Guides/Models-Reference/list_of_models/', 'telemetry': {'timesMs': {'add_documents.create_vectors': 0.32188600016525015, 'POST /indexes/my-first-index/documents': 15.058888000112347}}} status_code: 400, type: invalid_request, code: bad_request, link:

Desktop (please complete the following information):

• OS: Windows Docker WSL Cuda
• Browser Chrome
• Version N/A

Additional context Add any other context about the problem here.

[wanliAlex]

Hi, thanks for the detailed information. I am looking into this.

[wanliAlex]

Hey, this model requires custom code thus it is not supported in Marqo. We will add the compatibility of models in our documentation. Alternatively, you can try other text models in our model registry.