search

IBM / ibm-generative-ai-node-sdk

Library that provides convenient access to the Generative AI API from Node.js applications
MIT License
35 stars 6 forks source link

Production WatsonX endpoints #37

Closed garrettrowe closed 11 months ago

garrettrowe commented 1 year ago

Hi, the library seems to be pinned to the Tech Preview of WatsonX. The production WatsonX paths are quite different. Is there a near term plan to allow for the production paths to be used?

Tomas2D commented 1 year ago

Hello Garret,

Maybe you are looking for https://github.com/IBM/watsonxdata-node-sdk.

garrettrowe commented 1 year ago

That is a repo for the WatsonX.data - the lakehouse product.

WatsonX.ai (generativeAI / LLM's) have different production paths than the tech preview, which are hardcoded in this library.

Eg: Tech preview vs production endpoints & paths: Tech Preview: https://workbench-api.res.ibm.com/v1/generate Prod: https://us-south.ml.cloud.ibm.com/ml/v1-beta/generation/text?version=2023-05-29

Tomas2D commented 1 year ago

I see; we had an internal discussion about it, and we would like to collaborate on compatibility with WatsonX API.

Let's keep this issue open, and I will post an update here.

chasemcdo commented 1 year ago

Hey Team! Our team is also interested in a package for use with production WatsonX. Is there any update on this?

Tomas2D commented 1 year ago

Hey, we will discuss the SDKs on Thursday; if you need to work with WatsonX urgently, you can use their endpoint for the text generation via Rest API.

chasemcdo commented 1 year ago

Thanks Tomas! Yah, we’re presently using the Rest API directly, but we’re interested in a cleaner approach for potential integration with LangChain. Excited to see what comes of this :)

Tomas2D commented 11 months ago

There is no plan to align NodeJS SDK, only the Python SDK. I will reopen this thread if anything changes.

faileon commented 9 months ago

Thanks Tomas! Yah, we’re presently using the Rest API directly, but we’re interested in a cleaner approach for potential integration with LangChain. Excited to see what comes of this :)

Im sorry to resurect this thread but I am becoming desperate. Where can I find the REST API documentation so I can use the generate, generate_text and generate_text_stream functions for chat models in my nodeJS app, similar to the mentioned python package? I've been looking for hours and I can't find anything. The only thing close to it is the Watson ML API https://cloud.ibm.com/apidocs/machine-learning

chasemcdo commented 9 months ago

Thanks Tomas! Yah, we’re presently using the Rest API directly, but we’re interested in a cleaner approach for potential integration with LangChain. Excited to see what comes of this :)

Im sorry to resurect this thread but I am becoming desperate. Where can I find the REST API documentation so I can use the generate, generate_text and generate_text_stream functions for chat models in my nodeJS app, similar to the mentioned python package? I've been looking for hours and I can't find anything. The only thing close to it is the Watson ML API https://cloud.ibm.com/apidocs/machine-learning

Hey faileon, I haven't found much in terms of comprehensive documentation for interacting with the prompt lab model API directly. Though the docs you linked are helpful for auth and generation IAM tokens.

The best form of documentation I have found for using the prompt lab model API is the pop up code block within the prompt lab

image

Using these calls you can interact with the api to generate predictions using WatsonX.ai models. If this is what you are after I have recently added an integration to LangChainJS for just this which will likely be deployed by the end of this week.

Edit: This method only helps with text_generation and likely misses streaming and some other stuff, but depending on how desperate you are this might be sufficient

chasemcdo commented 9 months ago

@faileon for the more complex actions it may be possible to extract the URLs from the Python SDK.

Looks like the endpoints are visible as text or text_stream etc

faileon commented 9 months ago

@faileon for the more complex actions it may be possible to extract the URLs from the Python SDK.

Looks like the endpoints are visible as text or text_stream etc

Thanks, yup seems that reverse engineering python SDK is the only "sane" way. I also found an open api specs somewhere here - https://github.com/watson-developer-cloud/assistant-toolkit/blob/master/integrations/extensions/starter-kits/language-model-watsonx/watsonx-openapi.json

This is super bad developer experience so far...

ericsaleh commented 9 months ago

@faileon we are hoping to publish the API specs soon. The frustration and gap this creates in the meantime is totally understandable: it's a top priority for the watsonx team to close this