search

ibm-granite-community / pm

Granite Community Project Management
0 stars 0 forks source link

Make Granite Models available on Watsonx via API without sign up #37

Open AnthonyJAnnunziata opened 3 months ago

AnthonyJAnnunziata commented 3 months ago

A public endpoint with appropriate rate/consumption limits ill be published and available for any developer to use for Granite models starting with code 8B and 20B.

AnthonyJAnnunziata commented 3 months ago

Ritika has lead to identify the appropriate tech lead in Remus Lazar's team to work with on this. BU team will do the work to make the endpoint available.

deanwampler commented 3 months ago

"Informal" requirements for scoping the effort (to be clarified).

  1. Granite Code 8B and 20B (with different context lengths?).
  2. API access from recipes, etc. without a user account preferred.
  3. Sessions rate limited.
  4. Complete Replicate, watsonx.ai, Together AI, and possibly GitHub hosting integrations. For each:
    1. End to "hardening" as required: monitoring, resiliency, availability, etc.
    2. Utilization statistics for project OKR purposes.
    3. On demand, automated model updates.
  5. Streamline the Ollama installation for a local execution option, as documented in Gabe Goodhart's blog post.
  6. Recipe implementations, starting with text to SQL.
  7. Automated test suites for recipes, different models, and different deployments.