Pinecone Go SDK ·

This is the official Go SDK for Pinecone.

Documentation

To see the latest documentation for main, visit https://pkg.go.dev/github.com/pinecone-io/go-pinecone@main/pinecone.

To see the latest versioned-release's documentation, visit https://pkg.go.dev/github.com/pinecone-io/go-pinecone/pinecone.

Features

go-pinecone contains

• gRPC bindings for Data Plane operations
• REST bindings for Control Plane operations

See the Pinecone API Docs for more information.

Upgrading the SDK

To upgrade the SDK to the latest version, run:

go get -u github.com/pinecone-io/go-pinecone/pinecone@latest

Prerequisites

go-pinecone requires a Go version with modules support.

Installation

To install the Pinecone Go SDK, run the following in your terminal:

go get github.com/pinecone-io/go-pinecone/pinecone

For more information on setting up a Go project, see the Go documentation.

Usage

Initializing the client

Authenticating via an API key

When initializing the client with a Pinecone API key, you must construct a NewClientParams object and pass it to the NewClient function.

It's recommended that you set your Pinecone API key as an environment variable ("PINECONE_API_KEY") and access it that way. Alternatively, you can pass it in your code directly.

package main

import (
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)

    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }
}

Authenticating via custom headers

If you choose to authenticate via custom headers (e.g. for OAuth), you must construct a NewClientBaseParams object and pass it to NewClientBase.

Note: you must include the "X-Project-Id" header with your Pinecone project ID when authenticating via custom headers.

package main

import (
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
)

func main() {
    clientParams := pinecone.NewClientBaseParams{
        Headers: map[string]string{
            "Authorization": "Bearer " + "<your OAuth token>"
            "X-Project-Id":  "<Your Pinecone project ID>"
        },
    }

    pc, err := pinecone.NewClientBase(clientParams)

    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }
}

Indexes

Create indexes

Create a serverless index

The following example creates a serverless index in the us-east-1 region of AWS. For more information on serverless and regional availability, see Understanding indexes.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    indexName := "my-serverless-index"

    idx, err := pc.CreateServerlessIndex(ctx, &pinecone.CreateServerlessIndexRequest{
        Name:      indexName,
        Dimension: 3,
        Metric:    pinecone.Cosine,
        Cloud:     pinecone.Aws,
        Region:    "us-east-1",
    })

    if err != nil {
        log.Fatalf("Failed to create serverless index: %v", err)
    } else {
        fmt.Printf("Successfully created serverless index: %s", idx.Name)
    }
}

Create a pod-based index

The following example creates a pod-based index with a metadata configuration. If no metadata configuration is provided, all metadata fields are automatically indexed.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    indexName := "my-pod-index"

    podIndexMetadata := &pinecone.PodSpecMetadataConfig{
        Indexed: &[]string{"title", "description"},
    }

    idx, err := pc.CreatePodIndex(ctx, &pinecone.CreatePodIndexRequest{
        Name:           indexName,
        Dimension:      3,
        Metric:         pinecone.Cosine,
        Environment:    "us-west1-gcp",
        PodType:        "s1",
        MetadataConfig: podIndexMetadata,
    })

    if err != nil {
        log.Fatalf("Failed to create pod index: %v", err)
    } else {
        fmt.Printf("Successfully created pod index: %s", idx.Name)
    }

}

List indexes

The following example lists all indexes in your Pinecone project.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idxs, err := pc.ListIndexes(ctx)
    if err != nil {
        log.Fatalf("Failed to list indexes: %v", err)
    } else {
        fmt.Println("Your project has the following indexes:")
        for _, idx := range idxs {
            fmt.Printf("- \"%s\"\n", idx.Name)
        }
    }
}

Describe an index

The following example describes an index by name.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    indexName := "the-name-of-my-index"

    idx, err := pc.DescribeIndex(ctx, indexName)
    if err != nil {
        log.Fatalf("Failed to describe index: %s", err)
    } else {
        fmt.Printf("%+v", *idx)
    }
}

Delete an index

The following example deletes an index by name. Note: only indexes not protected by deletion protection may be deleted.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    indexName := "the-name-of-my-index"

    err = pc.DeleteIndex(ctx, indexName)
    if err != nil {
        log.Fatalf("Error: %v", err)
    } else {
        fmt.Printf("Index \"%s\" deleted successfully", indexName)
    }
}

Configure an index

There are multiple ways to configure Pinecone indexes. You are able to configure Deletion Protection for both pod-based and Serverless indexes. Additionally, you can configure the size of your pods and the number of replicas for pod-based indexes. Examples for each of these configurations are provided below.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    // To scale the size of your pods-based index from "x2" to "x4":
    _, err := pc.ConfigureIndex(ctx, "my-pod-index", pinecone.ConfigureIndexParams{PodType: "p1.x4"})
    if err != nil {
        fmt.Printf("Failed to configure index: %v\n", err)
    }

    // To scale the number of replicas to 4:
    _, err := pc.ConfigureIndex(ctx, "my-pod-index", pinecone.ConfigureIndexParams{Replicas: 4})
    if err != nil {
        fmt.Printf("Failed to configure index: %v\n", err)
    }

    // To scale both the size of your pods and the number of replicas:
    _, err := pc.ConfigureIndex(ctx, "my-pod-index", pinecone.ConfigureIndexParams{PodType: "p1.x4", Replicas: 4})
    if err != nil {
        fmt.Printf("Failed to configure index: %v\n", err)
    }

    // To enable deletion protection:
    _, err := pc.ConfigureIndex(ctx, "my-index", pinecone.ConfigureIndexParams{DeletionProtection: "enabled"})
    if err != nil {
        fmt.Printf("Failed to configure index: %v\n", err)
    }
}

Describe index statistics

The following examlpe describes the statistics of an index by name.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    indexName := "the-name-of-my-index"

    idx, err := pc.DescribeIndex(ctx, indexName)
    if err != nil {
        log.Fatalf("Failed to describe index \"%s\". Error:%s", idx.Name, err)
    } else {
        desc := fmt.Sprintf("Description: \n  Name: %s\n  Dimension: %d\n  Host: %s\n  Metric: %s\n"+
            "  DeletionProtection"+
            ": %s\n"+
            "  Spec: %+v"+
            "\n  Status: %+v\n",
            idx.Name, idx.Dimension, idx.Host, idx.Metric, idx.DeletionProtection, idx.Spec, idx.Status)
        fmt.Println(desc)
    }
}

Index Operations

Pinecone indexes support working with vector data using operations such as upsert, query, fetch, and delete.

Targeting an index

To perform data operations on an index, you target it using the Index method on a Client object. You will need your index's Host value, which you can retrieve via DescribeIndex or ListIndexes.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "pinecone-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host: %v: %v", idx.Host, err)
    }
}

Upsert vectors

The following example upserts vectors (both dense and sparse) and metadata to example-index.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "google.golang.org/protobuf/types/known/structpb"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "example-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host: %v: %v", idx.Host, err)
    }

    metadataMap := map[string]interface{}{
        "genre": "classical",
    }
    metadata, err := structpb.NewStruct(metadataMap)

    sparseValues := pinecone.SparseValues{
        Indices: []uint32{0, 1, 2, 3, 4, 5, 6, 7},
        Values:  []float32{1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0},
    }

    vectors := []*pinecone.Vector{
        {
            Id:           "A",
            Values:       []float32{0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1},
            Metadata:     metadata,
            SparseValues: &sparseValues,
        },
        {
            Id:           "B",
            Values:       []float32{0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2},
            Metadata:     metadata,
            SparseValues: &sparseValues,
        },
        {
            Id:           "C",
            Values:       []float32{0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3},
            Metadata:     metadata,
            SparseValues: &sparseValues,
        },
        {
            Id:           "D",
            Values:       []float32{0.4, 0.4, 0.4, 0.4, 0.4, 0.4, 0.4, 0.4},
            Metadata:     metadata,
            SparseValues: &sparseValues,
        },
    }

    count, err := idxConnection.UpsertVectors(ctx, vectors)
    if err != nil {
        log.Fatalf("Failed to upsert vectors: %v", err)
    } else {
        fmt.Printf("Successfully upserted %d vector(s)", count)
    }
}

Import vectors from object storage

You can now import vectors en masse from object storage. Import is a long-running, asynchronous operation that imports large numbers of records into a Pinecone serverless index.

In order to import vectors from object storage, they must be stored in Parquet files and adhere to the necessary file format. Your object storage must also adhere to the necessary directory structure.

The following example imports vectors from an Amazon S3 bucket into a Pinecone serverless index:

    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)

    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    }

    indexName := "sample-index"

    idx, err := pc.CreateServerlessIndex(ctx, &pinecone.CreateServerlessIndexRequest{
        Name:      indexName,
        Dimension: 3,
        Metric:    pinecone.Cosine,
        Cloud:     pinecone.Aws,
        Region:    "us-east-1",
    })

    if err != nil {
        log.Fatalf("Failed to create serverless index: %v", err)
    }

    idx, err = pc.DescribeIndex(ctx, "pinecone-index")

    if err != nil {
        log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host: %v: %v", idx.Host, err)
    }

    storageURI := "s3://my-bucket/my-directory/"

    errorMode := "abort" // Will abort if error encountered; other option: "continue"

    importRes, err := idxConnection.StartImport(ctx, storageURI, nil, (*pinecone.ImportErrorMode)(&errorMode))

    if err != nil {
        log.Fatalf("Failed to start import: %v", err)
    }

    fmt.Printf("import started with ID: %s", importRes.Id)

You can start, cancel, and check the status of all or one import operation(s).

Query an index

Query by vector values

The following example queries the index example-index with vector values and metadata filtering. Note: you can also query by sparse values; see sparse-dense documentation for examples.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "google.golang.org/protobuf/types/known/structpb"
    "log"
    "os"
)

func prettifyStruct(obj interface{}) string {
    bytes, _ := json.MarshalIndent(obj, "", "  ")
    return string(bytes)
}

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "example-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host, Namespace: "example-namespace"})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host %v: %v", idx.Host, err)
    }

    queryVector := []float32{0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3}

    metadataMap := map[string]interface{}{
        "genre": map[string]interface{}{
            "$eq": "documentary",
        },
        "year": 2019,
    }

    metadataFilter, err := structpb.NewStruct(metadataMap)
    if err != nil {
        log.Fatalf("Failed to create metadataFilter: %v", err)
    }

    res, err := idxConnection.QueryByVectorValues(ctx, &pinecone.QueryByVectorValuesRequest{
        Vector:         queryVector,
        TopK:           3,
        MetadataFilter: metadataFilter,
        IncludeValues:  true,
    })
    if err != nil {
        log.Fatalf("Error encountered when querying by vector: %v", err)
    } else {
        fmt.Printf(prettifyStruct(res))
    }
}

// Returns:
// {
//   "matches": [
//     {
//       "vector": {
//         "id": "B",
//         "values": [
//           0.2,
//           0.2,
//           0.2,
//           0.2,
//           0.2,
//           0.2,
//           0.2,
//           0.2
//         ]
//       },
//       "score": 1
//     },
//     {
//       "vector": {
//         "id": "C",
//         "values": [
//           0.3,
//           0.3,
//           0.3,
//           0.3,
//           0.3,
//           0.3,
//           0.3,
//           0.3
//         ]
//       },
//       "score": 1
//     },
//     {
//       "vector": {
//         "id": "A",
//         "values": [
//           0.1,
//           0.1,
//           0.1,
//           0.1,
//           0.1,
//           0.1,
//           0.1,
//           0.1
//         ]
//       },
//       "score": 1
//     }
//   ],
//   "usage": {
//     "read_units": 6
//   }
// }

Query by vector id

The following example queries the index example-index with a vector id value.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func prettifyStruct(obj interface{}) string {
    bytes, _ := json.MarshalIndent(obj, "", "  ")
    return string(bytes)
}

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "example-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host, Namespace: "example-namespace"})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host %v: %v", idx.Host, err)
    }

    vectorId := "vector-id"
    res, err := idxConnection.QueryByVectorId(ctx, &pinecone.QueryByVectorIdRequest{
        VectorId:      vectorId,
        TopK:          3,
        IncludeValues: true,
    })
    if err != nil {
        log.Fatalf("Error encountered when querying by vector ID `%v`: %v", vectorId, err)
    } else {
        fmt.Printf(prettifyStruct(res.Matches))
    }
}

Delete vectors

Delete vectors by ID

The following example deletes a vector by its ID value from example-index and example-namespace. You can pass a slice of vector IDs to DeleteVectorsById.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "example-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%s\". Error:%s", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host, Namespace: "example-namespace"})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host: %v. Error: %v", idx.Host, err)
    }

    vectorId := "your-vector-id"
    err = idxConnection.DeleteVectorsById(ctx, []string{vectorId})

    if err != nil {
        log.Fatalf("Failed to delete vector with ID: %s. Error: %s\n", vectorId, err)
    }
}

Delete vectors by filter

The following example deletes vectors from example-index using a metadata filter.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "google.golang.org/protobuf/types/known/structpb"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "example-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%s\". Error:%s", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host: %v. Error: %v", idx.Host, err)
    }

    filter, err := structpb.NewStruct(map[string]interface{}{
        "genre": "classical",
    })
    if err != nil {
        log.Fatalf("Failed to create metadata filter. Error: %v", err)
    }

    err = idxConnection.DeleteVectorsByFilter(ctx, filter)

    if err != nil {
        log.Fatalf("Failed to delete vector(s) with filter: %+v. Error: %s\n", filter, err)
    }
}

Delete all vectors in a namespace

The following example deletes all vectors from example-index and example-namespace.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "example-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%s\". Error:%s", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host, Namespace: "example-namespace"})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host: %v. Error: %v", idx.Host, err)
    }

    // deletes all vectors in "example-namespace"
    err = idxConnection.DeleteAllVectorsInNamespace(ctx)
    if err != nil {
        log.Fatalf("Failed to delete vectors in namespace: \"%s\". Error: %s", idxConnection.Namespace, err)
    }
}

Fetch vectors

The following example fetches vectors by ID from example-index and example-namespace.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func prettifyStruct(obj interface{}) string {
    bytes, _ := json.MarshalIndent(obj, "", "  ")
    return string(bytes)
}

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "example-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host, Namespace: "example-namespace"})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host %v: %v", idx.Host, err)
    }

    res, err := idxConnection.FetchVectors(ctx, []string{"id-1", "id-2"})
    if err != nil {
        log.Fatalf("Failed to fetch vectors: %v", err)
    } else {
        fmt.Printf(prettifyStruct(res))
    }
}

// Response:
// {
//   "vectors": {
//     "id-1": {
//       "id": "id-1",
//       "values": [
//         -0.0089730695,
//         -0.020010853,
//         -0.0042787646,
//         ...
//       ]
//     },
//     "id-2": {
//       "id": "id-2",
//       "values": [
//         -0.005380766,
//         0.00215196,
//         -0.014833462,
//         ...
//       ]
//     }
//   },
//   "usage": {
//     "read_units": 1
//   }
// }

Update vectors

The following example updates vectors by ID in example-index and example-namespace.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "pinecone-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host, Namespace: "ns1"})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host %v: %v", idx.Host, err)
    }

    id := "id-3"

    err = idxConnection.UpdateVector(ctx, &pinecone.UpdateVectorRequest{
        Id:     id,
        Values: []float32{4.0, 2.0},
    })
    if err != nil {
        log.Fatalf("Failed to update vector with ID %v: %v", id, err)
    }
}

List vectors

The ListVectors method can be used to list vector ids matching a particular id prefix. With clever assignment of vector ids, you can model hierarchical relationships across embeddings within the same document.

The following example lists all vector ids in example-index and example-namespace, with the prefix doc1.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func prettifyStruct(obj interface{}) string {
    bytes, _ := json.MarshalIndent(obj, "", "  ")
    return string(bytes)
}

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    idx, err := pc.DescribeIndex(ctx, "example-index")
    if err != nil {
        log.Fatalf("Failed to describe index \"%v\": %v", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host, Namespace: "example-namespace"})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host %v: %v", idx.Host, err)
    }

    limit := uint32(3)
    prefix := "doc1"

    res, err := idxConnection.ListVectors(ctx, &pinecone.ListVectorsRequest{
        Limit:  &limit,
        Prefix: &prefix,
    })
    if len(res.VectorIds) == 0 {
        fmt.Println("No vectors found")
    } else {
        fmt.Printf(prettifyStruct(res))
    }
}

// Response:
// {
//   "vector_ids": [
//     "doc1#chunk1",
//     "doc1#chunk2",
//     "doc1#chunk3"
//   ],
//   "usage": {
//     "read_units": 1
//   },
//   "next_pagination_token": "eyJza2lwX3Bhc3QiOiIwMDBkMTc4OC0zMDAxLTQwZmMtYjZjNC0wOWI2N2I5N2JjNDUiLCJwcmVmaXgiOm51bGx9"
// }

Collections

A collection is a static copy of an index. Collections are only available for pod-based indexes.

Create a collection

The following example creates a collection from a source index.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    collection, err := pc.CreateCollection(ctx, &pinecone.CreateCollectionRequest{
        Name:   "my-collection",
        Source: "my-source-index",
    })
    if err != nil {
        log.Fatalf("Failed to create collection: %v", err)
    } else {
        fmt.Printf("Successfully created collection \"%s\".", collection.Name)
    }
}

List collections

The following example lists all collections in your Pinecone project.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    collections, err := pc.ListCollections(ctx)
    if err != nil {
        log.Fatalf("Failed to list collections: %v", err)
    } else {
        if len(collections) == 0 {
            fmt.Printf("No collections found in project")
        } else {
            fmt.Println("Collections in project:")
            for _, collection := range collections {
                fmt.Printf("- %s\n", collection.Name)
            }
        }
    }
}

Describe a collection

The following example describes a collection by name.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    collection, err := pc.DescribeCollection(ctx, "my-collection")
    if err != nil {
        log.Fatalf("Error describing collection: %v", err)
    } else {
        fmt.Printf("Collection: %+v\n", *collection)
    }
}

Delete a collection

The following example deletes a collection by name.

package main

import (
    "context"
    "fmt"
    "github.com/pinecone-io/go-pinecone/pinecone"
    "log"
    "os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } else {
        fmt.Println("Successfully created a new Client object!")
    }

    collectionName := "my-collection"

    err = pc.DeleteCollection(ctx, collectionName)
    if err != nil {
        log.Fatalf("Failed to create collection: %s\n", err)
    } else {
        log.Printf("Successfully deleted collection \"%s\"\n", collectionName)
    }
}

Inference

The Client object has an Inference namespace which allows interacting with Pinecone's Inference API. The Inference API is a service that gives you access to embedding models hosted on Pinecone's infrastructure. Read more at Understanding Pinecone Inference.

Notes:

Models currently supported:

• Embedding: multilingual-e5-large
• Reranking: bge-reranker-v2-m3

Create Embeddings

Send text to Pinecone's inference API to generate embeddings for documents and queries.

    ctx := context.Background()

    pc, err := pinecone.NewClient(pinecone.NewClientParams{
        ApiKey: "YOUR_API_KEY",
    })
    if err !=  nil {
        log.Fatalf("Failed to create Client: %v", err)
    }

    embeddingModel := "multilingual-e5-large"
    documents := []string{
        "Turkey is a classic meat to eat at American Thanksgiving."
        "Many people enjoy the beautiful mosques in Turkey."
    }
    docParameters := pinecone.EmbedParameters{
        InputType: "passage",
        Truncate: "END",
    }

    docEmbeddingsResponse, err := pc.Inference.Embed(ctx, &pinecone.EmbedRequest{
        Model: embeddingModel,
        TextInputs: documents,
        Parameters: docParameters,
    })
    if err != nil {
        log.Fatalf("Failed to embed documents: %v", err)
    }
    fmt.Printf("docs embedding response: %+v", docEmbeddingsResponse)

    // << Upsert documents into Pinecone >>

    userQuery := []string{
        "How should I prepare my turkey?"
    }
    queryParameters := pinecone.EmbedParameters{
        InputType: "query",
        Truncate: "END",
    }
    queryEmbeddingsResponse, err := pc.Inference.Embed(ctx, &pinecone.EmbedRequest{
        Model: embeddingModel,
        TextInputs: userQuery,
        Parameters: queryParameters
    })
    if err != nil {
        log.Fatalf("Failed to embed query: %v", err)
    }
    fmt.Printf("query embedding response: %+v", queryEmbeddingsResponse)

    // << Send query to Pinecone to retrieve similar documents >>

Rerank documents

Rerank documents in descending relevance-order against a query.

Note: The score represents the absolute measure of relevance of a given query and passage pair. Normalized between [0, 1], the score represents how closely relevant a specific item and query are, with scores closer to 1 indicating higher relevance.

    ctx := context.Background()

    pc, err := pinecone.NewClient(pinecone.NewClientParams{
        ApiKey: "YOUR-API-KEY"
    })

    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    }

    rerankModel := "bge-reranker-v2-m3"
    query := "What are some good Turkey dishes for Thanksgiving?"

    documents := []pinecone.Document{
      {"title": "Turkey Sandwiches", "body": "Turkey is a classic meat to eat at American Thanksgiving."},
      {"title": "Lemon Turkey", "body": "A lemon brined Turkey with apple sausage stuffing is a classic Thanksgiving main course."},
      {"title": "Thanksgiving", "body": "My favorite Thanksgiving dish is pumpkin pie"},
      {"title": "Protein Sources", "body": "Turkey is a great source of protein."},
    }

    // Optional arguments
    topN := 3
    returnDocuments := false
    rankFields := []string{"body"}
    modelParams := map[string]string{
      "truncate": "END",
    }

    rerankRequest := pinecone.RerankRequest{
      Model:           rerankModel,
      Query:           query,
      Documents:       documents,
      TopN:            &topN,
      ReturnDocuments: &returnDocuments,
      RankFields:      &rankFields,
      Parameters:      &modelParams,
    }

    rerankResponse, err := pc.Inference.Rerank(ctx, &rerankRequest)

    if err != nil {
      log.Fatalf("Failed to rerank documents: %v", err)
    }

    fmt.Printf("rerank response: %+v", rerankResponse)

Support

To get help using go-pinecone you can file an issue on GitHub, visit the community forum, or reach out to support@pinecone.io.