snowflakedb / snowflake-connector-net

Snowflake Connector for .NET
Apache License 2.0
172 stars 130 forks source link

SNOW-1479655: VECTOR datatype is not supported #975

Closed myshon closed 2 weeks ago

myshon commented 2 weeks ago
  1. What version of .NET driver are you using?

3.1.0

  1. What operating system and processor architecture are you using?

MAC OS / M2.

  1. What version of .NET framework are you using?

    net 8.0

  2. What did you do?

I created table

create or replace TABLE RAG.CLUSTERS (
    OBJECT_ID VARCHAR(36) NOT NULL,
    TEXT VARCHAR(16777216) NOT NULL,
    EMBEDDINGS VECTOR(FLOAT, 1024) NOT NULL,
);

I retrieve data from the table using follow code

public class ClusterDto
{
    public required string ObjectId { get; set; }
    public required string Text { get; set; }
    public required string Embeddings { get; set; }
}

public interface ISemanticService
{
    Task<IReadOnlyCollection<ClusterDto>> GetClustersAsync();
}

public class SemanticSnowflakeService : ISemanticService
{
    private readonly SnowflakeDbConnectionProvider _connectionProvider;

    public SemanticSnowflakeService(SnowflakeDbConnectionProvider connectionProvider)
    {
        _connectionProvider = connectionProvider;
    }

    public async Task<IReadOnlyCollection<ClusterDto>> GetClustersAsync()
    {
        await using var conn = await _connectionProvider.GetConnection();
        await conn.OpenAsync();
        await using var cmd = conn.CreateCommand();
        cmd.CommandText = @$"SELECT OBJECT_ID, TEXT, EMBEDDINGS FROM ACME_DB.RAG.CLUSTERS limit 10;";

        // Error: Snowflake Internal Error: Unknow column type: vector
        await using var reader = await cmd.ExecuteReaderAsync();
        var result = new List<ClusterDto>();
        while (await reader.ReadAsync())
        {
            result.Add(new ClusterDto()
            {
                ObjectId = reader.GetString(0),
                Text = reader.GetString(1),
                Embeddings = reader.GetString(2)
            });
        }

        await conn.CloseAsync();
        return result;
    }
}

Reader throws error with message ❌ // Error: Snowflake Internal Error: Unknow column type: vector

  1. What did you expect to see?

I need to retrieve vector as string/array/json or whatever.

❓ Is there any workaround?

sfc-gh-dszmolka commented 2 weeks ago

hello - for the moment, unfortunately the error message is correct: https://docs.snowflake.com/en/sql-reference/data-types-vector#data-types

Note The VECTOR data type is only supported in SQL, the Python connector and the Snowpark Python library. No other languages are supported.

we'll announce in the driver changelogs once it becomes supported. Thank you for bearing with us !