search

dashjoin / platform

Dashjoin Is an Open Source & Cloud Native Low Code Development and Integration Platform that helps teams deliver applications faster 🚀 Uses a data-driven approach inspired by Linked Data to make use of your existing assets
https://dashjoin.com
GNU Affero General Public License v3.0
90 stars 8 forks source link

Support Postgres vector data types #391

Open uw4 opened 1 month ago

uw4 commented 1 month ago

When storing embeddings in pg vector we get these warnings:

2024-07-07 18:20:47,572 INFO [org.das.ser.Services] (executor-thread-161) Starting metadata collection for tenant=null 2024-07-07 18:20:47,933 SEVERE [org.das.ser.Metadata] (executor-thread-161) Type unknown: BIGSERIAL 2024-07-07 18:20:47,934 SEVERE [org.das.ser.Metadata] (executor-thread-161) Type unknown: VECTOR 2024-07-07 18:20:47,934 SEVERE [org.das.ser.Metadata] (executor-thread-161) Type unknown: BIGSERIAL 2024-07-07 18:20:47,935 SEVERE [org.das.ser.Metadata] (executor-thread-161) Type unknown: VECTOR 2024-07-07 18:20:47,936 SEVERE [org.das.ser.Metadata] (executor-thread-161) Type unknown: UUID 2024-07-07 18:20:47,936 SEVERE [org.das.ser.Metadata] (executor-thread-161) Type unknown: VECTOR

Tested with dockerhub container pgvector/pgvector:pg16

uw4 commented 1 month ago

Not critical as the table can still be browsed read-only:

image
uw4 commented 1 month ago

Typical stacktrace during browsing:

2024-07-09 18:30:50,839 INFO [org.das.ser.ExMapper] (executor-thread-167) REST exception: org.postgresql.util.PSQLException: ERROR: operator does not exist: bigint = character varying Hint: No operator matches the given name and argument types. You might need to add explicit type casts. Position: 49 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190) at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:134) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122) at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.dashjoin.service.SQLDatabase$Aspect.invoke(SQLDatabase.java:193) at jdk.proxy4/jdk.proxy4.$Proxy99.executeQuery(Unknown Source) at org.dashjoin.service.SQLDatabase.readInternal(SQLDatabase.java:835) at org.dashjoin.service.SQLDatabase.read(SQLDatabase.java:752) at org.dashjoin.service.Data.read(Data.java:702) at org.dashjoin.service.Data.read(Data.java:635) at org.dashjoin.service.Data.list(Data.java:931) at org.dashjoin.service.Data_ClientProxy.list(Unknown Source)