Compatibility of DoctrineVectorStore with doctrine migration

[SebastienTouze]

Hi o/

As the VectorType type is added to Doctrine on DoctrineVectorStore constructor, it is not available in DoctrineMigration which is not aware of this type. So using bin/console make:migration or equivalent bin/console doctrine:migration:diff result in error: Unknown column type "vector" requested.

One option seems to move the type declaration to doctrine dbal config file, with consequence to make the VectorType always available to doctrine. This is mostly adding some config info on the readme and maybe remove the integration in DoctrineVectorStore class.

If this proposition is OK for you I can open a PR

[SebastienTouze]

Example of config that seems to work

doctrine:
    dbal:
        url: '%env(resolve:DATABASE_URL)%'
        types:
            vector: LLPhant\Embeddings\VectorStores\Doctrine\VectorType
        mapping_types:
            enum: string
            vector: vector

But I get an error on length not being define though it is defined in default DoctrineEmbeddingEntityBase I extend. Operation 'VECTORs must have a length.' is not supported by platform.

I am using LLPhant v0.2.12

[chr-hertel]

Currently using vector: array as mapping type currently, but also still having issues with migrations. For me it is regenerating the same migrations all over again with this:

    public function up(Schema $schema): void
    {
        $this->addSql('ALTER TABLE snippet ALTER embedding TYPE vector(1536)');
        $this->addSql('COMMENT ON COLUMN snippet.embedding IS NULL');
    }

[arnaudgoulpeau]

Hi !

I used this bundle (mainly to get access to DQL functions) : https://packagist.org/packages/partitech/doctrine-pgvector If you follow their docs about configuring the new type, you shouldn't have problems. It works great on with doctrine/dbal 3.8 and doctrine/orm 2.19

Hope this helps