DARPA-ASKEM / terarium

https://app.terarium.ai
Apache License 2.0
12 stars 2 forks source link

Generate embeddings as part of model / document creation. #4021

Closed kbirk closed 2 days ago

kbirk commented 6 days ago

This PR includes the following:

Anytime a model card task completes successfully, if the document was public and non-temporary, it generates embeddings and indexes them. Therefore when running pdf extraction, it will automatically create the semantic search vectors for the document if it is publically searchable.

When creating / updating a public + non-temporary model, the amr is used to generate search embeddings. If model alignment is run with a document, the documents gollmCard, if present, will be used to generate new embeddings.