search

BaseModelAI / cleora

Cleora AI is a general-purpose model for efficient, scalable learning of stable and inductive entity embeddings for heterogeneous relational data.
https://cleora.ai
Other
485 stars 51 forks source link

Online learning #30

Closed sademakn closed 2 years ago

sademakn commented 3 years ago

Is there any way to update the model on the fly with new data?

piobab commented 3 years ago

Hi @sademakn !

The embeddings are stable, which means, you can re-run on (old + new) data, and the results will be aligned with the old embeddings, for any downstream purposes.

Currently on-the-fly updates are not implemented, but what would be required is as follows:

sademakn commented 3 years ago

Thanks for your well-organized answer, If someday the on-the-fly updates will be available, that would help a lot with large-scale and non-stop data streams. In my case, I use a cronjob to train my model on whole data(old + new) and each time it takes hours to build a new model and during the training also I'm receiving a bunch of new data! Is there any hope to achieve on-the-fly updates anytime soon?

piobab commented 3 years ago

For now there is no plan but I will come back to you next week because I have to check the pipeline. We'll see what we can do.

piobab commented 3 years ago

Hi @sademakn !

I checked and unfortunately there is no chance to provide the feature within 1 or 2 months.

We will be updating if something change.

piobab commented 2 years ago

@sademakn Hi! We encourage you to contribute. Unfortunately, it will not be possible to implement such a solution on our side. If it happens that a large group of users will expect such a feature, we will try to allocate time for implementation.