Automation to generate these at the end (or after n steps) of embedding training?

[mshostack]

This is such a great tool, but a bit clunky in the current form.

Is there a way to install this as an a1111 webui extension so that it can be configured via the webui to automatically generate these graphs and stats at the end of embedding training or every n steps? This feels like a really essential bit of data to know if you're overfitting your embedding. Also, if you have something set to run much longer but halfway through this flags the strength of the vectors is already in the red zone, it could pause training and save you GPU cycles.

[Zyin055]

This script was made as an experiment to try and figure out what was happening inside an embedding to hopefully learn how to create better embeddings.

So far, the loss graph and vector graph doesn't seem to give any data that we can take action on. But the vector strength does say how overtrained it is. The problem is that how overtrained an embedding is is subjective. Sometimes you want it to be high, sometimes you want it to be low.

Adding this script as an extension to A1111 web ui wouldn't really accomplish much because of this. The data isn't concrete enough to take reliable action.

[rockerBOO]

I think it would be nice to record the average vector strength and magnitude to compare over time. But like you're saying, may not be that impactful vs doing it when you can analyze it in full.

[Woisek]

This script was made as an experiment to try and figure out what was happening inside an embedding to hopefully learn how to create better embeddings.

To create better embeddings, one has to know how to interpret those graphs I guess. The question is: Where can such information be gatherd? Mostly the question: How should such a graph look like? What is important? What is common, what is strange in such a graph. Any hints on where to get this information ...?

And thanks for this nice analytics tool.