Store metadata of project training

[juhoinkinen]

It could be valuable to store some info about training of a project, which could also be shown by the CLI commands and REST API /projects/{project_id} method.

Currently show-project outputs the following:

Project ID:        tfidf-fi
Project Name:      TF-IDF Finnish
Language:          fi
Vocabulary:        yso
Vocab language:    fi
Access:            public
Trained:           True
Modification time: 2023-04-21 10:33:16

Maybe some of the following data could be added:

• [x] training state: trained or not-trained
• [x] vocabulary state: loaded or not-loaded
• [x] timestamp of the training run
• [ ] version of Annif that has been used in training/learning the project
• [ ] number of documents used to train the project
• [ ] name(s) of the file(s) via which the documents was/were given when training the project
• [ ] used backend parameters
• [ ] backend name
• [ ] duration of the training run
• [ ] ~size of the model on disk~
• [ ] timestamp(s) of learning run(s)

Also some more details on the training data (what?) and something that now goes to the debug log (what?).

These data for could be stored e.g. in a metadata file data/projects/the_project/annif_metadata along with the model file(s).

Edit: converted bullet list to checkbox list.

[osma]

These are all good ideas. Also it could be useful to have a "failed" status for projects/backends where the initialization fails for some reason - but that means making the initialization more careful.

I think it makes sense to start small and implement status features one (or two) at a time in separate PRs. For example trained vs. not-trained would be a good start.

Marking the issue as Long term because there are so many ideas here, but that doesn't mean we could implement some of them very soon.

[osma]

Tagging with 0.48 to indicate that some work in this direction (but not everything) should be done in that release

[psmukhopadhyay]

Does this element "number of examples in the training data" mean the number of records with which the model is trained? It will be very helpful if available in the show-project output. Another thought - the 'eval' command may store major metrics like F1@5, NDCG etc in the annif_metadata file every time it runs, otherwise (not run once,) F1@5 or NDCG may display 'NA'.

[juhoinkinen]

Does this element "number of examples in the training data" mean the number of records with which the model is trained? It will be very helpful if available in the show-project output.

Yes, I actually just modified that item in the list to "number of documents used to train the project" (for clarity) and added "name(s) of the file(s) via which the documents was/were given when training the project" (I have needed that information often; however, the full path to the file(s) should not be stored for security reasons), and reordered the list slightly to.