quelques fonctionalités à ajouter à alvisnlp

[mandiayba]

Ce serait bien si on peut avoir à terme ces fonctionalités qui peuvent etre utiles. Ils permettront au moins de faciliter la recherche d'erreur et l'intégration des plans alvis dans l'api wrapper

• alvisnlp -dryrun hello.plan : vérifier le syntaxe d'un plan et s'assurer que toutes les ressources indiquées en lecture seule sont disponibles
• alvisnlp -display params: afficher les alias definies dans le plan et sous plans pour une utilisation en ligne de commande
• alvisnlp -show-dag hello.plan: afficher l'arbre des plans
• alvisnlp -show-ressources hello.plan afficher les ressources indiquées dans chaque plan, ce serait bien de pouvoir identifier les ressources en lecteur seule et les ecriture/lecture (ressources en passage entre des modules).

[rbossy]

• alvisnlp -dryrun hello.plan : vérifier le syntaxe d'un plan et s'assurer que toutes les ressources indiquées en lecture seule sont disponibles

alvisnlp -noProcess.

Si tu préfères, on peut renommer l'option -dryRun.

• alvisnlp -display params: afficher les alias definies dans le plan et sous plans pour une utilisation en ligne de commande

Bonne idée. Ce ne devrait pas être très compliqué.

Cela nécessite de se mettre d'accord sur le format.

Pour info il y a déjà une option -writePlan qui fait plus ou moins cette analyse.

• alvisnlp -show-dag hello.plan: afficher l'arbre des plans

Bonne idée aussi.

Pareil, se mettre d'accord sur le format.

En attendant tu peux dériver cette information à partir du log. Par exemple:

[2021-12-15 17:23:24.834][alvisnlp] loading plan from opennlp.plan
[2021-12-15 17:23:24.851][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/share/classification-read-data.plan
[2021-12-15 17:23:24.875][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/share/classification-read-data.plan
[2021-12-15 17:23:24.876][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/share/classification-preprocess.plan
[2021-12-15 17:23:24.880][alvisnlp] loading plan from segmentation.plan
[2021-12-15 17:23:24.899][alvisnlp] loading plan from number-and-dates.plan
[2021-12-15 17:23:24.908][alvisnlp] finished loading number-and-dates.plan
[2021-12-15 17:23:24.934][alvisnlp] finished loading segmentation.plan
[2021-12-15 17:23:24.940][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/share/classification-preprocess.plan
[2021-12-15 17:23:24.940][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-all-folds.plan
[2021-12-15 17:23:24.941][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.952][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.952][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.955][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.956][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.959][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.959][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.963][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.966][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.969][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.969][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.972][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.972][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.975][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.975][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.978][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.979][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.981][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.981][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.984][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.984][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.987][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.987][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.990][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan
[2021-12-15 17:23:24.990][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-all-folds.plan
[2021-12-15 17:23:24.991][alvisnlp] loading plan from /home/rbossy/code/alvisnlp/alvisnlp-test/share/classification-evaluate.plan
[2021-12-15 17:23:24.995][alvisnlp] finished loading /home/rbossy/code/alvisnlp/alvisnlp-test/share/classification-evaluate.plan
[2021-12-15 17:23:24.995][alvisnlp] finished loading opennlp.plan

• alvisnlp -show-ressources hello.plan afficher les ressources indiquées dans chaque plan, ce serait bien de pouvoir identifier les ressources en lecteur seule et les ecriture/lecture (ressources en passage entre des modules).

Bonne idée, encore.

Pareil: format.

Pareil, tu peux voir cette info sur le log:

[2021-12-15 17:23:25.020][alvisnlp] External resources
[2021-12-15 17:23:25.021][opennlp.read-data.csv] R: source = /home/rbossy/code/alvisnlp/alvisnlp-test/share/classification.csv (/home/rbossy/code/alvisnlp/alvisnlp-test/share/classification-re
ad-data.plan)
[2021-12-15 17:23:25.022][opennlp.preprocessing.seg.preprocessing.latin] R: dictFile = latin_multi-word-expressions.txt (segmentation.plan)
[2021-12-15 17:23:25.028][opennlp.opennlp-all-folds.fold-0.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.028][opennlp.opennlp-all-folds.fold-0.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.029][opennlp.opennlp-all-folds.fold-1.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.029][opennlp.opennlp-all-folds.fold-1.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.030][opennlp.opennlp-all-folds.fold-2.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.030][opennlp.opennlp-all-folds.fold-2.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.031][opennlp.opennlp-all-folds.fold-3.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.031][opennlp.opennlp-all-folds.fold-3.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.032][opennlp.opennlp-all-folds.fold-4.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.033][opennlp.opennlp-all-folds.fold-4.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.033][opennlp.opennlp-all-folds.fold-5.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.033][opennlp.opennlp-all-folds.fold-5.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.034][opennlp.opennlp-all-folds.fold-6.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.034][opennlp.opennlp-all-folds.fold-6.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.035][opennlp.opennlp-all-folds.fold-7.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.035][opennlp.opennlp-all-folds.fold-7.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.035][opennlp.opennlp-all-folds.fold-8.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.036][opennlp.opennlp-all-folds.fold-8.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.036][opennlp.opennlp-all-folds.fold-9.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.036][opennlp.opennlp-all-folds.fold-9.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.037][opennlp.opennlp-all-folds.fold-10.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.037][opennlp.opennlp-all-folds.fold-10.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.037][opennlp.opennlp-all-folds.fold-11.train] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.037][opennlp.opennlp-all-folds.fold-11.predict] R: model = model.bin (/home/rbossy/code/alvisnlp/alvisnlp-test/opennlp/opennlp-fold.plan)
[2021-12-15 17:23:25.038][opennlp.eval.eval] R: outFile = evaluate.txt (/home/rbossy/code/alvisnlp/alvisnlp-test/share/classification-evaluate.plan)

D'une manière générale, il faut que je trouve moyen de fusionner cette issue avec #105 .