Training / web inconsistency

tinybike / argus

[DEPRECATED] Automated Q/A prototype

http://pasky.or.cz:5500

9 stars 6 forks source link

Training / web inconsistency #3

Open pasky opened 8 years ago

pasky commented 8 years ago

There seems to be some inconsistency between web interface and the training process:

Will the New England Patriots defeat the Seattle Seahawks in Super Bowl XLIX? in tests/ftrain.tsv is marked as YES 0.521451771259 with "as it happened" as the most relevant article.
In the web interface right now, with the same model, it's 0.08 for NO and ABC comes first by relevancy, with "as it happened" story as second

Silvicek commented 8 years ago

The sentence in ftrain.tsv is the first sentence that elasticsearch finds and not the most relevant one (this is an artifact from when we were using only one sentence for answering). As for the inconsistency between web and ftrain, did you confirm rewriting the f*.tsv files? (i dont thing the train.py -full rewrites the files).

pasky commented 8 years ago

Oh, thanks for that hint re sentences. Well, train.py asks whether the files should be rewritten and I said "yes", so they should be up to date, right?