search

nasa-jpl-memex / sce

Sparkler Crawl Environment - a packaged, dockerized version of http://github.com/USCDataScience/sparkler.git
http://irds.usc.edu/sparkler/
Apache License 2.0
4 stars 3 forks source link

Retreived 12 webpages in UI are already colored before voting for relevancy #36

Open ahmadika opened 6 years ago

ahmadika commented 6 years ago

When training a model using UI, if you search for new keywords, the new 12 webpages are already colored based on previous voting. This will confuse the voter if he/she has already voted for a webpages or not. The colors should be reset for each new search.

sce ui
sujen1412 commented 6 years ago

@ahmadika , the coloring of the UI for the new search term after one round of voting is done show the models performance. There is an accuracy score of the model, so when the user provides the same labels to the UI which the model "predicted" the accuracy goes high.

This is a way to indicate to the user, how a given page would be classified by the model that has been trained till now.

wmburke commented 6 years ago

Just to provide some additional context here, the accuracy score was previously ill-defined and so was removed from the interface. We can certainly (and probably should) look at ways to make this more clear as to what is going on - and potentially to re-introduce that score.

rduerr commented 5 years ago

Speaking as a user @wmburke yes the documentation should mention what is going on under the hood; but I also note that I did eventually figure out what it was doing....