search

WDscholia / scholia

Wikidata-based scholarly profiles
https://scholia.toolforge.org
Other
223 stars 81 forks source link

Run Scholia automatically for a set of test pages #990

Open Daniel-Mietchen opened 4 years ago

Daniel-Mietchen commented 4 years ago

Some useful starting points here would be

nunogit commented 4 years ago

@Daniel-Mietchen regarding this topic, I came across Selenium. I used it a couple of years ago, and I didn't even remember it until I came across it again.

Pro's: renders javascript, which would be great for us. Drivers for several browsers (although chrome is the one best supported). Con's: it's slower than most frameworks when crawling/testing for errors.

Do you know it? If so, what do you think?

nunogit commented 4 years ago

DataTables on pages are now being tested.

Code: https://github.com/nunogit/scholia-junit-selenium

CI / automated testing https://travis-ci.org/nunogit/scholia-junit-selenium

Next steps: Now that the basic infrastructure is set, the next steps are:

  1. extend to test other widgets 1.1 improve the way widgets / DataTables fail, to be easier to detect failures https://github.com/fnielsen/scholia/issues/1026
  2. Make a list (or dynamic list) of pages to test https://github.com/fnielsen/scholia/issues/1033 / https://www.wikidata.org/wiki/Wikidata:WikiProject_Scholia/Testing

In parallel test this against a proxy and/or with in-page optimization and assess if the failure rate decreases. Issues: https://github.com/fnielsen/scholia/issues/1024, https://github.com/fnielsen/scholia/issues/67 and https://github.com/fnielsen/scholia/issues/1012

Daniel-Mietchen commented 4 years ago

I just added some examples but think this flat file with just the Wikidata URLs is a bit hard to manage in the long run. What about using a Wikidata page as a source for the tests, e.g. something similar to but more machine friendly than https://www.wikidata.org/wiki/Wikidata:WikiProject_Scholia/Testing ?