Take over some load from the Wikidata servers

q-wiki / q-wiki-server

Q-Wiki is a Wikidata based game (Unity front-end with ASP.NET Core back-end) developed by HTW Berlin students in cooperation with Wikimedia Germany.

GNU General Public License v3.0

2 stars 0 forks source link

Take over some load from the Wikidata servers #62

Open rrrnld opened 5 years ago

rrrnld commented 5 years ago

We know that the Wikidata servers try to cache questions as aggressively as possible… we try to break this because we need shuffled responses that are hard to predict. It might make sense to take over some of that work to our own servers (which is basically shuffeling and limiting answers). Potential benefits:

Faster response times
Quicker query development due to simpler queries (even though these two parts don't really make up the biggest part in the complexity, it adds up)

Potential downsides:

Not all queries might fit the generic "shuffle the responses and take 4 out of them"-scheme

What do you think @maltegoetz?

rrrnld commented 5 years ago

An additional benefit: Queries become way easier to debug when you see all the responses! This could also benefit the promotional / debugging platform that we had in mind. You could say something like "I had a problem with question XY" and then mark the answer option that you think is wrong.

maltegoetz commented 5 years ago

In general this makes a lot of sense, but I think this is hard to "sell" during the final presentation. Also how would you store the data and how often should it be updated? SQL Database? Daily/weekly? How would you query the stored data?

rrrnld commented 5 years ago

Oh, I'd still run the actual queries against the SPARQL endpoints, so no caching on our side would be needed. It would just change the format of the query results a bit.

AlaaSarhan commented 4 years ago

I also thought initially that it means you'd query and cache results. You can probably improve a lot the performance and load on Wikidata servers with just simplifying the queries and do later transformations in game servers. When the game scales to a certain extent, caching/querying on separate servers might become a need. but indeed atm there's no real pressure on that I guess.

Love the idea in general!