search

gabrieldelaparra / SPARQLforHumans

(Q54871) (Q3428776) for (Q5) - SPARQL Queries for Humans is my M.Sc. C.S. Thesis project.
Apache License 2.0
1 stars 2 forks source link

How to know if an Index has Rank or not? #27

Closed gabrieldelaparra closed 6 years ago

gabrieldelaparra commented 6 years ago

Implement a HasRank method that returns true/false if the index is ranked.

gabrieldelaparra commented 6 years ago

Add a proposal for the non-ranked case on 47ce528a5d305d1a4d02a21adcd373e6f9f98618

gabrieldelaparra commented 6 years ago

Something to note about pagerank, the sum of all ranks must be 1.

gabrieldelaparra commented 6 years ago

Well, it seems like the boosts can't be updated after the index has been built. I have to modify the IndexBuilder so that it includes the ranks. Great.

Well the good news is that I thought that I would need to update the Create Index tests, but there like none functional tests, just to check for some documents, which is ok at this moment.

gabrieldelaparra commented 6 years ago

The bad news is that the method for calculating the ranks, reads all of the lines in the file. Which is needed to get the ranks. So, two times reading of the billion-lines input file. Not very performance friendly.

gabrieldelaparra commented 6 years ago

Using List to build the graph (for ranks) is eating too much ram. It will definetively not work on a big dataset. Also graph building is taking too much.