breezykermo / oak

1 stars 0 forks source link

26 download and load sift1m #34

Closed breezykermo closed 2 weeks ago

breezykermo commented 3 weeks ago

Resolves #26.

Downloads from webpage via FTP, and then assigns attributes to each of the vectors according to the method outlined in the ACORN paper (see readme in the commit for more information).

The current format of predicates is a CSV file that has the same name as the fvecs file, where each row represents one vector, and each column represents an attribute. We may need to change this representation in the future when we come to accommodate predicates over more complex attributes, but it seems fine as a starting point.

Proposing that we merge this into #12 rather than main, as I branched off there, and it's not blocking on anything.