search

phischu / hackage-analysis

Find out stuff about packages on hackage (especially how they evolve).
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

Chunk database insertion #17

Closed phischu closed 11 years ago

phischu commented 11 years ago

Currently the entire data is inserted into the database in one big transaction. This only works because only a tiny fraction of the data is used. Someone really needs to find a way to import data in chunks. One major obstacle is to connect these chunks via edges because then the actual node ids would have to be read back.

phischu commented 11 years ago

Still needs some major cleanup but works!