Closed jelmervdl closed 12 years ago
I concur that (1), (2) and (5) aren't proper solutions.
(3) is possible. The only issue is that it makes DirectoryCorpusReader::size() an O(n) operation, unless we store the number of entries somewhere on disk (changing the format). I'll start working on this.
I oppose (4) because it violates the RAII principle and complicates the CorpusReader logic greatly because it introduces signals into the backend library.
There's a new branch called lazydirs that implements option 3. I can succesfully open and view a directory treebank with it.
Since opening a file happens at the construction of the object, we cannot tell the object to stop opening a file (or query its progress) For the DirectoryCorpusReader this might be really useful. As far as I understand it isn't possbile to implement this in the DbCorpusReader because dbxml does not support it.
Todo: determine the priority of this feature
The problem: Cannot cancel the open file operation, wich can take a lot of time for big corpusses.
Possible solutions:
I think 3 or 4 are the best options. I think I would prefer 4 above 3 because it is less magical.