albfernandez / javadbf

Java library for reading and writing Xbase (dBase/DBF) files.
GNU Lesser General Public License v3.0
224 stars 98 forks source link

Use index files to search #39

Closed SirError closed 9 months ago

SirError commented 7 years ago

I'm not sure if this one is possible. To be honest I don't understand DBF very well, but my boss used to use them back then, now I need to migrate some data from DBF to Postgresql. Some files are really huge like (>1million rows). I have to filter the data based on the company's code. Today, I would have to iterate through the the whole file like this:

    while ((os = reader.nextRecord()) != null) {
        String cod = (String) os.get("EMPCOD");
        if (cod.equals('001'){
            System.out.println(cod);
        }
    }

Talking to my boss he said if we could use the index we wouldn't have to iterate through the file, so the process could be much faster.

So as I said before, not sure if this is possible, however it'd be great if we could have something like:


reader = new DBFReader(new FileInputStream(args[0]));
reader.filter().equals("EMPCOD", "001");            
Object[] rowObjects;
while ((rowObjects = reader.nextRecord()) != null) {
     for (int i = 0; i < rowObjects.length; i++) {
       System.out.println(rowObjects[i]);
    }
}

Then the library would use the index to filter only the rows with the condition.

Anyway, just an idea.

Thanks.

albfernandez commented 7 years ago

Sorry, but indexes are not supported at the moment

SirError commented 6 years ago

But do you think this might be possible someday?

albfernandez commented 6 years ago

It's difficult to implement and I've not much time, sorry.

I left this issue open, maybe someone can help with it.