jegesh / python-random-access-file-reader

Low memory usage random access reader for csv and general files
14 stars 6 forks source link
csv-parsing fileio python random-access

Random Access File Reader

| This is a small library that allows for reading any given line in a file without having to read all the lines before it or load the entire file into memory. Only the line indexes and lengths are held in memory, which enables random access even on very large files for a very minuscule memory cost.

Installation

pip install random-access-file-reader

Usage

Usage is very straightforward, and standard csv line endings (newline character), value delimiter (comma), and quotation character (double quote) are the defaults. These can be changed in the constructor.
The get_line() and get_line_dicts() methods return a list of rows.
Plain text file example:

::

from randomAccessReader import RandomAccessReader

reader = RandomAccessReader('~/myfile.txt')

# single line
line = reader.get_lines(2)[0]
print line

# multiple lines
lines = reader.get_lines(3, 3)
for l in lines:
    print l

| Optional arguments in the constructor:

| | Csv example:

::

from randomAccessReader import CsvRandomAccessReader
reader = CsvRandomAccessReader('~/myfile.csv')

# single line
line = reader.get_line_dicts(5)[0]
for x in line:
    print x + " = " line[x]

# multiple lines
lines = reader.get_line_dicts(6, 6)
for l in lines:
    for x in l:
        print x + " = " l[x]

| Optional arguments in the constructor: