static-frame / arraykit

Python C Extensions for StaticFrame
Other
8 stars 2 forks source link

Buffered byte writing to an open file #61

Open flexatone opened 2 years ago

flexatone commented 2 years ago

SF's NPYConvert.to_py has to do the following. This can probably be done more efficiently with the C API such that no intermediate arrays (each chunck) need to be created.

        if fortran_order and not flags.c_contiguous:
            for chunk in np.nditer(
                    array,
                    flags=cls.NDITER_FLAGS,
                    buffersize=buffersize,
                    order='F',
                    ):
                file.write(chunk.tobytes('C'))
        else:
            for chunk in np.nditer(
                    array,
                    flags=cls.NDITER_FLAGS,
                    buffersize=buffersize,
                    order='C',
                    ):
                file.write(chunk.tobytes('C'))