CZ-NIC / convey

CSV processing and web related data types mutual conversion
GNU General Public License v3.0
18 stars 4 forks source link

file descriptors #21

Closed e3rd closed 6 years ago

e3rd commented 7 years ago
# We should prefer have data in list a write them in a loop than join data to string and write it (twice as long). 
# Tested on local ram disk and distant network drive.

"""
Bigtext length 42743940
network drive: 1 loop, best of 1: 1.06 s per loop
network drive: 1 loop, best of 1: 1.39 s per loop
network drive: 1 loop, best of 1: 5min 51s per loop open(..,"w")
network drive: 1 loop, best of 1: 100 * (6min 37s) per loop open(..,"a")
"""

import lorem

path = "/tmp/ram/test"
#path = "/mnt/csirt/_test/test"

def init():
    koeficient = 30000
    sentences = [lorem.text() for x in range(koeficient)]
    return " ".join(sentences)
#bigtext = init()

print("Bigtext length", len(bigtext))

def x():    
    with open(path,"w") as f:
        f.write(bigtext)

def y():
    with open(path,"w") as f:
        for s in sentences:
            f.write(s)

def z():
    for s in sentences:
        with open(path,"w") as f:
            f.write(s)
#%timeit -n 1 -r 1 init()
#%timeit -n 1 -r 1 x()
%timeit -n 1 -r 1 y()
%timeit -n 1 -r 1 z()
e3rd commented 6 years ago

done and tested in 48610fcf0bf72ab7b8bb328349c73f0b272426c5