Closed chrisdone closed 6 years ago
I didn't find an obvious reason for the linear allocation. I suspect that it's actually the allocation of conduit constructors. Content to leave this alone.
Looks like this PR failed Travis:
pure
https://travis-ci.org/snoyberg/tar-conduit/jobs/327767885Test suite space: RUNNING...
Case Allocated Max Live GCs Check
tar 1 files 33,760 2,104 2,104 0 OK
tar 10 files 260,760 3,544 3,544 0 OK
tar 100 files 2,519,488 0 0 3 OK
tar 1000 files 25,127,488 0 0 32 OK
tar 10000 files 251,207,496 0 0 329 OK
tar file of 1 bytes 33,760 2,160 2,160 0 OK
tar file of 10 bytes 33,760 2,160 2,160 0 OK
tar file of 100 bytes 33,760 2,160 2,160 0 OK
tar file of 1000 bytes 33,760 2,160 2,160 0 OK
tar file of 10000 bytes 33,760 0 0 0 OK
untar 1 files 85,808 10,992 10,992 0 OK
untar 10 files 562,048 17,832 17,832 0 OK
untar 100 files 5,338,936 84,072 84,072 7 OK
untar 1000 files 53,118,672 746,472 746,472 79 OK
untar 10000 files 530,912,200 7,370,472 7,370,472 793 OK
untar file of 1 bytes 85,864 11,104 11,104 0 INVALID
untar file of 10 bytes 85,856 11,096 11,096 0 INVALID
untar file of 100 bytes 85,768 11,008 11,008 0 INVALID
untar file of 1000 bytes 85,376 10,616 10,616 0 INVALID
untar file of 10000 bytes 85,592 10,832 10,832 0 INVALID
Check problems:
untar file of 1 bytes
Exceeded maximum bytes or allocated bytes!
untar file of 10 bytes
Exceeded maximum bytes or allocated bytes!
untar file of 100 bytes
Exceeded maximum bytes or allocated bytes!
untar file of 1000 bytes
Exceeded maximum bytes or allocated bytes!
untar file of 10000 bytes
Exceeded maximum bytes or allocated bytes!
Test suite space: FAIL
Test suite logged to: dist/test/tar-conduit-0.2.0-space.log
1 of 2 test suites (1 of 2 test cases) passed.
As a continuation to be merged AFTER #10, this PR adds time and space tests for tar/untar.
For the most part space usage is constant. There is one interesting result in the case of
untar
:Indicates non-constant max residency for the untaring of n files as you scale n.
I'll take a cursory look through the untar function to see if there's an obvious change to improve this. I imagine it's not a particularly important use-case for IOHK, so it's probably fine like this.
Time numbers:
Allocation report