Don't add file without content

testing-cabal / testtools

Testtools - tasteful testing for python

https://testtools.readthedocs.io/en/latest/

Other

95 stars 88 forks source link

Don't add file without content #252

Closed therve closed 7 years ago

therve commented 7 years ago

In the test record API, skip adding file details to the record if there are not bytes. This reduces that amount of data kept when there is none.

freeekanayaka commented 7 years ago

Hi Thomas, long time no see, hope you're good!

How hard would it be to add a unit test for the change? I didn't yet look at existing unit-test coverage for that part of the code.

Also, a couple of questions about the issue:

Is that 25M the size of the subunit artifact for a single run or for N runs?
Did you already verify that the fix in this branch shaves that size significantly? How much?

Cheers

therve commented 7 years ago

Hi Free! Good to hear from you too.

I haven't looked at tests yet, I'll see how's that looking.

The 25M is for a single run. See for example http://logs.openstack.org/56/430056/1/check/gate-heat-python27-db-ubuntu-xenial/41e6c88/testrepository.subunit.gz for the compressed subunit version (50M uncompressed).
The fix reduces the size to about 4M.

freeekanayaka commented 7 years ago

I've modified subunitdetails slightly to inspect the subunit artifact linked above by Thomas.

There are 278024 empty file attachments (I didn't check the type, but presumably it's mostly logging indeed). If we consider an average subunit record size of 75 bytes per attachment (mostly the content-type string plus test ID string), that adds up to about 20M which confirms the analysis.