jankratochvil
commented
1 year ago It would be nice if you can click on resolve for comments which you find as addressed by your commit(s).
Closed wkia closed 1 year ago
jankratochvil
commented
1 year ago It would be nice if you can click on resolve for comments which you find as addressed by your commit(s).
AntonKozlov
commented
1 year ago There is a problem when you store an uncompressed image into a image dir which already contains compressed image, the restore will prefer the old compressed pages image (which will be out of sync with other files, thus the complete image dir is invalid). Not sure if to handle in CRaC JDK or in CRIU, e.g. by removing pages-1.img{,.comp} from the image directroy on every checkpoint.
In the rest, looks good!
With the new compression, we'll be able to remove external compression from the example-lambda (WIP: https://github.com/AntonKozlov/example-lambda/commit/5f17101861b75c5f364c4abad974bdf90a88ba21), and even have better restore results!
anton@mercury:~/proj/org-crac/example-lambda$ awk '/Billed Duration/' old.log
REPORT RequestId: df3c7070-f1ba-4d24-af2c-e05174a59fda Init Duration: 0.47 ms Duration: 633.33 ms Billed Duration: 634 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: f1333903-9c53-4c12-a029-13a5630b7172 Init Duration: 0.34 ms Duration: 635.58 ms Billed Duration: 636 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: d6f981ac-f379-4ef9-9905-2e20345d17e3 Init Duration: 0.31 ms Duration: 635.34 ms Billed Duration: 636 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: feaff9e8-f5c4-4bea-ac10-d463ea88687d Init Duration: 0.47 ms Duration: 621.41 ms Billed Duration: 622 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 2bdd494e-873b-4989-8d1f-8bed60c41947 Init Duration: 0.44 ms Duration: 634.22 ms Billed Duration: 635 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 6b7d49e9-9279-4d2a-ba75-88d5e65a61d5 Init Duration: 0.29 ms Duration: 590.21 ms Billed Duration: 591 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 8e308650-1d4d-4bcb-bbc7-afe73a9522c9 Init Duration: 0.39 ms Duration: 596.12 ms Billed Duration: 597 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 7f391b3b-8c06-47dd-8df9-eeb83bff3250 Init Duration: 0.81 ms Duration: 572.23 ms Billed Duration: 573 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 8092e26e-fa03-44c0-91b7-16daec98588d Init Duration: 0.50 ms Duration: 596.31 ms Billed Duration: 597 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 5238c48e-bd0d-40eb-9492-3920a37a9c83 Init Duration: 0.57 ms Duration: 634.59 ms Billed Duration: 635 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
anton@mercury:~/proj/org-crac/example-lambda$ awk '/Billed Duration/' new.log
REPORT RequestId: 22a2edfe-0aa4-4330-8989-6a9a30dbb05f Init Duration: 0.39 ms Duration: 539.29 ms Billed Duration: 540 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: dbdfd52a-f809-4aff-b0ff-4b2c4f003488 Init Duration: 0.38 ms Duration: 546.29 ms Billed Duration: 547 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: a1e17887-db1f-41af-a9cb-246d9fd6e680 Init Duration: 0.44 ms Duration: 530.68 ms Billed Duration: 531 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 8656af14-f247-4443-91f9-d09b69394e79 Init Duration: 0.67 ms Duration: 556.76 ms Billed Duration: 557 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 73618520-fd15-470f-9a27-b10151b24a59 Init Duration: 0.51 ms Duration: 557.70 ms Billed Duration: 558 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 0336e4c2-f2ad-4b8a-9534-673fa7d034ff Init Duration: 0.44 ms Duration: 549.58 ms Billed Duration: 550 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 92fc978a-ad1b-4700-84ac-5c662e37502a Init Duration: 0.26 ms Duration: 555.26 ms Billed Duration: 556 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 99fe7501-1a78-45b1-9ac5-f8850354dd11 Init Duration: 0.48 ms Duration: 554.17 ms Billed Duration: 555 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 30ff3e96-37a9-4e86-a7c0-ab39c09fef0c Init Duration: 0.48 ms Duration: 553.33 ms Billed Duration: 554 ms Memory Size: 3008 MB Max Memory Used: 3008 MB
REPORT RequestId: 92d5bc53-09ba-4004-ac82-167d755226a4 Init Duration: 0.17 ms Duration: 552.56 ms Billed Duration: 553 ms Memory Size: 3008 MB Max Memory Used: 3008 MB Hearing no objections, I'm integrating this.
These changes implement asynchronous decompression of an image (pages-1.img) on restore stage. CRIU compression can be enabled by specifying '--compress' command line option. Decompression is enabled automatically if a compressed image exists. LZ4 library is used as compression algorithm implementation.
On checkpoint:
After a process dump is completed, CRIU synchronously compresses 'pages-1.img' into 'pages-1.comp.img' at the same folder as the original file located, using LZ4 level 8. Then CRIU deletes the original img file.
On restore:
On start, CRIU creates the decompression thread, and decompresses 'pages-1.comp.img' asynchronously. The current implementation decompresses an image into '/tmp' folder. When the 'restore' procedure tries to obtain a file descriptor of the decompressed file, it waits for decompression is finished if needed. Then it restores the process as usual.