Currently, tart push might consume substantial amounts of memory when pushing an image such as tart create --linux linux:
This could've been solved by using an autoreleasepool:
diff --git a/Sources/tart/OCI/Layerizer/DiskV2.swift b/Sources/tart/OCI/Layerizer/DiskV2.swift
index 04e8c0c..f0b9d46 100644
--- a/Sources/tart/OCI/Layerizer/DiskV2.swift
+++ b/Sources/tart/OCI/Layerizer/DiskV2.swift
@@ -124,18 +124,20 @@ class DiskV2: Disk {
// Create a compressing filter that we will terminate upon
// reaching ``Self.layerLimitBytes`` of compressed data
let compressingFilter = try InputFilter(.compress, using: .lz4, bufferCapacity: bufferSizeBytes) { (length: Int) -> Data? in
- if compressedData.count >= Self.layerLimitBytes {
- return nil
- }
+ try autoreleasepool {
+ if compressedData.count >= Self.layerLimitBytes {
+ return nil
+ }
- guard let uncompressedChunk = try disk.read(upToCount: bufferSizeBytes) else {
- return nil
- }
+ guard let uncompressedChunk = try disk.read(upToCount: bufferSizeBytes) else {
+ return nil
+ }
- bytesRead += UInt64(uncompressedChunk.count)
- digest.update(uncompressedChunk)
+ bytesRead += UInt64(uncompressedChunk.count)
+ digest.update(uncompressedChunk)
- return uncompressedChunk
+ return uncompressedChunk
+ }
}
// Retrieve compressed data chunks, but normally no more than ``Self.layerLimitBytes`` bytes
However, the design of InputFilter makes things looking weird, because we'd seemingly pass released objects (?) for further consumption by the filter.
I've done some benchmarks against a local container registry, here are the results for autoreleasepool:
% hyperfine --runs 10 --prepare 'sync && sudo purge' './tart push --insecure linux 127.0.0.1:8080/a/b:latest'
Benchmark 1: ./tart push --insecure linux 127.0.0.1:8080/a/b:latest
Time (mean ± σ): 36.907 s ± 0.386 s [User: 26.139 s, System: 8.097 s]
Range (min … max): 36.297 s … 37.524 s 10 runs
And here are the results when using a memory-mapped disk:
% hyperfine --runs 10 --prepare 'sync && sudo purge' './tart push --insecure linux 127.0.0.1:8080/a/b:latest'
Benchmark 1: ./tart push --insecure linux 127.0.0.1:8080/a/b:latest
Time (mean ± σ): 38.253 s ± 0.841 s [User: 26.288 s, System: 9.497 s]
Range (min … max): 37.126 s … 39.834 s 10 runs
The results look pretty much the same, so I'd propose to fix the memory footprint for now using mmap(2) and then reconsider autoreleasepool later, once we gain some more insight into the autoreleasepool.
Currently,
tart push
might consume substantial amounts of memory when pushing an image such astart create --linux linux
:This could've been solved by using an
autoreleasepool
:However, the design of
InputFilter
makes things looking weird, because we'd seemingly pass released objects (?) for further consumption by the filter.I've done some benchmarks against a local container registry, here are the results for
autoreleasepool
:And here are the results when using a memory-mapped disk:
The results look pretty much the same, so I'd propose to fix the memory footprint for now using
mmap(2)
and then reconsiderautoreleasepool
later, once we gain some more insight into theautoreleasepool
.