Closed sahilsk closed 9 years ago
0.7 is a (quite) old version. You should consider using at least 0.8, or 0.9.
Also, can you investigate and point out what exactly is cluttering disk space? (if you are indeed using S3 there absolutely no reason for this, since there is no filesystem caching).
Thanks.
Hi @dmp42
can you help me understand these log lines? I am guessing that registry is caching images in tempfies so that it don't need to request them from s3. However, in doing so it's ignoring remaining disk space... somehow redis queue length is too big .....something like that. What do you suggest?
...
2015-03-07 00:32:03,479 INFO: ParallelKey: <Key: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/layer>; size=193320751
2015-03-07 00:32:03,494 INFO: ParallelKey: <Key: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/layer>; size=193320751
2015-03-07 00:32:03,520 INFO: ParallelKey: <Key: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/layer>; size=193320751
2015-03-07 00:32:03,521 INFO: ParallelKey: <Key: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/layer>; size=193320751
...
{"log":"2015-03-07 00:33:08,125 INFO: ParallelKey: \u003cKey: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/l
ayer\u003e; buffering complete at 32.8% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:33:08
.126475811Z"}
{"log":"2015-03-07 00:33:08,682 INFO: ParallelKey: \u003cKey: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/l
ayer\u003e; buffering complete at 20.0% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:33:08
.682632542Z"}
{"log":"2015-03-07 00:33:13,880 INFO: ParallelKey: \u003cKey: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/l
ayer\u003e; buffering complete at 22.5% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:33:13
.881879406Z"}
{"log":"2015-03-07 00:33:17,570 INFO: ParallelKey: \u003cKey: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/l
ayer\u003e; buffering complete at 0.1% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:33:17.
570786514Z"}
{"log":"2015-03-07 00:33:32,329 INFO: ParallelKey: \u003cKey: organizationimages,images/43fcdb249be8d2cdb8e46166fc4c399c2f5aa673a3ba90be6ad4d7c1a10a100a/l
ayer\u003e; buffering complete at 0.8% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:33:32.
330254169Z"}
{"log":"2015-03-07 00:34:41,233 INFO: ParallelKey: \u003cKey: organizationimages,images/d884e0950871aa73997efed77675146931c6fd75fcf0e0606f1fd35b046e079e/l
ayer\u003e; buffering complete at 0.0% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:34:41.
234784507Z"}
{"log":"2015-03-07 00:34:41,312 INFO: ParallelKey: \u003cKey: organizationimages,images/d884e0950871aa73997efed77675146931c6fd75fcf0e0606f1fd35b046e079e/l
ayer\u003e; buffering complete at 60.0% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:34:41
.312638242Z"}
{"log":"2015-03-07 00:34:41,326 INFO: ParallelKey: \u003cKey: organizationimages,images/d884e0950871aa73997efed77675146931c6fd75fcf0e0606f1fd35b046e079e/l
ayer\u003e; buffering complete at 67.5% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:34:41
.326851049Z"}
{"log":"2015-03-07 00:34:41,621 INFO: ParallelKey: \u003cKey: organizationimages,images/d884e0950871aa73997efed77675146931c6fd75fcf0e0606f1fd35b046e079e/l
ayer\u003e; buffering complete at 53.8% of the total transfer; now serving straight from the tempfile\n","stream":"stderr","time":"2015-03-07T00:34:41
.622623091Z"}
The boto library uses parallel keys to multiplex requests to the s3 backend, and is apparently responsible for this...
I guess we should disable ParallelKeys entirely (see also #956)
Still, you should really update your registry.
Hi,
In merely 2 days of operation docker registry(v0.7.3) is filling up disk space to 100%. And when we restart the resgitry disk space go back 0, clearing up all cached stuff. Should not be its registry task to purge cache if disk reaching threshold value? Any help here would be great.
I'm running registry inside docker with redis lru cache (it's also containerized) enabled.
I am using s3 and my configuration looks like this: