search

spotify / XCRemoteCache

Other
830 stars 53 forks source link

error: XCRemoteCache cannot be initialized with a consumer mode. Error: noArtifactsToReuse #86

Closed aturan23 closed 2 years ago

aturan23 commented 2 years ago

My integration setup Automatic integration using xcprepare integrate ...

.rcinfo:

primary_repo: XYZ.git
primary_branch: dev
cache_addresses:
- http://localhost:8080/cache

producer:

xcremotecache/xcprepare integrate --input XYZ.xcodeproj --mode producer --final-producer-target XYZ

consumer:

xcremotecache/xcprepare integrate --input XYZ.xcodeproj --mode consumer

Minimal reproduction of the problem with instructions

Producer Logs

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration

/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/

/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh

10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf

10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf

/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh

/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh

/docker-entrypoint.sh: Configuration complete; ready for start up

172.17.0.1 - - [16/Feb/2022:12:39:48 +0000] "PUT /cache/file/205a0bc8a669a6ca4e9ae5cc649ad81c.zip HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:39:48 +0000] "PUT /cache/file/d284af114efa98b27224c66216381c09.zip HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:39:48 +0000] "PUT /cache/meta/a9dce047d8a4fd37e6236a0041b03ea3ba619d8e-MB-XYZExtension-Debug-iphonesimulator-13C100-b74ff62c1fe3a4cab72d464c04ac1f4f.json HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:39:48 +0000] "PUT /cache/meta/a9dce047d8a4fd37e6236a0041b03ea3ba619d8e-MB-XYZExtensionUI-Debug-iphonesimulator-13C100-b0ecc6077e8e2a2ccdbcd6ff472ccff0.json HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:40:16 +0000] "PUT /cache/file/6f604311c43eee7050e215229d51a771.zip HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:40:16 +0000] "PUT /cache/meta/a9dce047d8a4fd37e6236a0041b03ea3ba619d8e-MB-XYZExtension-Debug-iphonesimulator-13C100-03fd65f0c2f2972cf8b38af23c97fb7d.json HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:42:01 +0000] "PUT /cache/file/fcb00f22a4504673380f974be243a395.zip HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:42:01 +0000] "PUT /cache/meta/a9dce047d8a4fd37e6236a0041b03ea3ba619d8e-MB-XYZExtension-Debug-iphonesimulator-13C100-807b9a67c40db2b7c9fb7afed62040f0.json HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:42:01 +0000] "PUT /cache/file/087a3cc7a43d51020605097c693ef420.zip HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:42:01 +0000] "PUT /cache/meta/a9dce047d8a4fd37e6236a0041b03ea3ba619d8e-XYZExtension-Debug-iphonesimulator-13C100-1efeb380bf880878b4e0451cac0b1f7b.json HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:42:01 +0000] "PUT /cache/file/d3ee1382cfc8ee0159b043e9fa6fb65d.zip HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:42:01 +0000] "PUT /cache/meta/a9dce047d8a4fd37e6236a0041b03ea3ba619d8e-MB-XYZExtension-Debug-iphonesimulator-13C100-ed2de1ea7ef7e20cbe5ad91f9098cd12.json HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:58:29 +0000] "PUT /cache/file/e0b4645c0bf04ecfcd9fc05b1089ef9a.zip HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:58:29 +0000] "PUT /cache/meta/a9dce047d8a4fd37e6236a0041b03ea3ba619d8e-XYZ-Debug-iphonesimulator-13C100-80eb0cccefe0ae4acf672c5523bebbd1.json HTTP/1.1" 201 0 "-" "xcpostbuild (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [16/Feb/2022:12:58:31 +0000] "PUT /cache/marker/a9dce047d8a4fd37e6236a0041b03ea3ba619d8e-Debug-iphonesimulator-13C100-5 HTTP/1.1" 201 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

Consumer Logs

 2022/02/16 13:15:48 [error] 31#31: *9 open() "/tmp/cache/cache/nginx-health" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/nginx-health HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:48 +0000] "HEAD /cache/nginx-health HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:48 [error] 31#31: *9 open() "/tmp/cache/cache/nginx-health" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/nginx-health HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:48 +0000] "HEAD /cache/nginx-health HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:48 [error] 31#31: *9 open() "/tmp/cache/cache/nginx-health" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/nginx-health HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:48 +0000] "HEAD /cache/nginx-health HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:48 [error] 31#31: *9 open() "/tmp/cache/cache/marker/ec97526a50481f1f3655cd63fdacdfe0b4e16816-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/ec97526a50481f1f3655cd63fdacdfe0b4e16816-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:48 +0000] "HEAD /cache/marker/ec97526a50481f1f3655cd63fdacdfe0b4e16816-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/b7201e2993c2f1e890c860741b691fe937e428ac-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/b7201e2993c2f1e890c860741b691fe937e428ac-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/b7201e2993c2f1e890c860741b691fe937e428ac-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/3538b470e26098a0758dc8cf95fa9bf10d410028-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/3538b470e26098a0758dc8cf95fa9bf10d410028-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/3538b470e26098a0758dc8cf95fa9bf10d410028-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/08571090d17810dd1ae7fdb01c49c21df38a3e39-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/08571090d17810dd1ae7fdb01c49c21df38a3e39-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/08571090d17810dd1ae7fdb01c49c21df38a3e39-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/4004ba2250cfaec66336f304181c3e8f1cf64a7e-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/4004ba2250cfaec66336f304181c3e8f1cf64a7e-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/4004ba2250cfaec66336f304181c3e8f1cf64a7e-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/038541815c72a461cbb851f548bc0ce957dc213c-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/038541815c72a461cbb851f548bc0ce957dc213c-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/038541815c72a461cbb851f548bc0ce957dc213c-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/92d5ff06f8f4ae65ec4baab85bc828df1877c792-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/92d5ff06f8f4ae65ec4baab85bc828df1877c792-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/92d5ff06f8f4ae65ec4baab85bc828df1877c792-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/0772ade22b0f31d5eb585c1d421abcb7b7f64f80-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/0772ade22b0f31d5eb585c1d421abcb7b7f64f80-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/0772ade22b0f31d5eb585c1d421abcb7b7f64f80-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/5789427d3cf952bad33bd7ad82b507face7a8e4b-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/5789427d3cf952bad33bd7ad82b507face7a8e4b-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/5789427d3cf952bad33bd7ad82b507face7a8e4b-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/16 13:15:49 [error] 31#31: *9 open() "/tmp/cache/cache/marker/c7119ef77c1a6798655c18dd6445f7a0e83f0e90-Debug-iphonesimulator-13C100-5" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/marker/c7119ef77c1a6798655c18dd6445f7a0e83f0e90-Debug-iphonesimulator-13C100-5 HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [16/Feb/2022:13:15:49 +0000] "HEAD /cache/marker/c7119ef77c1a6798655c18dd6445f7a0e83f0e90-Debug-iphonesimulator-13C100-5 HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

Environment

xwlxwl commented 2 years ago

you should mark your Artifacts,like that. xcremotecache/xcprepare mark --configuration Debug --platform iphonesimulator

xwlxwl commented 2 years ago

you should mark your Artifacts,like that. xcremotecache/xcprepare mark --configuration Debug --platform iphonesimulator

if your source code not in master ,you will add primary_branch: "XXXXX" in your .rcinfo, otherwise it will find last commit in master branch

aturan23 commented 2 years ago

you should mark your Artifacts,like that. xcremotecache/xcprepare mark --configuration Debug --platform iphonesimulator

I tried automatically integrate. I think this command executes automatically

if your source code not in master ,you will add primary_branch: "XXXXX" in your .rcinfo, otherwise it will find last commit in master branch

I wrote primary branch on .rcinfo

aturan23 commented 2 years ago

It's like my mistake. Now all works correctly. Steps:

  1. Move XCRemoteCache folder into project. create .rcinfo
  2. Clean Build & Remove Derived data
  3. Run producer command
  4. After build did git reset --hard
  5. Repeat 1 & 2 step.
  6. Run consumer command.

On executing consumer docker prints:

2022/02/17 11:36:03 [error] 33#33: *9 open() "/tmp/cache/cache/nginx-health" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/nginx-health HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [17/Feb/2022:11:36:03 +0000] "HEAD /cache/nginx-health HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/17 11:36:03 [error] 33#33: *9 open() "/tmp/cache/cache/nginx-health" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/nginx-health HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [17/Feb/2022:11:36:03 +0000] "HEAD /cache/nginx-health HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

2022/02/17 11:36:03 [error] 33#33: *9 open() "/tmp/cache/cache/nginx-health" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "HEAD /cache/nginx-health HTTP/1.1", host: "localhost:8080"

172.17.0.1 - - [17/Feb/2022:11:36:03 +0000] "HEAD /cache/nginx-health HTTP/1.1" 404 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

172.17.0.1 - - [17/Feb/2022:11:36:04 +0000] "HEAD /cache/marker/87531d87edf3be5a43cf9aaea0615812ff3b9298-Debug-iphonesimulator-13C100-5 HTTP/1.1" 200 0 "-" "xcprepare (unknown version) CFNetwork/1327.0.4 Darwin/21.2.0"

What it means?

  1. Run build on Xcode. And on docker GET requests successfully finished.

After some experiments get maximum -35% build time. Is it ok?)

I noticed one more thing. After a successful build, if you do Clean Build & Remove Derived Data it again, then I don't see any download logs in the docker. It turns out that the consumer download happens only for the first time?

polac24 commented 2 years ago
  1. If you ask what is HEAD /cache/nginx-health HTTP/1.1" 404 -> XCRemoteCache makes some probe requests to find which remote cache to use. It is not perfect and makes that step even if you have a single cache server. [PR welcome]
  2. 35% cache hit is very low. If you don't make any changes comparing the producer, the consumer should have 100%.
  3. There is a local cache - default at ~/Library/Caches/XCRemoteCache`

Please review your consumer logs and double-check your meta json files don't contain absolute paths.

aturan23 commented 2 years ago

Thanks!