photon_lib issue in Mariner

[juliusl]

What happened in your environment?

2023/09/18 23:32:09|INFO |th=000078E7023FBF40|/src/src/image_service.cpp:148|load_cred_from_http:request url: <REDACTED>
2023/09/18 23:32:09|ERROR|th=000078E7023FBF40|/src/build/_deps/photon-src/net/curl.cpp:89|curl_perform:failed to curl_multi_add_handle(): API function called from within callback
2023/09/18 23:32:09|ERROR|th=000078E7023FBF40|/src/src/image_service.cpp:152|load_cred_from_http:connect to auth component failed. http response code: -1 errno=5(Input/output error)
2023/09/18 23:32:09|ERROR|th=000078E7023FBF40|/src/build/_deps/photon-src/net/curl.cpp:89|curl_perform:failed to curl_multi_add_handle(): The easy handle is already added to a multi handle

This error is happening for all curl calls. It looks like the first curl call at the top of the logs succeeds and all subsequent calls start to fail,

2023/09/18 23:32:06|INFO |th=000078E70D6A8E80|/src/src/image_service.cpp:148|load_cred_from_http:request url: <REDACTED>
2023/09/18 23:32:07|INFO |th=000078E70D6A8E80|/src/src/image_service.cpp:160|load_cred_from_http:traceId: ${trace_id}, succ: 1
2023/09/18 23:32:07|INFO |th=000078E70D6A8E80|/src/src/image_service.cpp:249|reload_auth:auth found for <REDACTED>
2023/09/18 23:32:07|INFO |th=000078E70D6A8E80|/src/src/overlaybd/registryfs/registryfs.cpp:330|operator():authenticate for: <REDACTED>, time used: 26 ms
2023/09/18 23:32:07|ERROR|th=000078E70EEB9280|/src/build/_deps/photon-src/net/curl.cpp:89|curl_perform:failed to curl_multi_add_handle(): API function called from within callback
2023/09/18 23:32:07|ERROR|th=000078E70FEC9300|/src/build/_deps/photon-src/net/curl.cpp:89|curl_perform:failed to curl_multi_add_handle(): API function called from within callback
2023/09/18 23:32:07|ERROR|th=000078E70FEC9300|/src/src/overlaybd/registryfs/registryfs.cpp:253|getActualUrl:Failed to get actual url [url=<REDACTED>][ret=-1]

What did you expect to happen?

The i/o request completes successfully.

How can we reproduce it?

1) Create a mariner machine 2) Setup overlaybd 3) Remote pull from a private registry

What is the version of your Overlaybd?

0.6.15

What is your OS environment?

Mariner

Are you willing to submit PRs to fix it?

• [X] Yes, I am willing to fix it.

[Coldwings]

It is a known issue. In libcurl ver. 7.59.0 a number of checks were added for recusive calls. The PhotonLibOS using a special way to interacting with libcurl (by multicurl handler and libcurl event notifications), used to fits libcurl version lower than 7.59.0, but may fail in newer versions.

Provides another version of libcurl (7.42 or newer, but should be older than 7.59.0) helps to resolve this problem.

PhotonLibOS will migrate to own-implemented http client, instead of libcurl. OverlayBD will also change to photon::net::http in the future. That would helps resolve this problem permanently.

[northtyphoon]

7.59.0 was released 2018/1/24 which is very old. Should overlaybd build script pin to a particular version?

[juliusl]

It is a known issue. In libcurl ver. 7.59.0 a number of checks were added for recusive calls. The PhotonLibOS using a special way to interacting with libcurl (by multicurl handler and libcurl event notifications), used to fits libcurl version lower than 7.59.0, but may fail in newer versions.

Provides another version of libcurl (7.42 or newer, but should be older than 7.59.0) helps to resolve this problem.

PhotonLibOS will migrate to own-implemented http client, instead of libcurl. OverlayBD will also change to photon::net::http in the future. That would helps resolve this problem permanently.

@Coldwings Is it possible to statically link libcurl for the mariner build?

[lihuiba]

Should overlaybd build script pin to a particular version?

Yes