Unable to connect to registries that require SNI

benwaffle commented 8 months ago

What happened in your environment?

I tried to pull an image from registry.fly.io

I am unable to make an HTTPS request over TLS to https://registry.fly.io, because it requires SNI.

❯ ./output/sni 
2024/03/07 02:35:44|INFO |th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/io/epoll.cpp:289|new_epoll_engine:Init event engine: epoll
2024/03/07 02:35:44|INFO |th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/io/signal.cpp:265|sync_signal_init:signalfd initialized
2024/03/07 02:35:44|INFO |th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/io/epoll.cpp:289|new_epoll_engine:Init event engine: epoll
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/curl.cpp:228|libcurl_init:libcurl version libcurl/8.6.0 OpenSSL/3.2.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 libssh2/1.11.0 nghttp2/1.60.0 nghttp3/1.2.0
2024/03/07 02:35:44|INFO |th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/io/epoll.cpp:289|new_epoll_engine:Init event engine: epoll
2024/03/07 02:35:44|INFO |th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/io/epoll.cpp:289|new_epoll_engine:Init event engine: epoll
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/url.cpp:44|from_string:[url=https://registry.fly.io]
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/message.cpp:312|reset:requst reset [u.host()=registry.fly.io][enable_proxy=0]
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/client.cpp:153|do_roundtrip:[concurreny=1]
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/client.cpp:67|dial:Dial to registry.fly.io 443
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/client.cpp:71|dial:Connecting 77.83.143.221:443 ssl: 1
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/security-context/tls-stream.cpp:402|new_tls_stream:New tls stream on [ctx=0000593F3F7A2EC0][base=0000593F3F78A070]
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/client.cpp:81|dial:Connected 77.83.143.221:443 host : registry.fly.io ssl: 1 0000593F3F7E1670
2024/03/07 02:35:44|DEBUG|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/client.cpp:165|do_roundtrip:Sending request 2 /
2024/03/07 02:35:44|ERROR|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/message.cpp:133|send_header:send header failed
2024/03/07 02:35:44|ERROR|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/client.cpp:169|do_roundtrip:send header failed, retry
2024/03/07 02:35:44|ERROR|th=0000593F3F78E900|/home/ben/dev/overlaybd/build/_deps/photon-src/net/http/client.cpp:253|call:connection failed
result: -1

What did you expect to happen?

I expected the exit code to be 0, and no errors in the logs.

How can we reproduce it?

Try running this program:

#include "CLI11.hpp"
#include <photon/photon.h>
#include <photon/common/alog.h>
#include <photon/net/http/verb.h>
#include <photon/net/http/client.h>
#include <photon/net/security-context/tls-stream.h>

using HTTP_OP = photon::net::http::Client::OperationOnStack<64 * 1024 - 1>;

int main(int argc, char **argv) {
    CLI::App app{"ben's cli test"};

    set_log_output_level(0);
    photon::init(photon::INIT_EVENT_DEFAULT, photon::INIT_IO_DEFAULT);
    DEFER({photon::fini();});

    auto tls = photon::net::new_tls_context();
    auto m_client = photon::net::http::new_http_client(nullptr, tls);

    HTTP_OP op(m_client, photon::net::http::Verb::GET, "https://registry.fly.io");
    op.follow = 0;
    op.retry = 0;
    op.req.headers.range(0, 0);
    op.req.headers.insert("flyio-debug", "doit");
    int res = op.call();

    printf("result: %d\n", res);
    exit(res);
}

What is the version of your Overlaybd?

Git main branch - commit bcb8f2103d3c4d28dd8c09e0d96142036fbf29b6

What is your OS environment?

Arch Linux

Are you willing to submit PRs to fix it?

[X] Yes, I am willing to fix it.

benwaffle commented 8 months ago

SNI can be implemented in tls-stream.cpp in Photon using this function when setting up the TLS connection

SSL_set_tlsext_host_name(ssl, "registry.fly.io");

BigVan commented 8 months ago

Yes, we need to update our HTTPClient and TLS-context in photonLibOS to support SNI. Have you tried switch the back-to-source FS 'registryFsVersion'? the default is 'v2', you might change it to 'v1' which will use libcurl for TLS handshake https://github.com/containerd/overlaybd/blob/bcb8f2103d3c4d28dd8c09e0d96142036fbf29b6/src/config.h#L150

benwaffle commented 8 months ago

It seems like registryFS v1 does support SNI correctly, thanks. I also managed to get SNI working in v2.

Why is v1 deprecated? Which one do you recommend I use?

liulanzheng commented 8 months ago

@benwaffle v1 uses dynamically linked libcurl, which often leads to compatibility issues across different os and libcurl versions. Additionally, v2's performance is superior to v1.

benwaffle commented 7 months ago

Once you upgrade the Photon dependency to 0.6.16, we can close this

containerd / overlaybd