HttpRequests doesn't work?

[amertsalov]

Hi! I faced with a problem while doing simple http request. I used examples/https_client.cpp. I patched it, add resolver in connect. (Want to work with hostname, not ip)

(.venv) amertsalov@vm-msk-amertsalov:~/tmp/CppServer$ git diff examples/https_client.cpp
diff --git a/examples/https_client.cpp b/examples/https_client.cpp
index dbb365b8..70556554 100644
--- a/examples/https_client.cpp
+++ b/examples/https_client.cpp
@@ -47,6 +47,8 @@ int main(int argc, char** argv)
     // Create a new HTTP client
     auto client = std::make_shared<CppServer::HTTP::HTTPSClientEx>(service, context, address, port);

+    std::shared_ptr<CppServer::Asio::TCPResolver> resolver = std::make_shared<CppServer::Asio::TCPResolver>(service);
+
     std::cout << "Press Enter to stop the client or '!' to reconnect the client..." << std::endl;

     try
@@ -62,7 +64,7 @@ int main(int argc, char** argv)
             if (line == "!")
             {
                 std::cout << "Client reconnecting...";
-                client->IsConnected() ? client->ReconnectAsync() : client->ConnectAsync();
+                client->IsConnected() ? client->ReconnectAsync() : client->ConnectAsync(resolver);
                 std::cout << "Done!" << std::endl;
                 continue;
             }
(.venv) amertsalov@vm-msk-amertsalov:~/tmp/CppServer$

Output

(.venv) amertsalov@vm-msk-amertsalov:~/tmp/CppServer/build$ ../temp/cppserver-example-https_client cat-fact.herokuapp.com 443
HTTPS server address: cat-fact.herokuapp.com
HTTPS server port: 443

Asio service starting...Done!
Press Enter to stop the client or '!' to reconnect the client...
!
Client reconnecting...Done!
GET /facts
^C

Curl working pretty fine for this url "curl -v https://cat-fact.herokuapp.com/facts/"

Could you please point on my mistake?

[chronoxor]

Will check it on my side soon

[chronoxor]

Your server requires Host header ("Host: cat-fact.herokuapp.com").

Here is a fully working example. Just print "GET /facts" in console:

using System;
using System.Linq;
using System.Net;
using System.Security.Authentication;
using NetCoreServer;

namespace HttpsClient
{
    class Program
    {
        static void Main(string[] args)
        {
            // HTTPS server address
            string address = "cat-fact.herokuapp.com";
            if (args.Length > 0)
                address = args[0];

            // HTTPS server port
            int port = 443;
            if (args.Length > 1)
                port = int.Parse(args[1]);

            Console.WriteLine($"HTTPS server address: {address}");
            Console.WriteLine($"HTTPS server port: {port}");

            Console.WriteLine();

            // Create and prepare a new SSL client context
            var context = new SslContext(SslProtocols.Tls13, (sender, certificate, chain, sslPolicyErrors) => true);

            // Create a new HTTPS client
            var client = new HttpsClientEx(context, Dns.GetHostAddresses(address).FirstOrDefault(), port);

            Console.WriteLine("Press Enter to stop the client or '!' to reconnect the client...");

            // Perform text input
            for (;;)
            {
                string line = Console.ReadLine();
                if (string.IsNullOrEmpty(line))
                    break;

                // Reconnect the client
                if (line == "!")
                {
                    Console.Write("Client reconnecting...");
                    if (client.IsConnected)
                        client.ReconnectAsync();
                    else
                        client.ConnectAsync();
                    Console.WriteLine("Done!");
                    continue;
                }

                var commands = line.Split(' ');
                if (commands.Length < 2)
                {
                    Console.WriteLine("HTTP method and URL must be entered!");
                    continue;
                }

                if (commands[0].ToUpper() == "GET")
                {
                    client.Request.Clear();
                    client.Request.SetBegin("GET", commands[1]);
                    client.Request.SetHeader("Host", "cat-fact.herokuapp.com");
                    client.Request.SetBody();
                    var response = client.SendRequest(client.Request).Result;
                    Console.WriteLine(response);
                }
                else
                    Console.WriteLine("Unknown HTTP method");
            }

            // Disconnect the client
            Console.Write("Client disconnecting...");
            client.Disconnect();
            Console.WriteLine("Done!");
        }
    }
}

[chronoxor]

To get more information about connecting errors, just inherit your client from MyClient : HttpsClientEx and override OnError() method.

[amertsalov]

Thank you for quick response! I didn't get.. It's not even a c++. I spend all day today, trying to make library working, and of course i tried to set headers like curl set, and log error in OnError method. Nothing helped. I find out that you library has low latency on sending http request + it has quite good design. That's why i'm REALLY interesting to make some working example, not only in benchmarks. I will really appreciate if you help me with it.

I tried convert you code to c++.

(.venv) amertsalov@vm-msk-amertsalov:~/tmp/CppServer$ cat LOG
diff --git a/examples/https_client.cpp b/examples/https_client.cpp
index dbb365b8..474f170b 100644
--- a/examples/https_client.cpp
+++ b/examples/https_client.cpp
@@ -13,6 +13,16 @@

 #include <iostream>

+
+class MyClient: public CppServer::HTTP::HTTPSClientEx {
+public:
+  using CppServer::HTTP::HTTPSClientEx::HTTPSClientEx;
+
+  virtual void onError(int error, const std::string& category, const std::string& message) {
+    std::cout << "onError: " << error << " " << category << " " << message << std::endl;
+  }
+};
+
 int main(int argc, char** argv)
 {
     // HTTP server address
@@ -45,8 +55,8 @@ int main(int argc, char** argv)
     context->load_verify_file("../tools/certificates/ca.pem");

     // Create a new HTTP client
-    auto client = std::make_shared<CppServer::HTTP::HTTPSClientEx>(service, context, address, port);
-
+    auto client = std::make_shared<MyClient>(service, context, address, port);
+    std::shared_ptr<CppServer::Asio::TCPResolver> resolver = std::make_shared<CppServer::Asio::TCPResolver>(service);
     std::cout << "Press Enter to stop the client or '!' to reconnect the client..." << std::endl;

     try
@@ -62,7 +72,7 @@ int main(int argc, char** argv)
             if (line == "!")
             {
                 std::cout << "Client reconnecting...";
-                client->IsConnected() ? client->ReconnectAsync() : client->ConnectAsync();
+                client->IsConnected() ? client->ReconnectAsync() : client->ConnectAsync(resolver);
                 std::cout << "Done!" << std::endl;
                 continue;
             }
@@ -81,8 +91,14 @@ int main(int argc, char** argv)
             }
             else if (CppCommon::StringUtils::ToUpper(commands[0]) == "GET")
             {
-                auto response = client->SendGetRequest(commands[1]).get();
-                std::cout << response << std::endl;
+              auto request = CppServer::HTTP::HTTPRequest();
+              request.Clear();
+              request.SetBegin("GET", "/facts");
+              request.SetHeader("Host", "cat-fact.herokuapp.com");
+              request.SetBody();
+
+              auto response = client->SendRequest(request).get();
+              std::cout << response;
             }
             else if (CppCommon::StringUtils::ToUpper(commands[0]) == "POST")
             {

The code above stuck as well and didn't log any error.

If you interesting in backtrace:

Thread 3 (Thread 0x7ffff6d6f700 (LWP 9544) "cppserver-examp"):
#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x555555677ae8) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x555555677a88, cond=0x555555677ac0) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x555555677ac0, mutex=0x555555677a88) at pthread_cond_wait.c:647
#3  0x00005555555f8b43 in asio::detail::scheduler::do_run_one(asio::detail::conditionally_enabled_mutex::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) ()
#4  0x00005555555ee671 in asio::detail::scheduler::run(std::error_code&) ()
#5  0x0000555555609445 in asio::detail::posix_thread::func<asio::detail::resolver_service_base::work_scheduler_runner>::run() ()
#6  0x00005555555f640d in asio_detail_posix_thread_function ()
#7  0x00007ffff778d609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#8  0x00007ffff76b2133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7ffff7570700 (LWP 9543) "cppserver-examp"):
#0  0x00007ffff76b246e in epoll_wait (epfd=4, events=0x7ffff756f620, maxevents=128, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1  0x00005555555f4fac in asio::detail::epoll_reactor::run(long, asio::detail::op_queue<asio::detail::scheduler_operation>&) ()
#2  0x00005555555f8bb8 in asio::detail::scheduler::do_run_one(asio::detail::conditionally_enabled_mutex::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) ()
#3  0x00005555555ee671 in asio::detail::scheduler::run(std::error_code&) ()
#4  0x00005555555ee515 in asio::io_context::run() ()
#5  0x00005555555892f1 in CppServer::Asio::Service::ServiceThread(std::shared_ptr<CppServer::Asio::Service> const&, std::shared_ptr<asio::io_context> const&) ()
#6  0x00007ffff79e8de4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007ffff778d609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#8  0x00007ffff76b2133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7ffff7573c00 (LWP 9539) "cppserver-examp"):
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1  0x00007ffff79e6859 in std::__atomic_futex_unsigned_base::_M_futex_wait_until(unsigned int*, unsigned int, bool, std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long, std::ratio<1l, 1000000000l> >) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#2  0x000055555556ac30 in std::__basic_future<CppServer::HTTP::HTTPResponse>::_M_get_result() const ()
#3  0x0000555555568e5c in std::future<CppServer::HTTP::HTTPResponse>::get() ()
#4  0x0000555555567754 in main ()
(gdb)

[chronoxor]

Sorry, I missed that the issue was created in CppServer, not NetCoreServer)) Will check it in c++ soon...

[chronoxor]

Here is a C++ working example on connecting to your server:

#include "asio_service.h"

#include "server/http/https_client.h"
#include "string/string_utils.h"

#include <iostream>

int main(int argc, char** argv)
{
    // HTTP server address
    std::string address = "cat-fact.herokuapp.com";
    if (argc > 1)
        address = argv[1];
    // HTTP server port
    int port = 443;
    if (argc > 2)
        port = std::atoi(argv[2]);

    std::cout << "HTTPS server address: " << address << std::endl;
    std::cout << "HTTPS server port: " << port << std::endl;

    std::cout << std::endl;

    // Create a new Asio service
    auto service = std::make_shared<AsioService>();

    // Start the Asio service
    std::cout << "Asio service starting...";
    service->Start();
    std::cout << "Done!" << std::endl;

    // Create and prepare a new SSL client context
    auto context = std::make_shared<CppServer::Asio::SSLContext>(asio::ssl::context::tlsv12);
    context->set_default_verify_paths();
    context->set_root_certs();
    context->set_verify_mode(asio::ssl::verify_peer | asio::ssl::verify_fail_if_no_peer_cert);
    context->set_verify_callback(asio::ssl::rfc2818_verification(address));

    // Create a new HTTP client
    auto client = std::make_shared<CppServer::HTTP::HTTPSClientEx>(service, context, address, port);
    auto resolver = std::make_shared<CppServer::Asio::TCPResolver>(service);

    std::cout << "Press Enter to stop the client or '!' to reconnect the client..." << std::endl;

    try
    {
        // Perform text input
        std::string line;
        while (getline(std::cin, line))
        {
            if (line.empty())
                break;

            // Reconnect the client
            if (line == "!")
            {
                std::cout << "Client reconnecting...";
                client->IsConnected() ? client->ReconnectAsync() : client->ConnectAsync(resolver);
                std::cout << "Done!" << std::endl;
                continue;
            }

            auto commands = CppCommon::StringUtils::Split(line, ' ', true);
            if (commands.size() < 2)
            {
                std::cout << "HTTP method and URL must be entered!" << std::endl;
                continue;
            }

            if (CppCommon::StringUtils::ToUpper(commands[0]) == "GET")
            {
                auto& request = client->request();
                request.Clear();
                request.SetBegin("GET", commands[1]);
                request.SetHeader("Host", address);
                request.SetBody();
                auto response = client->SendRequest(request).get();
                std::cout << response << std::endl;
            }
            else
                std::cout << "Unknown HTTP method: " << commands[0] << std::endl;
        }
    }
    catch (const std::exception& ex)
    {
        std::cerr << ex.what() << std::endl;
    }

    // Stop the Asio service
    std::cout << "Asio service stopping...";
    service->Stop();
    std::cout << "Done!" << std::endl;

    return 0;
}

[chronoxor]

It seems your HTTPS server doesn't support TLS 1.3 properly, so I changed to TLS 1.2 and the client works fine for me

[amertsalov]

Thank you, you are right. Now it's working!