rootrunner commented 1 month ago

Apologies for the bad formatting but it just wouldn't accept it as it should.

I have a linux application and the last class I've added uses CrowCpp to serve a website that'll become the gui in the end. When I interrupt the program by pressing CTRL-C the main program loop just keeps on running. I do not have that issue with instances of other classes. Everything shuts down cleanly but if I dare run the website it no longer works. I have eventhandlers in place for amongst others SIGHUP and SIGINT. SIGINT is the equivalent of pressing CTRL-C in a terminal. Both can be sent as kill -SIGHUP pid and kill -SIGINT pid. The latter is thus the equivalent op pressing CTRL-C.

I made a POC that only installs 2 signal handlers that exhibits the same problem. It could be a bit shorter but I kept the main program loop the same as in the real code.

`#include

include

#include <mutex>
#include <thread>
#include <atomic>
#include <crow.h>   
class MyCrowApp {
public:
    MyCrowApp(unsigned int port) : port(port) {
        std::cout << "Constructing MyCrowApp instance." << std::endl;
        processingThread = std::thread(&MyCrowApp::runWebserver, this); // Start webserver in a separate thread.
    }
    ~MyCrowApp() {
        app.stop();
        if (processingThread.joinable()) {
            processingThread.join();
        }
        std::cout << "Destructed MyCrowApp instance." << std::endl;
    }

    void runWebserver() {
        std::cout << "Webserver starting ..." << std::endl;
        // Define the endpoint at the root directory
        CROW_ROUTE(app, "/")([](){
            return "Hello world";
        });
        // Set the port, configure to run on multiple threads, and run the app
        app.port(port).multithreaded().run(); // This is a blocking call
        std::cout << "Webserver stopped ..." << std::endl;
    }

private:
    unsigned int port;
    crow::SimpleApp app;
    std::thread processingThread;
};

void signalHandler( int signum ) {
    std::cout << "Received signal " << signum << " -> Exiting gracefully." << std::endl;
    exit(signum);
}

int main() {
    std::cout << "PID " << getpid() << std::endl;
    signal(SIGHUP, signalHandler);
    signal(SIGINT, signalHandler);
    std::atomic_bool keepOnRunning = true;

    MyCrowApp myApp(10088);

    std::condition_variable cv;
    std::mutex cv_m;
    std::thread t([&] {
        while (keepOnRunning) {
            std::this_thread::sleep_for(std::chrono::seconds(1));
            cv.notify_one();
        }
    });
    std::unique_lock<std::mutex> lk(cv_m);
    while (keepOnRunning) {
        cv.wait(lk);
        std::cout << "Sign of life ..." << std::endl;
    }
    t.join();

    exit(EXIT_SUCCESS);
}`

The output when sending SIGINT: ./sandbox4 PID 12069 Constructing MyCrowApp instance. Webserver starting ... (2024-06-15 16:59:35) [INFO ] Crow/master server is running at http://0.0.0.0:10088 using 8 threads (2024-06-15 16:59:35) [INFO ] Call app.loglevel(crow::LogLevel::Warning) to hide Info level logs. Sign of life ... Sign of life ... -> Command executed -> kill -SIGINT 12069 ( It wouldn't make a difference if I do CTRL-C ) (2024-06-15 16:59:50) [INFO ] Closing IO service 0x73e4f4001fc0 (2024-06-15 16:59:50) [INFO ] Closing IO service 0x73e4f4001fc8 (2024-06-15 16:59:50) [INFO ] Closing IO service 0x73e4f4001fd0 (2024-06-15 16:59:50) [INFO ] Closing IO service 0x73e4f4001fd8 (2024-06-15 16:59:50) [INFO ] Closing IO service 0x73e4f4001fe0 (2024-06-15 16:59:50) [INFO ] Closing IO service 0x73e4f4001fe8 (2024-06-15 16:59:50) [INFO ] Closing IO service 0x73e4f4001ff0 (2024-06-15 16:59:50) [INFO ] Closing main IO service (0x73e4f4001078) (2024-06-15 16:59:50) [INFO ] Exiting. Webserver stopped ... -> The main program loop just continues ... Sign of life ... Sign of life ... -> Command executed -> kill -9 12069 Killed

The ouput when sending SIGHUP:

./sandbox4 
PID 12094
Constructing MyCrowApp instance.
Webserver starting ...
(2024-06-15 17:04:18) [INFO    ] Crow/master server is running at http://0.0.0.0:10088 using 8 threads
(2024-06-15 17:04:18) [INFO    ] Call `app.loglevel(crow::LogLevel::Warning)` to hide Info level logs.
Sign of life ...
Sign of life ...
Sign of life ...
-> Command executed -> kill -SIGHUP 12094
Received signal 1 -> Exiting gracefully.

Notice there is no output from Crow nor log messages from myApp's destructor.

I experimented with Crow's run_async() method and tried several approaches but nothing works. It is as if Crow does something to the signals that interferes with the other code. I've got 15 classes in it with x instances and everything destructs neatly but not so the Crow class. I don't have a clue anymore. If somebody has an idea do elaborate please.

gittiver commented 1 month ago

Maybe it works if App::signal_add (int signal_number) will be called for SIGINT and SIGHUP? The server has an initially empty set of signals so maybe they need to be added to stop the server.

rootrunner commented 1 month ago

It now looks like void runWebserver() { std::cout << "Webserver starting ..." << std::endl; // Define the endpoint at the root directory CROW_ROUTE(app, "/")([](){ return "Hello world"; }); app.signal_add(SIGINT); app.signal_add(SIGHUP); // Set the port, configure to run on multiple threads, and run the app app.port(port).multithreaded().run(); // This is a blocking call std::cout << "Webserver stopped ..." << std::endl; } But the behavior is the same. If that is what you mean. I couldn't find signal.add() in https://crowcpp.org/master/reference/index.html?

gittiver commented 1 month ago

Its signal_add self_t & | signal_add (int signal_number)

hmm, maybe it does not work.

rootrunner commented 1 month ago

My POC code has some issues but the basic issue remains. I remade it and added a compile time flag. Should you want to try: If you set basicTest to true and then interrupt it with SIGINT or SIGHUP you'll see it exiting nicely.

Like so:

Sign of life from main thread ... Sign of life from workerthread ... Received signal 1 -> Exiting gracefully. Sign of life from workerthread ... Sign of life from main thread ... Sign of life from workerthread ... Workerthread stopped ... Destructed Worker instance. Ending mainthread ...

If you set it to true it'll start crow and then the issue appears. The program is a daemon so it really needs to be able to stop as it should. It's a pity. Crow really looks like it's a perfect fit. I really want to make it work. I don't understand the root cause.

Edit: that formatting is really weird... These are the includes: ostream csignal mutex thread atomic condition_variable crow.h

`#include

include

include // https://crowcpp.org

std::atomic_bool basicTest = true; // Compile time flag. Set to false to test crow.

std::atomic_bool keepOnRunning = true; // Main program loop control.

class Worker { public: Worker() : keepOnWorking(true), workerThreadIsRunning(false) { std::cout << "Constructing Worker instance." << std::endl; processingThread = std::thread(&Worker::doWork, this); // Start webserverThread in a separate thread. }

~Worker() {
    if(basicTest){
        keepOnWorking = false;
    }
    else {
        app.stop();
    }
    if (processingThread.joinable()) {
        processingThread.join();
    }
    std::cout << "Destructed Worker instance." << std::endl;
}

void doWork() {
    workerThreadIsRunning = true;
    std::cout << "Workerthread starting..." << std::endl;

    if(basicTest){
        std::mutex cv_m;
        std::unique_lock<std::mutex> lk(cv_m);
        std::condition_variable cv;
        while (keepOnWorking) {
            cv.wait_for(lk, std::chrono::milliseconds(500), [this] { return !keepOnWorking; });
            std::cout << "Sign of life from workerthread ..." << std::endl;
        }
    }
    else{
        // Define the endpoint at the root directory
        CROW_ROUTE(app, "/")([](){
            return "Hello world";
        });
        // Set the port, configure to run on multiple threads, and run the app
        app.signal_add(SIGINT);
        app.signal_add(SIGHUP);
        app.port(10000).multithreaded().run(); // This is a blocking call
    }
    workerThreadIsRunning = false;
    std::cout << "Workerthread stopped ..." << std::endl;
}

private: crow::SimpleApp app; std::atomic_bool keepOnWorking; std::atomic_bool workerThreadIsRunning; std::thread processingThread; };

void signalHandler(int signum) { std::cout << "Received signal " << signum << " -> Exiting gracefully." << std::endl; keepOnRunning = false; }

int main() { std::cout << "PID " << getpid() << std::endl; signal(SIGHUP, signalHandler); signal(SIGINT, signalHandler);

{
    Worker worker;

    std::mutex cv_m;
    std::condition_variable cv;
    std::unique_lock<std::mutex> lk(cv_m);
    while (keepOnRunning) {
        cv.wait_for(lk, std::chrono::seconds(1), [] { return !keepOnRunning; });
        std::cout << "Sign of life from main thread ..." << std::endl;
    }
}

std::cout << "Ending mainthread ..." << std::endl;
return EXIT_SUCCESS;

} `

rootrunner commented 1 month ago

This version works for SIGHUP but not for SIGINT ->

include

std::atomic_bool basicTest = false; // Compile time flag. Set to false to test crow. std::atomic_bool keepOnRunning = true; // Main program loop control.

class Worker { public: Worker() : keepOnWorking(true), workerThreadIsRunning(false) { std::cout << "Constructing Worker instance." << std::endl; processingThread = std::thread(&Worker::doWork, this); // Start webserverThread in a separate thread. }

~Worker() {
    if (processingThread.joinable()) {
        processingThread.join();
    }
    std::cout << "Destructed Worker instance." << std::endl;
}

void doWork() {
    workerThreadIsRunning = true;
    std::cout << "Workerthread starting..." << std::endl;

    if (basicTest) {
        std::mutex cv_m;
        std::unique_lock<std::mutex> lk(cv_m);
        std::condition_variable cv;
        while (keepOnWorking) {
            cv.wait_for(lk, std::chrono::milliseconds(500), [this] { return !keepOnWorking; });
            std::cout << "Sign of life from workerthread ..." << std::endl;
        }
    } else {
        // Define the endpoint at the root directory
        CROW_ROUTE(app, "/")([]() {
            return "Hello world";
        });

        // Set the port, configure to run on multiple threads, and run the app
        std::thread crowThread([this]() {
            app.port(10000).multithreaded().run(); // This is a blocking call
        });

        // Wait for keepOnRunning to be false
        while (keepOnRunning) {
            std::this_thread::sleep_for(std::chrono::milliseconds(500));
            std::cout << "Sign of life from workerthread ..." << std::endl;
        }

        // Stop Crow and join the Crow thread
        app.stop();
        if (crowThread.joinable()) {
            crowThread.join();
        }
    }

    workerThreadIsRunning = false;
    std::cout << "Workerthread stopped ..." << std::endl;
}

private: crow::SimpleApp app; std::atomic_bool keepOnWorking; std::atomic_bool workerThreadIsRunning; std::thread processingThread; };

void signalHandler(int signum) { std::cout << "Received signal " << signum << " -> Exiting gracefully." << std::endl; keepOnRunning = false; }

int main() { std::cout << "PID " << getpid() << std::endl; signal(SIGHUP, signalHandler); signal(SIGINT, signalHandler);

{
    Worker worker;

    std::mutex cv_m;
    std::condition_variable cv;
    std::unique_lock<std::mutex> lk(cv_m);
    while (keepOnRunning) {
        cv.wait_for(lk, std::chrono::seconds(1), [] { return !keepOnRunning; });
        std::cout << "Sign of life from main thread ..." << std::endl;
    }
}

std::cout << "Ending mainthread ..." << std::endl;
return EXIT_SUCCESS;

}

Sending kill -SIGHUP 16645 stops it as it should ->

PID 16645 Constructing Worker instance. Workerthread starting... (2024-06-16 15:04:00) [INFO ] Crow/master server is running at http://0.0.0.0:10000 using 8 threads (2024-06-16 15:04:00) [INFO ] Call app.loglevel(crow::LogLevel::Warning) to hide Info level logs. Sign of life from workerthread ... Sign of life from main thread ... Sign of life from workerthread ... Sign of life from workerthread ... Received signal 1 -> Exiting gracefully. Sign of life from workerthread ... (2024-06-16 15:04:14) [INFO ] Closing IO service 0x79d1e0001ee0 (2024-06-16 15:04:14) [INFO ] Closing IO service 0x79d1e0001ee8 (2024-06-16 15:04:14) [INFO ] Closing IO service 0x79d1e0001ef0 (2024-06-16 15:04:14) [INFO ] Closing IO service 0x79d1e0001ef8 (2024-06-16 15:04:14) [INFO ] Closing IO service 0x79d1e0001f00 (2024-06-16 15:04:14) [INFO ] Closing IO service 0x79d1e0001f08 (2024-06-16 15:04:14) [INFO ] Closing IO service 0x79d1e0001f10 (2024-06-16 15:04:14) [INFO ] Closing main IO service (0x79d1e0000f98) (2024-06-16 15:04:14) [INFO ] Exiting. Workerthread stopped ... Sign of life from main thread ... Destructed Worker instance. Ending mainthread ...

Sending kill -SIGINT 16666 just immediately stops Crow but the application doesn't see the signal?

PID 16666 Constructing Worker instance. Workerthread starting... (2024-06-16 15:06:01) [INFO ] Crow/master server is running at http://0.0.0.0:10000 using 8 threads (2024-06-16 15:06:01) [INFO ] Call app.loglevel(crow::LogLevel::Warning) to hide Info level logs. Sign of life from workerthread ... Sign of life from main thread ... Sign of life from workerthread ... Sign of life from workerthread ... Sign of life from main thread ... Sign of life from workerthread ... Sign of life from workerthread ... ^C(2024-06-16 15:06:04) [INFO ] Closing IO service 0x7e5e58001ee0 (2024-06-16 15:06:04) [INFO ] Closing IO service 0x7e5e58001ee8 (2024-06-16 15:06:04) [INFO ] Closing IO service 0x7e5e58001ef0 (2024-06-16 15:06:04) [INFO ] Closing IO service 0x7e5e58001ef8 (2024-06-16 15:06:04) [INFO ] Closing IO service 0x7e5e58001f00 (2024-06-16 15:06:04) [INFO ] Closing IO service 0x7e5e58001f08 (2024-06-16 15:06:04) [INFO ] Closing IO service 0x7e5e58001f10 (2024-06-16 15:06:04) [INFO ] Closing main IO service (0x7e5e58000f98) (2024-06-16 15:06:04) [INFO ] Exiting. Sign of life from main thread ... Sign of life from workerthread ... Sign of life from workerthread ... Sign of life from workerthread ... Sign of life from workerthread ...

kill -9 16666 ->

Killed

It seems that Crow is intercepting the SIGINT signal and not allowing it to propagate to the rest of the application?

I tried using using a signal relay mechanism within the application to no avail. The issue stays.

include

std::atomic_bool basicTest = false; // Compile time flag. Set to false to test crow. std::atomic_bool keepOnRunning = true; // Main program loop control.

class Worker { public: Worker() : keepOnWorking(true), workerThreadIsRunning(false) { std::cout << "Constructing Worker instance." << std::endl; processingThread = std::thread(&Worker::doWork, this); // Start webserverThread in a separate thread. }

~Worker() {
    if (processingThread.joinable()) {
        processingThread.join();
    }
    std::cout << "Destructed Worker instance." << std::endl;
}

void doWork() {
    workerThreadIsRunning = true;
    std::cout << "Workerthread starting..." << std::endl;

    if (basicTest) {
        std::mutex cv_m;
        std::unique_lock<std::mutex> lk(cv_m);
        std::condition_variable cv;
        while (keepOnWorking) {
            cv.wait_for(lk, std::chrono::milliseconds(500), [this] { return !keepOnWorking; });
            std::cout << "Sign of life from workerthread ..." << std::endl;
        }
    } else {
        // Define the endpoint at the root directory
        CROW_ROUTE(app, "/")([]() {
            return "Hello world";
        });

        //app.signal_add(SIGINT); // It never works ...
        //app.signal_add(SIGHUP); // If uncommented it doesn't work for SIGHUP

        // Set the port, configure to run on multiple threads, and run the app
        std::thread crowThread([this]() {
            app.port(10000).multithreaded().run(); // This is a blocking call
        });

        // Wait for keepOnRunning to be false
        while (keepOnRunning) {
            std::this_thread::sleep_for(std::chrono::milliseconds(500));
            std::cout << "Sign of life from workerthread ..." << std::endl;
        }

        // Stop Crow and join the Crow thread
        app.stop();
        if (crowThread.joinable()) {
            crowThread.join();
        }
    }

    workerThreadIsRunning = false;
    std::cout << "Workerthread stopped ..." << std::endl;
}

private: crow::SimpleApp app; std::atomic_bool keepOnWorking; std::atomic_bool workerThreadIsRunning; std::thread processingThread; };

void relaySignalHandler(int signum) { std::cout << "Relayed signal " << signum << " -> Exiting gracefully." << std::endl; keepOnRunning = false; }

void signalHandler(int signum) { if (signum == SIGINT) { std::raise(SIGUSR1); } }

int main() { std::cout << "PID " << getpid() << std::endl; signal(SIGHUP, relaySignalHandler); signal(SIGINT, signalHandler); signal(SIGUSR1, relaySignalHandler);

{
    Worker worker;

    std::mutex cv_m;
    std::condition_variable cv;
    std::unique_lock<std::mutex> lk(cv_m);
    while (keepOnRunning) {
        cv.wait_for(lk, std::chrono::seconds(1), [] { return !keepOnRunning; });
        std::cout << "Sign of life from main thread ..." << std::endl;
    }
}

std::cout << "Ending mainthread ..." << std::endl;
return EXIT_SUCCESS;

}

So I basically can never interrupt the program whilst SIGHUP and likely other do work correctly. I think I'm out of options?

rootrunner commented 1 month ago

I came up with a likely workable solution, but I still need to think it through some more, by creating a child process. I think there is something wrong with how crow handles SIGINT. But I can absolutely be completely wrong.
I would rather not have to approach it this way but I can't solve the issue as I don't really understand it. I'm not sure atm if it'll allow me to do everything I want in the real program. I would rather run the webserver in another thread in the child process but If I do that I can't seem to properly terminate it ... unclear what the outcome will be. The poc code looks like this now:

include

include <sys/wait.h>

include

include "crow.h"

// Simulated resources (database and logger) class Database { public: void connect() { std::cout << "Database connected." << std::endl; } };

class Logger { public: void log(const std::string& message) { std::cout << "Logging: " << message << std::endl; } };

std::atomic_bool keepRunning = true; pid_t childPid = 0;

void signalHandler(int signum) { if (signum == SIGINT || signum == SIGHUP) { std::cout << "Received signal " << signum << ". Stopping child process." << std::endl; if (childPid != 0) { kill(childPid, SIGTERM); // Send termination signal to child process } keepRunning = false; } }

void crowServerFunction(Database& db, Logger& logger) { // Example Crow server code crow::SimpleApp app;

CROW_ROUTE(app, "/")
([&db, &logger](){
    // Access database and logger here
    db.connect();
    logger.log("Request handled.");

    return "Hello, World!";
});

std::cout << "Child Process (Crow Server) starting..." << std::endl;

// Run the Crow server
app.port(8080).multithreaded().run();

std::cout << "Child Process (Crow Server) stopped." << std::endl;

}

int main() { signal(SIGINT, signalHandler); signal(SIGHUP, signalHandler);

std::cout << "Parent Process PID: " << getpid() << std::endl;

// Simulated resources
Database db;
Logger logger;

// Fork a child process
childPid = fork();

if (childPid == -1) {
    std::cerr << "Failed to fork child process." << std::endl;
    return EXIT_FAILURE;
} else if (childPid == 0) {
    // Child process (Crow server)
    crowServerFunction(db, logger); // Pass resources to child process
    return EXIT_SUCCESS; // Child process exits after Crow server stops
} else {
    // Parent process
    std::cout << "Parent Process starting..." << std::endl;

    while (keepRunning) {
        // Main application logic
        std::this_thread::sleep_for(std::chrono::seconds(1));
        std::cout << "Sign of life from Parent Process..." << std::endl;
    }

    // Wait for the child process to terminate
    int status;
    waitpid(childPid, &status, 0);

    std::cout << "Parent Process exiting." << std::endl;
}

return EXIT_SUCCESS;

}

and it stops with SIGINT and SIGHUP showing

./sandbox6 Parent Process PID: 18421 Parent Process starting... Child Process (Crow Server) starting... (2024-06-16 17:16:39) [INFO ] Crow/master server is running at http://0.0.0.0:8080 using 8 threads (2024-06-16 17:16:39) [INFO ] Call app.loglevel(crow::LogLevel::Warning) to hide Info level logs. Sign of life from Parent Process... Sign of life from Parent Process... Sign of life from Parent Process... Received signal 1. Stopping child process. (2024-06-16 17:16:52) [INFO ] Closing IO service 0x58121a4237b0 (2024-06-16 17:16:52) [INFO ] Closing IO service 0x58121a4237b8 (2024-06-16 17:16:52) [INFO ] Closing IO service 0x58121a4237c0 (2024-06-16 17:16:52) [INFO ] Closing IO service 0x58121a4237c8 (2024-06-16 17:16:52) [INFO ] Closing IO service 0x58121a4237d0 (2024-06-16 17:16:52) [INFO ] Closing IO service 0x58121a4237d8 (2024-06-16 17:16:52) [INFO ] Closing IO service 0x58121a4237e0 (2024-06-16 17:16:52) [INFO ] Closing main IO service (0x58121a4229f8) (2024-06-16 17:16:52) [INFO ] Exiting. Child Process (Crow Server) stopped. Sign of life from Parent Process... Parent Process exiting.

witcherofthorns commented 1 month ago

Hi @rootrunner, your application architecture is of course your personal business, but I would recommend that you run Crow as the last one in the chain of service launches (databases, cache, etc.), you need to understand that Crow is mainly launched in the main blocking mode flow. I would recommend that you be careful when working in threads and processes, this may lead to undefined behavior of your backend

I want to share my example of launching different services, maybe you will find it useful I have each service class declared in a separate class and is a small wrapper that inside contains the connection logic and connection check. In the future I am going to add an error and reconnection callback for each of them, but even so it looks much simpler and more convenient

In addition, each of these classes correctly finishes its work after calling the destructor, even if, for example, RabbitMQ was launched in a separate thread to work in non-blocking mode, this can guarantee a complete stop of all your running services

int main(int argc, char const *argv[]) {
    MongoDB mongo("mongodb://user:pass@localhost:27017");
    RabbitMQ rabbitmq;
    RabbitMQHandler rabbitmqHandler(rabbitmq);

    Crow::App<Authorization> app;
    route_auth(app);
    route_user(app);

    AuthRepository::Start(mongo);
    UserRepository::Start(mongo, rabbitmq);

    rabbitmq.ConsumeNewThread(                // non-blocking run
        "amqp://user:pass@localhost",
        "exchange", "queue", "route",
        RabbitQueueType::Durable,
        RabbitExchangeType::Direct
    );

    app.port(18080).multithreaded().run();    // blocking run
    return 0;
}

Yes, in fact, I’m making a detached thread for RabbitMQ, but even it can be successfully completed using atomic values or forwarded pointers to the UV driver on which RabbitMQ is running and stop it, the same can be done with any libraries that initially do not allow running asynchronously

Well, I hope this is useful for you

witcherofthorns commented 1 month ago

if you want to make a parallel worker or calculation, the best idea would be to make it separately from the HTTP API server, let's say use RabbitMQ or Kafka on a separate host, this is a more local solution, if you need bare sockets for the worker, you can use ZeroMQ to build your network infrastructure from scratch is quite an interesting activity, but it requires a lot of free time, but as always, it’s up to you to decide ✌️

rootrunner commented 3 weeks ago

@gittiver I see you labeled it as a possible bug. Should you be able to solve it do reply please because I'm running into issues with the child process approach I mentioned last. I'm still trying to solve them though ...

CrowCpp / Crow

Can't stop a running instance of a class that uses CrowCpp #835

include

include

include

include

include

include

include // https://crowcpp.org

include

include

include

include

include

include

include

include

include

include

include

include

include

include

include

include

include

include

include <sys/wait.h>

include

include

include "crow.h"