select_and_execute: select() error 4: Interrupted system call

[Delitants]

Keeps randomly crashing when working as sat ip client. Systemd set to restart it automatically.

untitled text 7.txt

[Unit] Description=Multi-threaded satipserver After=network-online.target

[Service] User=root Group=root

WorkingDirectory=/root/minisatip

Type=forking ExecStart=/usr/local/bin/minisatip -I minisatip -O -s dvbs2:192.168.20.230:554 -y 554 -l general -F /var/log/minisatip.log TimeoutSec=120 Restart=on-failure RestartSec=5 StartLimitInterval=350 StartLimitBurst=20

[Install] WantedBy=multi-user.target

[catalinii]

After crashing, is there something in syslog / dmeag

[Delitants]

After crashing, is there something in syslog / dmeag

Aug 30 16:08:57 neolo-KBN-I kernel: [ 5195.551698] dvb_ca_en50221: dvb_ca adapter 0: DVB CAM detected and initialised successfully Aug 30 16:08:57 neolo-KBN-I systemd[1]: minisatip.service: Deactivated successfully. Aug 30 16:09:19 neolo-KBN-I systemd[1]: Reloading.

That's it. But service is already enabled.

[catalinii]

Can you build with “make debug” and try to repro?

[lars18th]

Hi @Delitants and @catalinii ,

If this trouble is really related to the satipc functionality I need to comment that I've detected a big problem with this module. So I'm working on a fix. Perhaps it will solve (or not) your problem. So I suggest waiting for it to be shared (hopefully within a week).

Regards.

[Jalle19]

It doesn't crash, it exits cleanly due to:

[30/08 14:32:26.045 main]: select_and_execute: select() error 4: Interrupted system call
[30/08 14:32:26.046 main]: The main loop ended, run_loop = 0

No idea why that happens though.

[Delitants]

Hi @Delitants and @catalinii ,

If this trouble is really related to the satipc functionality I need to comment that I've detected a big problem with this module. So I'm working on a fix. Perhaps it will solve (or not) your problem. So I suggest waiting for it to be shared (hopefully within a week).

Regards.

What satipc is responsible for?

It doesn't crash, it exits cleanly due to:

[30/08 14:32:26.045 main]: select_and_execute: select() error 4: Interrupted system call
[30/08 14:32:26.046 main]: The main loop ended, run_loop = 0

No idea why that happens though.

Yes it exits itself right after start, sometimes it takes me 3 attempts to get it running. Happens randomly.

[lars18th]

Hi @Delitants

No idea why that happens though.

Yes it exits itself right after start, sometimes it takes me 3 attempts to get it running. Happens randomly.

I suffer this problem when some memory allocation fails. Please, check buffer sizes and the total ram used in your server. And enable full debug to check it.

@catalinii , regarding to the problem of exhausted memory allocation: you have any idea to improve it? Perhaps an enforced restart? Or printing more debug info?

[catalinii]

This should display allocation info, so if there is any leak it should be visible there.

is this a problem in the latest minor version only?

[lars18th]

Hi @catalinii ,

@catalinii , regarding to the problem of exhausted memory allocation: you have any idea to improve it? Perhaps an enforced restart? Or printing more debug info?

This should display allocation info, so if there is any leak it should be visible there.

More or less the LOG displays a message when a memory allocation fails. However, when using it in a server you don't see the error. I suggest to add some info in the WEB UI reflecting the problem. Perhaps with a simple red text indicating: Memory exhausted!. What you think?

[Delitants]

Server has plenty of free memory, it exits right after start, would it be a leak, it would consume all ram within some prolonged time (hours, days) and then fail. But it fails 5 sec after start. Few attempts to start it eventually starts it ok. It's like starting an old car, where you need to twist a key several times and get lucky.

[lars18th]

Hi,

The relevant info in the LOG is:

[30/08 14:32:26.045 main]: select_and_execute: select() error 4: Interrupted system call
[30/08 14:32:26.046 main]: The main loop ended, run_loop = 0

And it's relevant too that the message RECEIVED SIGNAL ... is not printed (check https://github.com/catalinii/minisatip/blob/d1a668bc5cb2754c01f01e9b24cd4d8deafe6f25/src/utils.c#L280C13-L280C28 )

After a lot of search I can assume that:

1. The program is receiving from the poll loop an EINTR error, that indicates that one INTERRUPT is received.
2. The log is not printing any SIGNAL info, and this indicates that the signal received is SIGINT or SIGTERM.
3. Therefore, something in the server is generating this signal OUTSIDE from the minisatip process.

As a consequence I suggest to @Delitants to execute the minisatip process manually (starting from the command line with the -f parameter inside a tmux or screen session and check if something fails. Perhaps the trouble is generated by the systemd manager. And if this is true, then you need to check your service configuration.

I hope this helps.

[lars18th]

Hi @Delitants ,

Inside your unit description change the type:

[Unit]
Description=Multi-threaded satipserver
After=network-online.target

[Service]
...
#Type=forking
Type=simple
...

I'm running one minisatip instance with this configuration in one system using systemd without troubles. I feel this will fix your problem. Please try it!

[Jalle19]

Remember to add the -f flag if you want to run as a Type=simple systemd service.

[lars18th]

Hi @Jalle19 ,

Remember to add the -f flag if you want to run as a Type=simple systemd service.

Yes. It's mandatory. However, now I have a question: When using systemd to run the service, and in mode Type=simple with the -f flag to run in foreground... how to enable/use the LOG file? Like with -F /var/log/minisatip.log. Any idea?

ANSWER: Using StandardOutput unit option in systemd 240+.

[lars18th]

Hi @Delitants ,

Try this unit configuration:

[Unit]
Description=Multi-threaded satipserver
After=network-online.target

[Service]
User=root
Group=root
#WorkingDirectory=/root/minisatip
Type=simple
ExecStart=/usr/local/bin/minisatip -I minisatip -O -s dvbs2:192.168.20.230:554 -y 554 -l general -f
TimeoutSec=120
Restart=on-failure
RestartSec=5
StartLimitInterval=350
StartLimitBurst=20

StandardOutput=append:/var/log/minisatip.log
StandardError=append:/var/log/minisatip.log

[Install]
WantedBy=multi-user.target

This will be valid only if your systemd is version 240 or newer (check it with systemctl --version).

Regards.

[Delitants]

Hi @Delitants ,

Try this unit configuration:

[Unit]
Description=Multi-threaded satipserver
After=network-online.target

[Service]
User=root
Group=root
#WorkingDirectory=/root/minisatip
Type=simple
ExecStart=/usr/local/bin/minisatip -I minisatip -O -s dvbs2:192.168.20.230:554 -y 554 -l general -f
TimeoutSec=120
Restart=on-failure
RestartSec=5
StartLimitInterval=350
StartLimitBurst=20

StandardOutput=append:/var/log/minisatip.log
StandardError=append:/var/log/minisatip.log

[Install]
WantedBy=multi-user.target

This will be valid only if your systemd is version 240 or newer (check it with systemctl --version).

Regards.

It exits even if I start it with a bare command line, not related to systemd I guess

[Jalle19]

Just in case the problem is logging to a file with -F, can you try removing that?

[lars18th]

Hi @Delitants ,

It exits even if I start it with a bare command line, not related to systemd I guess

It exits even if I start it with a bare command line, not related to systemd I guess

Yes, try to execute it using the command line and removing (one by one) some parameters. Check first the -F command.

[Delitants]

Hi @Delitants ,

It exits even if I start it with a bare command line, not related to systemd I guess

It exits even if I start it with a bare command line, not related to systemd I guess

Yes, try to execute it using the command line and removing (one by one) some parameters. Check first the -F command.

With -f in systemd file, it fails to start properly systemctl restart minisatip hangs up....

UDP: Oh, changed forking to simple, now it starts ok. Watching. Log output works.

[lars18th]

UDP: Oh, changed forking to simple, now it starts ok. Watching. Log output works.

Therefore, the problem is related to the systemd. I'm happy you have fixed the trouble. You want (@Delitants ) to close this issue?

[Delitants]

UDP: Oh, changed forking to simple, now it starts ok. Watching. Log output works.

Therefore, the problem is related to the systemd. I'm happy you have fixed the trouble. You want (@Delitants ) to close this issue?

No it’s not related to it, has nothing to do with it. It keeps crashing with or without systemd configured. Moreover, this problem is very old, not appeared just recently. And the fact that it takes few attempts to start, makes me think it has some problem with ram allocation or something.

[lars18th]

Hi @Delitants ,

To be more clear: If your instance of minisatip is restarting with a message like select_and_execute: select() error 4: Interrupted system call then your system is sending a SIGINT or SIGTERM call to the minisatip process. And this is not a bug/error or crash of minisatip. This is an external event. So I suggest to check your environment. Perhaps you can try to execute it in a different machine with a different OS using the command line and check if it runs forever, or it restarts.

You agree?

[catalinii]

Can you try to run:

strace -f -s 200 -o /tmp/x.out ./minisatip -f … Then upload /tmp/x.out