sorin-ionescu commented 10 years ago

In inotify.c, the following code does not seem to work.

  while( scanning )
  {
    if( quitting )
      goto quitting;
    sleep(1);
  }

I get the following output after which it hangs and I have to terminate it manually with pkill -KILL minidlnad.

[2014/01/02 16:51:15] minidlna.c:153: warn: received signal 2, good-bye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye
[2014/01/02 16:51:15] minissdp.c:734: maxdebug: Sending ssdp:byebye

[1]    12265 killed     ./minidlnad

sorin-ionescu commented 10 years ago

It gets stuck after the following line in minidlnad.c.

pthread_join(inotify_thread, NULL);

sorin-ionescu commented 10 years ago

I figured out what the problem is. The following function call in inotify.c blocks.

kevent(global_kqueue_handle, NULL, 0, &ke, 1, NULL)

So, when I try to quit MiniDLNA, it fails to do so because it's still waiting for an event. The only way to quit it cleanly is to generate an event by moving a file into the watched directories.

In order to fix this, kevent will manually have to be triggered from the main thread, perhaps using EVFILT_USER.

stesser commented 10 years ago

Am 03.01.2014 04:20, schrieb Sorin Ionescu:

I figured out what the problem is. The following function call in inotify.c blocks.

kevent(global_kqueue_handle, NULL, 0, &ke, 1, NULL)

So, when I try to quit MiniDLNA, it fails to do so because it's still waiting for an event. The only way to quit it cleanly is to generate an event by moving a file into the watched directories.

In order to fix this, kevent will manually have to be triggered from the main thread, perhaps using |EVFILT_USER|.

The problem is, that the inotify thread checks "quitting" before the signal handler had a chance to set it to 1.

I have locally fixed this by adding "sleep(1)" in the following place:

--- inotify.c~ 2014-01-02 23:48:35.343595045 +0100 +++ inotify.c 2014-01-03 15:04:56.188043417 +0100 @@ -831,6 +831,7 @@ if ( global_kqueue_handle < 0 ) { DPRINTF(E_ERROR, L_INOTIFY, "kqueue() failed: %s\n", strerror(errno));

sleep(1); return 0; }

This 1 second sleep only occurs after a signal has been received, not during normal operation.

Regards, STefan

stesser commented 10 years ago

Am 03.01.2014 15:20, schrieb Stefan Esser:

Am 03.01.2014 04:20, schrieb Sorin Ionescu:

I figured out what the problem is. The following function call in inotify.c blocks.

kevent(global_kqueue_handle, NULL, 0, &ke, 1, NULL)

So, when I try to quit MiniDLNA, it fails to do so because it's still waiting for an event. The only way to quit it cleanly is to generate an event by moving a file into the watched directories.

In order to fix this, kevent will manually have to be triggered from the main thread, perhaps using |EVFILT_USER|.

The problem is, that the inotify thread checks "quitting" before the signal handler had a chance to set it to 1.

I have locally fixed this by adding "sleep(1)" in the following place:

--- inotify.c~ 2014-01-02 23:48:35.343595045 +0100 +++ inotify.c 2014-01-03 15:04:56.188043417 +0100 @@ -831,6 +831,7 @@ if ( global_kqueue_handle < 0 ) { DPRINTF(E_ERROR, L_INOTIFY, "kqueue() failed: %s\n", strerror(errno));

sleep(1); return 0; }

This 1 second sleep only occurs after a signal has been received, not during normal operation.

Sorry for the wrong patch ...

I created it for the purpose of this mail from a clean check-out, but put it into the wrong block.

The sleep(1) has to be added to the if block in the main loop:

    while( !quitting )
    {
            struct kevent ke;
            if ( kevent(global_kqueue_handle, NULL, 0, &ke, 1, NULL)

== -1 ) { DPRINTF(E_WARN, L_INOTIFY, "kevent polling failure: %s\n", strerror(errno)); sleep(1); // **** continue; }

This sleep(1) assures that quitting is set when the sleep ends, and the loop is terminated.

Regards, STefan

sorin-ionescu commented 10 years ago

I fear that sleep can create a race condition. I've been toying with something better, manually signalling using EVFILT_USER from minidlna.c. I'll open a pull request with my fixes.

sorin-ionescu commented 10 years ago

Your repository is different from the patch you submitted. Whitespace is all over the place.

sorin-ionescu commented 10 years ago

Since your repository is different from the patch you have submitted, I have posted the patch bellow. This also fixes #4.

This is a patch to your patch. So first, you have to apply that one.

diff --git 1/configure.ac 2/configure.ac
index f9fcfc0..73e8fb2 100644
--- 1/configure.ac
+++ 2/configure.ac
@@ -455,6 +455,8 @@ AC_CHECK_LIB(vorbisfile, vorbis_comment_query,

 AC_CHECK_HEADERS([arpa/inet.h asm/unistd.h endian.h machine/endian.h fcntl.h libintl.h locale.h netdb.h netinet/in.h stddef.h stdlib.h string.h sys/file.h sys/inotify.h sys/ioctl.h sys/param.h sys/socket.h sys/time.h unistd.h sys/event.h])

+AC_CHECK_FUNCS([getrlimit setrlimit])
+
 AC_CHECK_FUNCS(inotify_init, AC_DEFINE(HAVE_INOTIFY,1,[Whether kernel has inotify support]), [
     AC_MSG_CHECKING([for __NR_inotify_init syscall])
     AC_COMPILE_IFELSE(
diff --git 1/inotify.c 2/inotify.c
index 5e7d806..5e467e1 100644
--- 1/inotify.c
+++ 2/inotify.c
@@ -806,13 +806,9 @@ quitting:

 #else
 void *
-start_kqueue()
+start_kqueue(void* arg)
 {
-  int global_kqueue_handle = -1;
-
-  global_kqueue_handle = kqueue();
-  if ( global_kqueue_handle < 0 )
-    DPRINTF(E_ERROR, L_INOTIFY, "kqueue() failed: %s\n", strerror(errno));
+  int global_kqueue_handle = *((int *)arg);

   while( scanning )
   {
@@ -821,20 +817,48 @@ start_kqueue()
     sleep(1);
   }

+#if defined(HAVE_GETRLIMIT) && defined(HAVE_SETRLIMIT)
+  struct rlimit rl;
+
+  if (!getrlimit(RLIMIT_NOFILE, &rl))
+  {
+    rl.rlim_cur = rl.rlim_max;
+    if (setrlimit(RLIMIT_NOFILE, &rl) != 0)
+    {
+#if defined(__APPLE__) && defined(RLIMIT_NOFILE) && defined(OPEN_MAX)
+      // On Mac OS X, setrlimit(RLIMIT_NOFILE, &rl) fails to set
+      // rlim_cur above OPEN_MAX even if rlim_max > OPEN_MAX.
+      if (rl.rlim_cur > OPEN_MAX)
+      {
+        rl.rlim_cur = OPEN_MAX;
+        setrlimit(RLIMIT_NOFILE, &rl);
+      }
+#endif
+    }
+  }
+#endif
+
  inotify_create_watches(global_kqueue_handle);
  if (setpriority(PRIO_PROCESS, 0, 19) == -1)
    DPRINTF(E_WARN, L_INOTIFY,  "Failed to reduce kqueue thread priority\n");
  sqlite3_release_memory(1<<31);
  av_register_all();

+ struct kevent ke;
+ EV_SET(&ke, 0, EVFILT_USER, EV_ADD, 0, NULL, NULL);
+ if (kevent(global_kqueue_handle, &ke, 1, NULL, 0, NULL) == -1)
+   DPRINTF(E_ERROR, L_INOTIFY, "Failed to register kqueue quitting event\n");
+ EV_SET(&ke, 0, 0, 0, 0, 0, 0);
  while( !quitting )
  {
-   struct kevent ke;
    if ( kevent(global_kqueue_handle, NULL, 0, &ke, 1, NULL) == -1 )
    {
      DPRINTF(E_WARN, L_INOTIFY,  "kevent polling failure: %s\n", strerror(errno));
   }

+   if (ke.filter == EVFILT_USER && ke.ident == 0)
+     continue;
+
    /*DPRINTF(E_DEBUG, L_INOTIFY,  "GOT KEVENT:\n"
      "ident=0x%X, filter=0x%X, flags=0x%X, fflags=0x%X, data=0x%X, udata=0x%X\n", 
      ke.ident, ke.filter, ke.flags, ke.fflags, ke.data, ke.udata);*/
diff --git 1/minidlna.c 2/minidlna.c
index 154ddc1..3838c6b 100644
--- 1/minidlna.c
+++ 2/minidlna.c
@@ -75,6 +75,10 @@
 #include <libintl.h>
 #endif

+#ifdef HAVE_SYS_EVENT_H
+#include <sys/event.h>
+#endif
+
 #include "upnpglobalvars.h"
 #include "sql.h"
 #include "upnphttp.h"
@@ -958,6 +962,9 @@ main(int argc, char **argv)
    struct sockaddr_in tivo_bcast;
    struct timeval lastbeacontime = {0, 0};
 #endif
+#ifdef HAVE_SYS_EVENT_H
+  int global_kqueue_handle = -1;
+#endif

    for (i = 0; i < L_MAX; i++)
        log_level[i] = E_WARN;
@@ -1003,7 +1010,11 @@ main(int argc, char **argv)
     if (!sqlite3_threadsafe() || sqlite3_libversion_number() < 3005001)
       DPRINTF(E_ERROR, L_GENERAL, "SQLite library is not threadsafe!  "
       "Kqueue will be disabled.\n");
-    else if (pthread_create(&inotify_thread, NULL, start_kqueue, NULL) != 0)
+
+    global_kqueue_handle = kqueue();
+    if (global_kqueue_handle < 0)
+      DPRINTF(E_ERROR, L_INOTIFY, "kqueue() failed: %s\n", strerror(errno));
+    if (pthread_create(&inotify_thread, NULL, start_kqueue, (void *)&global_kqueue_handle) != 0)
       DPRINTF(E_FATAL, L_GENERAL, "ERROR: pthread_create() failed for start_kqueue. EXITING\n");
   }    

@@ -1283,8 +1294,15 @@ shutdown:
        close(lan_addr[i].snotify);
    }

-   if (inotify_thread)
+  if (inotify_thread) {
+#ifdef HAVE_SYS_EVENT_H
+    struct kevent quit_event;
+    EV_SET(&quit_event, 0, EVFILT_USER, 0, NOTE_TRIGGER, NULL, NULL);
+    if (kevent(global_kqueue_handle, &quit_event, 1, NULL, 0, NULL ) == -1)
+      DPRINTF(E_ERROR, L_GENERAL, "Failed to trigger quit event: %s\n", strerror(errno));
+#endif
        pthread_join(inotify_thread, NULL);
+  }

    sql_exec(db, "UPDATE SETTINGS set VALUE = '%u' where KEY = 'UPDATE_ID'", updateID);
    sqlite3_close(db);

sorin-ionescu commented 10 years ago

I have put both patches, yours and mine, in a gist.

stesser commented 10 years ago

Am 23.01.2014 02:43, schrieb Sorin Ionescu:

Since your repository is different from the patch you have submitted http://sourceforge.net/p/minidlna/patches/106/, I have posted the patch bellow. This also fixes #4

Thank you for sending the patch, I'll give it a try.

Sorry, I do not know how to work with GIT (just RCS/CVS/SVN) and have based my changes on the sources used by the FreeBSD port of minidlna.

Based on the latest FreeBSD port version, I have modified minidlna to use poll() instead of select(). The reason is the kqueue() code in the notify handler, which opens a file descriptor per watched directory (some 4000 in my case). Since that number exceeds the default limit of select(), I had to define FD_SETSIZE (I used 8192 to allow for some growth). But select() is inefficient with that number of file descriptors, poll() is a much better choice.

I attach my (working but not cleaned up) version of the poll patch to this mail. Let me know if you want me to explain the implementation. A few remarks:

I used a secondary array to hold information on the polled file descriptors. This array replaces all but one of the linked lists of the select() case. There are helper functions that add or remove file descriptors from the poll array, or change the events to watch for. These have been added to upnpevents.h, just because that header is included in the relevant C files, but the definitions should be moved to a better place (that's part of the missing cleanup).

And the large patch of upnphttp.c is mostly a white-space change (deeper indentation level of most of a large function).

The changes are quite localized, in fact. Instead of FD_SET the code maintains the poll array. Instead of testing FD_ISSET it checks the revents field of each file descriptor in the poll array. The secondary array (pollinfo[]) keeps information on the type of file descriptor and a pointer to the object that holds it's state (this pointer replaces the linked lists). Some further code simplification is possible, but I tried to keep as much of the previous code as possible, in order to simplify testing and verification nof the patch.

The patch also adds a timeout needed for LG TVs, which open new HTTP sockets (even when watching TV) and tend to leave them connected but idle until they timeout (after hours). This seems to overload the TV's processor, leading to teared display frames and tens of seconds of latency for remote control actions (sometimes more than a minute). I had put debugging code into minidlna and found that the TV opens hundreds of unused HTTP sockets. I added code to close any HTTP socket that has not received any data after 5 seconds (for production code, that might be a tuneable in the config file). This does not affect normal operation of any of my devices, since only the LG TV opens such connections without ever using them.

I have not seperated the connecting socket timeout from the poll patch, but plan to do so while performing a cleanup of the patch.

Please let me know, what you think about this patch and whether you think it might become part of the minidlna distribution.

Best regards, STefan --- /usr/ports/net/minidlna/work.nopoll/minidlna-1.1.1/minidlna.c 2014-01-03 23:42:34.987346000 +0100 +++ ./minidlna.c 2014-01-19 13:47:33.954480041 +0100 @@ -68,6 +68,7 @@

include

+#include

include "config.h"

@@ -100,6 +101,11 @@

define sqlite3_threadsafe() 0

endif

+static struct pollfd pollfds; +static struct pollinfo *pollinfo; +static int pollalloc; +static int pollused; + / OpenAndConfHTTPSocket() :

setup the socket used to handle incoming HTTP connections. */ static int @@ -936,22 +942,148 @@ return 0; }

+#ifdef DEBUG +static char * +pollfdtypename(int pollfdtype) +{

//DPRINTF(E_DEBUG, L_GENERAL, "pollfdtypename(%d)\n", pollfdtype);
switch(pollfdtype) {
case POLLTYPE_SSDP:
return "SSDP";
case POLLTYPE_HTTPL:
return "HTTP(listen)";
case POLLTYPE_BEACON:
return "BEACON";
case POLLTYPE_MONITOR:
return "MONITOR";
case POLLTYPE_HTTP:
return "HTTP";
case POLLTYPE_UPNP:
return "UPNP";
}
return "undef"; +} +#endif + +void +setpollevents(int fd, int pollevents, int pollfdtype, void *pollobj) +{
int i;
for (i = 0; i < pollused; i++)
{
if (pollfds[i].fd == fd)
{ +#ifdef DEBUG
if ((pollinfo[i].pollfdtype != pollfdtype) || (pollinfo[i].pollobj != pollobj))
{
DPRINTF(E_FATAL, L_GENERAL, "setpollevents(%d): conflicting entry at index=%d, mode=%d/%d, fdtype=%s/%s, obj=%p/%p\n",
fd, i, pollfds[i].events, pollevents,
pollfdtypename(pollinfo[i].pollfdtype), pollfdtypename(pollfdtype),
pollinfo[i].pollobj, pollobj);
} +#endif
pollfds[i].events = pollevents;
return;
}
}
DPRINTF(E_FATAL, L_GENERAL, "setpollevents(%d): entry not found. EXITING\n", fd); +} + +void +addpollfd(int fd, int events, int pollfdtype, void *pollobj) +{ +#ifdef DEBUG
int i;
for (i = 0; i < pollused; i++)
{
if (pollfds[i].fd == fd)
{
if ((pollfds[i].events != events) || (pollinfo[i].pollfdtype != pollfdtype) || (pollinfo[i].pollobj != pollobj))
DPRINTF(E_FATAL, L_GENERAL, "addpollfd(%d): conflicting entry at index=%d, events=%d/%d, fdtype=%s/%s, obj=%p/%p\n",
fd, i, pollfds[i].events, events,
pollfdtypename(pollinfo[i].pollfdtype), pollfdtypename(pollfdtype),
pollinfo[i].pollobj, pollobj);
DPRINTF(E_FATAL, L_GENERAL, "addpollfd(%d): already exists at index=%d, events=%d\n", fd, i, events);
return;
}
} +#endif
if (pollused == pollalloc)
{
pollalloc += 8;
pollfds = realloc(pollfds, pollalloc * sizeof(*pollfds));
pollinfo = realloc(pollinfo, pollalloc * sizeof(*pollinfo));
if (pollfds == NULL || pollinfo == NULL)
DPRINTF(E_FATAL, L_GENERAL, "Failed to allocate %d poll structures. EXITING\n",
pollalloc);
} +#ifdef DEBUG
DPRINTF(E_MAXDEBUG, L_GENERAL, "addpollfd(%d, %d, %s, %p) index=%d\n",
fd, events, pollfdtypename(pollfdtype), pollobj, pollused); +#endif
pollfds[pollused].fd = fd;
pollfds[pollused].events = events;
pollfds[pollused].revents = 0;
pollinfo[pollused].pollfdtype = pollfdtype;
pollinfo[pollused].pollobj = pollobj;
pollused++; +} + +void +delpollfd(int fd) +{
int i;
for (i = pollused -1; i >= 0; i--)
{
if (pollfds[i].fd == fd)
{
DPRINTF(E_MAXDEBUG, L_GENERAL, "delpollfd(%d): index=%d of %d entries\n", fd, i, pollused);
pollused--;
if (i != pollused)
{
pollfds[i] = pollfds[pollused];
pollinfo[i] = pollinfo[pollused];
}
return;
}
}
DPRINTF(E_ERROR, L_GENERAL, "delpollfd(%d): poll structure not found in %d entries!\n", fd, pollused); +} + +void +CloseAllSocketsExcept(int keep) +{
int i; +#ifdef DEBUG
int pid = getpid(); +#endif
for (i = pollused -1; i >= 0; i--)
{
int fd = pollfds[i].fd;
if (fd >= 0 && fd != keep)
{ +#ifdef DEBUG
DPRINTF(E_DEBUG, L_GENERAL, "CloaseAllSocketsExcept(%d): close fd %d (pid=%d)\n", keep, fd, pid); +#endif
delpollfd(fd);
close(fd);
}
} +} + /* === main === / / process HTTP or SSDP requests _/ int main(int argc, char *_argv) {
time_t now; int ret, i; int sudp = -1, shttpl = -1; int smonitor = -1;
LIST_HEAD(httplisthead, upnphttp) upnphttphead;
struct upnphttp * e = 0;
struct upnphttp * next;
fd_set readset; /* for select() */
fd_set writeset;
int nhttp; struct timeval timeout, timeofday, lastnotifytime = {0, 0}; time_t lastupdatetime = 0;
int max_fd = -1;
int polltimeout; int last_changecnt = 0; pid_t scanner_pid = 0; pthread_t inotify_thread = 0; @@ -962,6 +1094,7 @@ struct timeval lastbeacontime = {0, 0};
endif
now = time(NULL); for (i = 0; i < L_MAX; i++) log_level[i] = E_WARN;
ifdef ENABLE_NLS

@@ -981,8 +1114,6 @@ DPRINTF(E_WARN, L_GENERAL, "SQLite library is old. Please use version 3.5.1 or newer.\n"); }

- LIST_INIT(&upnphttphead);

ret = open_db(NULL);
if (ret == 0)
{

@@ -1048,6 +1179,13 @@

SendSSDPGoodbyes();

addpollfd(sudp, POLLIN, POLLTYPE_SSDP, NULL);
if(smonitor >= 0)
addpollfd(smonitor, POLLIN, POLLTYPE_MONITOR, NULL);
addpollfd(shttpl, POLLIN, POLLTYPE_HTTPL, NULL); +#ifdef TIVO_SUPPORT
addpollfd(sbeacon, POLLIN, POLLTYPE_BEACON, NULL); +#endif /* main loop */ while (!quitting) { @@ -1118,155 +1256,137 @@ updateID++; } }
/* select open sockets (SSDP, HTTP listen, and all HTTP soap sockets) */

- FD_ZERO(&readset);

if (sudp >= 0)
{
FD_SET(sudp, &readset);
max_fd = MAX(max_fd, sudp);

- }

if (shttpl >= 0)
{
FD_SET(shttpl, &readset);
max_fd = MAX(max_fd, shttpl);
} -#ifdef TIVO_SUPPORT
if (sbeacon >= 0)
{
FD_SET(sbeacon, &readset);
max_fd = MAX(max_fd, sbeacon);
} -#endif
if (smonitor >= 0)
{
FD_SET(smonitor, &readset);
max_fd = MAX(max_fd, smonitor);

- }

i = 0; /* active HTTP connections count */
for (e = upnphttphead.lh_first; e != NULL; e = e->entries.le_next)
for (i = 0; i <pollused; i++)
if (pollinfo[i].pollfdtype == POLLTYPE_UPNP)
upnpevents_setupobj(pollinfo[i].pollobj);
polltimeout = timeout.tv_sec * 1000 + timeout.tv_usec / 1000;
DPRINTF(E_DEBUG, L_GENERAL, "main: poll(%d fds, %dms)\n", pollused, polltimeout);
ret = poll(pollfds, pollused, polltimeout);
now = time(NULL);
if (ret < 0) {
if ((e->socket >= 0) && (e->state <= 2))
if(quitting) goto shutdown;
if(errno == EINTR) {
FD_SET(e->socket, &readset);
max_fd = MAX(max_fd, e->socket);
i++;
DPRINTF(E_DEBUG, L_GENERAL, "poll() interrupted\n", i);
continue; }
DPRINTF(E_ERROR, L_GENERAL, "poll(all): %s\n", strerror(errno));
DPRINTF(E_FATAL, L_GENERAL, "Failed to poll open sockets. EXITING\n"); }
nhttp = 0;
for (i = pollused - 1; i >= 0; i--)
{
int fd = pollfds[i].fd;
int revents = pollfds[i].revents;
int fdtype = pollinfo[i].pollfdtype;
void *obj = pollinfo[i].pollobj;
if (revents != 0)
pollinfo[i].timestamp = now;
ifdef DEBUG
/* for debug */
if (i > 1)
DPRINTF(E_DEBUG, L_GENERAL, "%d active incoming HTTP connections\n", i);
DPRINTF(E_MAXDEBUG, L_GENERAL, "Poll result(%d): socket=%d type=%s events=%d revents=%d\n",
i, fd, pollfdtypename(pollinfo[i].pollfdtype), pollfds[i].events, revents);
endif
FD_ZERO(&writeset);

upnpevents_selectfds(&readset, &writeset, &max_fd);

ret = select(max_fd+1, &readset, &writeset, 0, &timeout);

if (ret < 0)
{
if(quitting) goto shutdown;
if(errno == EINTR) continue;
DPRINTF(E_ERROR, L_GENERAL, "select(all): %s\n", strerror(errno));
DPRINTF(E_FATAL, L_GENERAL, "Failed to select open sockets. EXITING\n");
}
upnpevents_processfds(&readset, &writeset);
/* process SSDP packets */
if (sudp >= 0 && FD_ISSET(sudp, &readset))
{
/_DPRINTF(E_DEBUG, LGENERAL, "Received UDP Packet\n");/
ProcessSSDPRequest(sudp, (unsigned short)runtime_vars.port);
}
switch (fdtype) {
case POLLTYPE_SSDP:
if (revents)
ProcessSSDPRequest(fd, (unsigned short)runtime_vars.port);
break;
ifdef TIVO_SUPPORT
if (sbeacon >= 0 && FD_ISSET(sbeacon, &readset))
{
/_DPRINTF(E_DEBUG, LGENERAL, "Received UDP Packet\n");/
ProcessTiVoBeacon(sbeacon);
}
case POLLTYPE_BEACON:
if (revents)
ProcessTiVoBeacon(fd);
break;
endif
if (smonitor >= 0 && FD_ISSET(smonitor, &readset))
{
ProcessMonitorEvent(smonitor);
}
/* increment SystemUpdateID if the content database has changed,
* and if there is an active HTTP connection, at most once every 2 seconds */
if (i && (timeofday.tv_sec >= (lastupdatetime + 2)))
{
if (scanning || sqlite3_total_changes(db) != last_changecnt)
{
updateID++;
last_changecnt = sqlite3_total_changes(db);
upnp_event_var_change_notify(EContentDirectory);
lastupdatetime = timeofday.tv_sec;
}
}
/* process active HTTP connections */
for (e = upnphttphead.lh_first; e != NULL; e = e->entries.le_next)
{
if ((e->socket >= 0) && (e->state <= 2) && (FD_ISSET(e->socket, &readset)))
Process_upnphttp(e);
}
/* process incoming HTTP connections */
if (shttpl >= 0 && FD_ISSET(shttpl, &readset))
{
int shttp;
socklen_t clientnamelen;
struct sockaddr_in clientname;
clientnamelen = sizeof(struct sockaddr_in);
shttp = accept(shttpl, (struct sockaddr *)&clientname, &clientnamelen);
if (shttp<0)
{
DPRINTF(E_ERROR, L_GENERAL, "accept(http): %s\n", strerror(errno));
}
else
{
struct upnphttp * tmp = 0;
DPRINTF(E_DEBUG, L_GENERAL, "HTTP connection from %s:%d\n",
inet_ntoa(clientname.sin_addr),
ntohs(clientname.sin_port) );
/*if (fcntl(shttp, F_SETFL, O_NONBLOCK) < 0) {
DPRINTF(E_ERROR, L_GENERAL, "fcntl F_SETFL, O_NONBLOCK\n");
}*/
/* Create a new upnphttp object and add it to
* the active upnphttp object list */
tmp = New_upnphttp(shttp);
if (tmp)
case POLLTYPE_MONITOR:
if (revents)
ProcessMonitorEvent(fd);
break;
case POLLTYPE_HTTP: {
tmp->clientaddr = clientname.sin_addr;
LIST_INSERT_HEAD(&upnphttphead, tmp, entries);
struct upnphttp * e = obj;
if (revents)
{
if ((e->socket >= 0) && (e->state <= 2))
{
DPRINTF(E_MAXDEBUG, L_GENERAL, "Poll HTTP: fd=%d, state=%d\n", e->socket, e->state);
Process_upnphttp(e);
DPRINTF(E_MAXDEBUG, L_GENERAL, "Poll HTTP: fd=%d, state=%d (AFTER Process_upnphttp)\n", e->socket, e->state);
}
}
else
{
CheckIdle_upnphttp(obj, pollinfo[i].timestamp);
}
if(e->state >= 100)
{
DPRINTF(E_MAXDEBUG, L_GENERAL, "Poll-Loop: Closing HTTP socket(%d) for fd=%d\n", e->socket, fd);
Delete_upnphttp(e);
}
else
nhttp++; }
else
break;
case POLLTYPE_UPNP:
if (revents) {
DPRINTF(E_ERROR, L_GENERAL, "New_upnphttp() failed\n");
close(shttp);
DPRINTF(E_MAXDEBUG, L_GENERAL, "Poll-Loop: upnpevents_processpoll_one: index=%i obj=%p\n", i, obj);
if(fd >= 0)
{
upnpevents_processpoll_one(obj);
}
else
DPRINTF(E_FATAL, L_GENERAL, "upnpevents_processpoll_one: invalid filedescriptor %d, pollin=%d pollout=%d obj=%p. EXITING\n",
fd, (revents & POLLIN) != 0, (revents & POLLOUT) != 0, obj); }
DPRINTF(E_MAXDEBUG, L_GENERAL, "Poll-Loop: upnpevents_process_cleanup_notify_one(%d): obj=%p fd=%d\n", i, obj, fd);
upnpevents_process_cleanup_notify_one(obj);
break;
case POLLTYPE_HTTPL:
/* process new incoming HTTP connections */
if (revents)
OpenSocket_upnphttp(fd);
break;
default:
DPRINTF(E_FATAL, L_GENERAL, "Illegal poll type %d at pollfds index=%d fd=%d obj=%p\n",
fdtype, i, fd, obj);
break; } }
/* delete finished HTTP connections */
for (e = upnphttphead.lh_first; e != NULL; e = next)
/* increment SystemUpdateID if the content database has changed,
* and if there is an active HTTP connection, at most once every 2 seconds */
if (nhttp != 0) {
next = e->entries.le_next;
if(e->state >= 100)
upnpevents_process_cleanup_subscribers();
if (timeofday.tv_sec >= (lastupdatetime + 2)) {
LIST_REMOVE(e, entries);
Delete_upnphttp(e);
if (scanning || sqlite3_total_changes(db) != last_changecnt)
{
updateID++;
last_changecnt = sqlite3_total_changes(db);
upnp_event_var_change_notify(EContentDirectory);
lastupdatetime = timeofday.tv_sec;
DPRINTF(E_DEBUG, L_GENERAL, "Update SQL timestamp(time=%d)\n", timeofday.tv_sec);
```
        }
  }
}
```
}

shutdown:
DPRINTF(E_DEBUG, L_GENERAL, "Shutting down!\n"); /* kill the scanner */ if (scanning && scanner_pid) kill(scanner_pid, 9);

/* close out open sockets */
while (upnphttphead.lh_first != NULL)
/* index must go from high to low! */
for (i = pollused - 1; i >= 0; i--) {
e = upnphttphead.lh_first;
LIST_REMOVE(e, entries);
Delete_upnphttp(e);
if (pollinfo[i].pollfdtype == POLLTYPE_HTTP)
{
Delete_upnphttp(pollinfo[i].pollobj);
```
}
```
} if (sudp >= 0) close(sudp); --- /usr/ports/net/minidlna/work.nopoll/minidlna-1.1.1/upnpevents.c 2014-01-03 23:42:34.987346000 +0100 +++ ./upnpevents.c 2014-01-19 13:40:58.575778780 +0100 @@ -61,6 +61,7 @@

include <arpa/inet.h>

include

include

+#include

include "upnpevents.h"

include "minidlnapath.h"

@@ -136,7 +137,9 @@ tmp->uuid[sizeof(tmp->uuid)-1] = '\0'; snprintf(tmp->uuid+37, 5, "%04lx", random() & 0xffff); }
DPRINTF(E_MAXDEBUG, L_HTTP, "newSubscriber(%p): url=%s callback=%s uuid=%s\n",

tmp, eventurl, tmp->callback, tmp->uuid);

return tmp; }

@@ -165,12 +167,16 @@ renewSubscription(const char * sid, int sidlen, int timeout) { struct subscriber * sub;

DPRINTF(E_MAXDEBUG, L_HTTP, "renewSubscription: timeout=%d uuid=%.*s\n", timeout, sidlen, sid); for(sub = subscriberlist.lh_first; sub != NULL; sub = sub->entries.le_next) { if(memcmp(sid, sub->uuid, 41) == 0) {
DPRINTF(E_MAXDEBUG, L_HTTP, "renewSubscription(%p): timeout=%d uuid=%s fd=%d\n",
sub, timeout, sub->uuid, sub->notify ? sub->notify->s : -1); sub->timeout = (timeout ? time(NULL) + timeout : 0); return 0; } }
DPRINTF(E_MAXDEBUG, L_HTTP, "renewSubscription(?): timeout=%d uuid=%.*s\n", timeout, sidlen, sid); return -1; }

@@ -187,11 +193,13 @@ if(sub->notify) { sub->notify->sub = NULL; }

DPRINTF(E_MAXDEBUG, L_HTTP, "removeSubscriber(%.*s): sub=%p\n", sidlen, sid, sub); LIST_REMOVE(sub, entries); free(sub); return 0; } }
DPRINTF(E_ERROR, L_HTTP, "removeSubscriber(%.*s): FAILED!\n", sidlen, sid); return -1; }

@@ -247,6 +255,8 @@ if(sub) sub->notify = obj; LIST_INSERT_HEAD(&notifylist, obj, entries);

DPRINTF(E_MAXDEBUG, L_HTTP, "upnp_event_create_notify(%p): fd=%d, sub=%p, uuid=%s\n", sub, obj->s, obj->sub, obj->sub->uuid);
addpollfd(obj->s, 0, POLLTYPE_UPNP, obj); return; error: if(obj->s >= 0) @@ -254,6 +264,18 @@ free(obj); }

+/* close the socket connected to the notify object / +static void +upnp_event_notify_close(struct upnp_event_notify \ obj) +{

DPRINTF(E_DEBUG, L_HTTP, "upnp_event_notify_close(%p): closing fd %d\n", obj, obj->s);
if (obj->s >= 0) {
delpollfd(obj->s);
close(obj->s);
obj->s = -1;
} +} + static void upnp_event_notify_connect(struct upnp_event_notify * obj) { @@ -388,6 +410,29 @@ } }

+void +upnpevents_setupobj(void * pollobj) +{

struct upnp_event_notify * obj = pollobj;
switch(obj->state) {
case ECreated:
upnp_event_notify_connect(obj);
if(obj->state != EConnecting)
break;
case EConnecting:
case ESending:
DPRINTF(E_DEBUG, L_HTTP, "upnpevents_setupobj: prepare UPNP socket for output(%d, %p)\n", obj->s, obj);
setpollevents(obj->s, POLLOUT, POLLTYPE_UPNP, obj);
break;
case EWaitingForResponse:
DPRINTF(E_DEBUG, L_HTTP, "upnpevents_setupobj: prepare UPNP socket for input(%d, %p)\n", obj->s, obj);
setpollevents(obj->s, POLLIN, POLLTYPE_UPNP, obj);
break;
default:
break;
} +} + static void upnp_event_process_notify(struct upnp_event_notify * obj) { @@ -404,95 +449,73 @@ upnp_event_recv(obj); break; case EFinished:
close(obj->s);
obj->s = -1;
upnp_event_notify_close(obj); break; default: DPRINTF(E_ERROR, L_HTTP, "upnp_event_process_notify: unknown state\n"); } }

-void upnpevents_selectfds(fd_set readset, fd_set writeset, int * max_fd) +void upnpevents_processpoll_one(void *pollobj) {

struct upnp_event_notify * obj;
for(obj = notifylist.lh_first; obj != NULL; obj = obj->entries.le_next) {
//if (obj->s > FD_SETSIZE)
// DPRINTF(E_FATAL, L_HTTP,
// "upnpevents_selectfds: file descriptor %d too big for select, limit is %d\n",
// obj->s, FD_SETSIZE);
DPRINTF(E_DEBUG, L_HTTP, "upnpevents_selectfds: %p %d %d\n",
obj, obj->state, obj->s);
if(obj->s >= 0) {
switch(obj->state) {
case ECreated:
upnp_event_notify_connect(obj);
if(obj->state != EConnecting)
break;
case EConnecting:
case ESending:
FD_SET(obj->s, writeset);
if(obj->s > *max_fd)
*max_fd = obj->s;
break;
case EWaitingForResponse:
FD_SET(obj->s, readset);
if(obj->s > *max_fd)
*max_fd = obj->s;
break;
default:
break;
}
}
}
struct upnp_event_notify * obj = pollobj;
if(obj->s >= 0)
upnp_event_process_notify(obj);
else
DPRINTF(E_FATAL, L_HTTP, "upnpevents_processpoll_one(%d): obj=%p\n",
obj->s, obj->state, obj); }

-void upnpevents_processfds(fd_set readset, fd_set writeset) +void upnpevents_process_cleanup_notify_one(void *pollobj) {

struct upnp_event_notify * obj;
struct upnp_event_notify * next;
struct subscriber * sub;
struct subscriber * subnext;
time_t curtime;
for(obj = notifylist.lh_first; obj != NULL; obj = obj->entries.le_next) {
DPRINTF(E_DEBUG, L_HTTP, "%s: %p %d %d %d %d\n",
"upnpevents_processfds", obj, obj->state, obj->s,
FD_ISSET(obj->s, readset), FD_ISSET(obj->s, writeset));
struct upnp_event_notify * obj = pollobj;
if(obj->state == EError || obj->state == EFinished) { if(obj->s >= 0) {
if(FD_ISSET(obj->s, readset) || FD_ISSET(obj->s, writeset))
upnp_event_process_notify(obj);
}
}
obj = notifylist.lh_first;
while(obj != NULL) {
next = obj->entries.le_next;
if(obj->state == EError || obj->state == EFinished) {
if(obj->s >= 0) {
close(obj->s);
}
if(obj->sub)
obj->sub->notify = NULL;
DPRINTF(E_DEBUG, L_HTTP, "upnpevents_process_cleanup_one: delpollfd(%d)\n", obj->s);
upnp_event_notify_close(obj);
}
if(obj->sub)
obj->sub->notify = NULL;
if 0 /* Just let it time out instead of explicitly removing the subscriber _/
```
    /_ remove also the subscriber from the list if there was an error */
```
if(obj->state == EError && obj->sub) {
LIST_REMOVE(obj->sub, entries);
free(obj->sub);
}
if(obj->state == EError && obj->sub) {
LIST_REMOVE(obj->sub, entries);
free(obj->sub);
endif
free(obj->buffer);
LIST_REMOVE(obj, entries);
free(obj);
}
obj = next;
DPRINTF(E_DEBUG, L_HTTP, "upnpevents_process_cleanup_one: freeing object %p\n", obj);
free(obj->buffer);
LIST_REMOVE(obj, entries);
free(obj); } +} + +void upnpevents_process_cleanup_subscribers(void) +{
time_t curtime;
struct subscriber * sub;
struct subscriber * subnext; /* remove timeouted subscribers */ curtime = time(NULL);
DPRINTF(E_DEBUG, L_HTTP, "upnpevents_process_cleanup(time=%d subfirst=%p)\n", curtime, subscriberlist.lh_first); +//int i = 0; for(sub = subscriberlist.lh_first; sub != NULL; ) { +//DPRINTF(E_DEBUG, L_HTTP, "upnpevents_process_cleanup(%d): timeout=%d notifyptr=%p\n", i, sub->timeout, sub->notify); subnext = sub->entries.le_next; +// if (sub->notify != NULL) { +// struct upnp_event_notify * obj = sub->notify; +// DPRINTF(E_DEBUG, L_HTTP, "upnpevents_process_cleanup: subscriber (%p) has notify object %p\n", +// sub, obj); +// for ( ;obj != NULL; obj = obj->entries.le_next) { +// DPRINTF(E_DEBUG, L_HTTP, "upnpevents_process_cleanup: subscriber (%p) has notify object %p\n", +// sub, obj); +// } +// } if(sub->timeout && curtime > sub->timeout && sub->notify == NULL) {
```
    DPRINTF(E_DEBUG, L_HTTP, "upnpevents_process_cleanup: delete subscriber (%p) after timeout\n", sub);
  LIST_REMOVE(sub, entries);
  free(sub);
}
sub = subnext;
```
+//i++; }

}

--- /usr/ports/net/minidlna/work.nopoll/minidlna-1.1.1/upnpevents.h 2013-11-02 02:06:41.000000000 +0100 +++ ./upnpevents.h 2014-01-19 13:45:30.596987501 +0100 @@ -66,8 +66,29 @@

int renewSubscription(const char * sid, int sidlen, int timeout);

-void upnpevents_selectfds(fd_set readset, fd_set writeset, int * max_fd); -void upnpevents_processfds(fd_set readset, fd_set writeset); +#include +struct pollinfo {

void *pollobj;
time_t timestamp;
int pollfdtype; +#define POLLTYPE_SSDP 1 +#define POLLTYPE_HTTPL 2 +#define POLLTYPE_BEACON 3 +#define POLLTYPE_MONITOR 4 +#define POLLTYPE_HTTP 5 +#define POLLTYPE_UPNP 6 +}; + +void upnpevents_setupobj(void pollobj); +void upnpevents_processpoll_one(void pollobj); +void upnpevents_process_cleanup_notify_one(void pollobj); +void upnpevents_process_cleanup_subscribers(void); + +void addpollfd(int fd, int mode, int pollfdtype, void pollobj); +void setpollevents(int fd, int events, int pollfdtype, void *pollobj); +void delpollfd(int fd); + +void CloseAllSocketsExcept(int fd);

ifdef USE_MINIUPNPDCTL

void write_events_details(int s); --- /usr/ports/net/minidlna/work.nopoll/minidlna-1.1.1/upnphttp.c 2013-11-02 02:06:41.000000000 +0100 +++ ./upnphttp.c 2014-01-19 13:45:18.701758007 +0100 @@ -94,6 +94,40 @@ E_RENEW };

+void +OpenSocket_upnphttp(int shttpl) +{

int shttp;
socklen_t clientnamelen;
struct sockaddr_in clientname;
clientnamelen = sizeof(struct sockaddr_in);
shttp = accept(shttpl, (struct sockaddr *)&clientname, &clientnamelen);
if (shttp<0)
{
DPRINTF(E_ERROR, L_GENERAL, "accept(http): %s\n", strerror(errno));
}
else
{
struct upnphttp * tmp = 0;
DPRINTF(E_DEBUG, L_GENERAL, "HTTP connection from %s:%d\n",
inet_ntoa(clientname.sin_addr), ntohs(clientname.sin_port) );
/* Create a new upnphttp object and add it to
* the active upnphttp object list */
tmp = New_upnphttp(shttp);
if (tmp)
{
tmp->clientaddr = clientname.sin_addr;
DPRINTF(E_DEBUG, L_GENERAL, "main: addpollfd(%d->%d) (new http socket)\n", shttpl, shttp);
addpollfd(shttp, POLLIN, POLLTYPE_HTTP, tmp);
}
else
{
DPRINTF(E_ERROR, L_GENERAL, "New_upnphttp() failed\n");
close(shttp);
}
} +} + struct upnphttp * New_upnphttp(int s) { @@ -115,11 +149,30 @@ { DPRINTF(E_ERROR, L_HTTP, "CloseSocket_upnphttp: close(%d): %s\n", h->socket, strerror(errno)); }
DPRINTF(E_MAXDEBUG, L_HTTP, "CloseSocket_upnphttp: close(%d) and delete socket\n", h->socket);
delpollfd(h->socket); h->socket = -1; h->state = 100; }

void +CheckIdle_upnphttp(struct upnphttp * h, time_t timestamp) +{
time_t now = time(NULL);
if (now - timestamp > 5) {
if (h->state == 0) {
DPRINTF(E_INFO, L_HTTP, "CheckIdle_upnphttp(%d): Closing idle socket after %d seconds of inactivity\n", h->socket, now - timestamp);
CloseSocket_upnphttp(h);
}
else
{
DPRINTF(E_WARN, L_HTTP, "CheckIdle_upnphttp(%d): Socket in state %d has been idle for %d seconds, waiting for more data\n",
h->socket, h->state, now - timestamp);
}
} +} + +void Delete_upnphttp(struct upnphttp * h) { if(h) @@ -791,8 +844,8 @@ for(i = 0; i<15 && p != '\r'; i++) HttpVer[i] = (p++); HttpVer[i] = '\0';
/*DPRINTF(E_INFO, L_HTTP, "HTTP REQUEST : %s %s (%s)\n",
HttpCommand, HttpUrl, HttpVer);*/
DPRINTF(E_MAXDEBUG, L_HTTP, "HTTP REQUEST (fd=%d) : %s %s (%s)\n",
```
h->socket, HttpCommand, HttpUrl, HttpVer);
```
/* set the interface here initially, in case there is no Host header */ for(i = 0; i<n_lan_addr; i++) @@ -812,6 +865,8 @@ { if( h->req_chunklen ) {
DPRINTF(E_MAXDEBUG, L_HTTP, "HTTP REQUEST: fd=%d buflen=%d chunklen=%d\n",
h->socket, h->req_buflen, h->req_chunklen); h->state = 2; return; } @@ -842,143 +897,147 @@ h->req_command = EPost; ProcessHTTPPOST_upnphttp(h); }
else if((strcmp("GET", HttpCommand) == 0) || (strcmp("HEAD", HttpCommand) == 0))
else {
if( ((strcmp(h->HttpVer, "HTTP/1.1")==0) && !(h->reqflags & FLAG_HOST)) || (h->reqflags & FLAG_INVALID_REQ) )
{
DPRINTF(E_WARN, L_HTTP, "Invalid request, responding ERROR 400. (No Host specified in HTTP headers?)\n");
Send400(h);
return;
}
/* 7.3.33.4 */
else if( (h->reqflags & (FLAG_TIMESEEK|FLAG_PLAYSPEED)) &&
!(h->reqflags & FLAG_RANGE) )
setpollevents(h->socket, POLLOUT, POLLTYPE_HTTP, h);
if((strcmp("GET", HttpCommand) == 0) || (strcmp("HEAD", HttpCommand) == 0)) {
DPRINTF(E_WARN, L_HTTP, "DLNA %s requested, responding ERROR 406\n",
h->reqflags&FLAG_TIMESEEK ? "TimeSeek" : "PlaySpeed");
Send406(h);
return;
}
else if(strcmp("GET", HttpCommand) == 0)
{
h->req_command = EGet;
}
else
{
h->req_command = EHead;
}
if(strcmp(ROOTDESC_PATH, HttpUrl) == 0)
{
/* If it's a Xbox360, we might need a special friendly_name to be recognized */
if( client_types[h->req_client].type == EXbox )
if( ((strcmp(h->HttpVer, "HTTP/1.1")==0) && !(h->reqflags & FLAG_HOST)) || (h->reqflags & FLAG_INVALID_REQ) )
{
DPRINTF(E_WARN, L_HTTP, "Invalid request, responding ERROR 400. (No Host specified in HTTP headers?)\n");
Send400(h);
return;
}
/* 7.3.33.4 */
else if( (h->reqflags & (FLAG_TIMESEEK|FLAG_PLAYSPEED)) &&
!(h->reqflags & FLAG_RANGE) ) {
char model_sav[2];
i = 0;
memcpy(model_sav, modelnumber, 2);
strcpy(modelnumber, "1");
if( !strchr(friendly_name, ':') )
DPRINTF(E_WARN, L_HTTP, "DLNA %s requested, responding ERROR 406\n",
h->reqflags&FLAG_TIMESEEK ? "TimeSeek" : "PlaySpeed");
Send406(h);
return;
}
else if(strcmp("GET", HttpCommand) == 0)
{
h->req_command = EGet;
}
else
{
h->req_command = EHead;
}
if(strcmp(ROOTDESC_PATH, HttpUrl) == 0)
{
/* If it's a Xbox360, we might need a special friendly_name to be recognized */
if( client_types[h->req_client].type == EXbox )
{
char model_sav[2];
i = 0;
memcpy(model_sav, modelnumber, 2);
strcpy(modelnumber, "1");
if( !strchr(friendly_name, ':') )
{
i = strlen(friendly_name);
snprintf(friendly_name+i, FRIENDLYNAME_MAX_LEN-i, ": 1");
}
sendXMLdesc(h, genRootDesc);
if( i )
friendly_name[i] = '\0';
memcpy(modelnumber, model_sav, 2);
}
else if( client_types[h->req_client].flags & FLAG_SAMSUNG_TV ) {
i = strlen(friendly_name);
snprintf(friendly_name+i, FRIENDLYNAME_MAX_LEN-i, ": 1");
sendXMLdesc(h, genRootDescSamsung);
}
else
{
sendXMLdesc(h, genRootDesc); }
sendXMLdesc(h, genRootDesc);
if( i )
friendly_name[i] = '\0';
memcpy(modelnumber, model_sav, 2); }
else if( client_types[h->req_client].flags & FLAG_SAMSUNG_TV )
else if(strcmp(CONTENTDIRECTORY_PATH, HttpUrl) == 0) {
sendXMLdesc(h, genRootDescSamsung);
sendXMLdesc(h, genContentDirectory); }
else
else if(strcmp(CONNECTIONMGR_PATH, HttpUrl) == 0) {
sendXMLdesc(h, genRootDesc);
sendXMLdesc(h, genConnectionManager); }
}
else if(strcmp(CONTENTDIRECTORY_PATH, HttpUrl) == 0)
{
sendXMLdesc(h, genContentDirectory);
}
else if(strcmp(CONNECTIONMGR_PATH, HttpUrl) == 0)
{
sendXMLdesc(h, genConnectionManager);
}
else if(strcmp(X_MS_MEDIARECEIVERREGISTRAR_PATH, HttpUrl) == 0)
{
sendXMLdesc(h, genX_MS_MediaReceiverRegistrar);
}
else if(strncmp(HttpUrl, "/MediaItems/", 12) == 0)
{
SendResp_dlnafile(h, HttpUrl+12);
}
else if(strncmp(HttpUrl, "/Thumbnails/", 12) == 0)
{
SendResp_thumbnail(h, HttpUrl+12);
}
else if(strncmp(HttpUrl, "/AlbumArt/", 10) == 0)
{
SendResp_albumArt(h, HttpUrl+10);
}
ifdef TIVO_SUPPORT
else if(strncmp(HttpUrl, "/TiVoConnect", 12) == 0)
{
if( GETFLAG(TIVO_MASK) )
else if(strcmp(X_MS_MEDIARECEIVERREGISTRAR_PATH, HttpUrl) == 0)
{
sendXMLdesc(h, genX_MS_MediaReceiverRegistrar);
}
else if(strncmp(HttpUrl, "/MediaItems/", 12) == 0)
{
SendResp_dlnafile(h, HttpUrl+12);
}
else if(strncmp(HttpUrl, "/Thumbnails/", 12) == 0)
{
SendResp_thumbnail(h, HttpUrl+12);
}
else if(strncmp(HttpUrl, "/AlbumArt/", 10) == 0)
{
SendResp_albumArt(h, HttpUrl+10);
} +#ifdef TIVO_SUPPORT
else if(strncmp(HttpUrl, "/TiVoConnect", 12) == 0) {
if( *(HttpUrl+12) == '?' )
if( GETFLAG(TIVO_MASK) ) {
ProcessTiVoCommand(h, HttpUrl+13);
if( *(HttpUrl+12) == '?' )
{
ProcessTiVoCommand(h, HttpUrl+13);
}
else
{
DPRINTF(E_WARN, L_HTTP, "Invalid TiVo request! %s\n", HttpUrl+12);
Send404(h);
} } else {
DPRINTF(E_WARN, L_HTTP, "Invalid TiVo request! %s\n", HttpUrl+12);
DPRINTF(E_WARN, L_HTTP, "TiVo request with out TiVo support enabled! %s\n",
HttpUrl+12); Send404(h); } } +#endif
else if(strncmp(HttpUrl, "/Resized/", 9) == 0)
{
SendResp_resizedimg(h, HttpUrl+9);
}
else if(strncmp(HttpUrl, "/icons/", 7) == 0)
{
SendResp_icon(h, HttpUrl+7);
}
else if(strncmp(HttpUrl, "/Captions/", 10) == 0)
{
SendResp_caption(h, HttpUrl+10);
}
else if(strcmp(HttpUrl, "/") == 0)
{
SendResp_presentation(h);
} else {
DPRINTF(E_WARN, L_HTTP, "TiVo request with out TiVo support enabled! %s\n",
HttpUrl+12);
DPRINTF(E_WARN, L_HTTP, "%s not found, responding ERROR 404\n", HttpUrl); Send404(h); } }
endif
else if(strncmp(HttpUrl, "/Resized/", 9) == 0)
else if(strcmp("SUBSCRIBE", HttpCommand) == 0) {
SendResp_resizedimg(h, HttpUrl+9);
h->req_command = ESubscribe;
ProcessHTTPSubscribe_upnphttp(h, HttpUrl); }
else if(strncmp(HttpUrl, "/icons/", 7) == 0)
else if(strcmp("UNSUBSCRIBE", HttpCommand) == 0) {
SendResp_icon(h, HttpUrl+7);
}
else if(strncmp(HttpUrl, "/Captions/", 10) == 0)
{
SendResp_caption(h, HttpUrl+10);
}
else if(strcmp(HttpUrl, "/") == 0)
{
SendResp_presentation(h);
h->req_command = EUnSubscribe;
ProcessHTTPUnSubscribe_upnphttp(h, HttpUrl); } else {
DPRINTF(E_WARN, L_HTTP, "%s not found, responding ERROR 404\n", HttpUrl);
Send404(h);
DPRINTF(E_WARN, L_HTTP, "Unsupported HTTP Command %s\n", HttpCommand);
Send501(h); } }
else if(strcmp("SUBSCRIBE", HttpCommand) == 0)
{
h->req_command = ESubscribe;
ProcessHTTPSubscribe_upnphttp(h, HttpUrl);
}
else if(strcmp("UNSUBSCRIBE", HttpCommand) == 0)
{
h->req_command = EUnSubscribe;
ProcessHTTPUnSubscribe_upnphttp(h, HttpUrl);
}
else
{
DPRINTF(E_WARN, L_HTTP, "Unsupported HTTP Command %s\n", HttpCommand);
Send501(h);
} }

@@ -993,6 +1052,7 @@ { case 0: n = recv(h->socket, buf, 2048, 0); +// DPRINTF(E_DEBUG, L_HTTP, "recv (state0): fd=%d ret=%d\n", h->socket, n); if(n<0) { DPRINTF(E_ERROR, L_HTTP, "recv (state0): %s\n", strerror(errno)); @@ -1000,7 +1060,7 @@ } else if(n==0) {

DPRINTF(E_WARN, L_HTTP, "HTTP Connection closed unexpectedly\n");
DPRINTF(E_INFO, L_HTTP, "HTTP Connection closed unexpectedly (socket %d)\n", h->socket); h->state = 100; } else @@ -1213,6 +1273,8 @@ if( try_sendfile ) { send_size = ( ((end_offset - offset) < MAX_BUFFER_SIZE) ? (end_offset - offset + 1) : MAX_BUFFER_SIZE);
DPRINTF(E_MAXDEBUG, L_HTTP, "send_file(%d): file fd=%d offset=%lld send_size=%lld\n",
h->socket, sendfd, offset, send_size); ret = sys_sendfile(h->socket, sendfd, &offset, send_size); if( ret == -1 ) { @@ -1586,6 +1648,7 @@ CloseSocket_upnphttp(h); goto resized_error; }
CloseAllSocketsExcept(h->socket);
endif

if( h->reqflags & (FLAG_XFERSTREAMING|FLAG_RANGE) ) { @@ -1823,6 +1886,7 @@ CloseSocket_upnphttp(h); goto error; }
CloseAllSocketsExcept(h->socket);

endif

DPRINTF(E_INFO, L_HTTP, "Serving DetailID: %lld [%s]\n", id, last_file.path); --- /usr/ports/net/minidlna/work.nopoll/minidlna-1.1.1/upnphttp.h 2013-11-02 02:06:41.000000000 +0100 +++ ./upnphttp.h 2014-01-19 13:45:57.871991347 +0100 @@ -136,10 +136,18 @@ struct upnphttp * New_upnphttp(int);

+/* OpenSocketupnphttp() / +void +OpenSocketupnphttp(int shttpl); + / CloseSocket_upnphttp() / void CloseSocket_upnphttp(struct upnphttp );

+/* CheckIdleupnphttp() / +void +CheckIdle_upnphttp(struct upnphttp * h, timet timestamp); + / Delete_upnphttp() / void Delete_upnphttp(struct upnphttp ); --- /usr/ports/net/minidlna/work.nopoll/minidlna-1.1.1/upnpsoap.c 2013-11-02 02:06:41.000000000 +0100 +++ ./upnpsoap.c 2014-01-13 21:49:22.798865941 +0100 @@ -104,6 +104,7 @@ h->res_buflen += sizeof(afterbody) - 1;

SendResp_upnphttp(h);

+// DPRINTF(E_MAXDEBUG, L_HTTP, "Send reply (fd=%d): %.*s\n", h->socket, h->res_buflen, h->res_buf); CloseSocket_upnphttp(h); }

sorin-ionescu commented 10 years ago

I have solved the number of open file descriptors problem in my patch to your kqueue patch by requesting the maximum allowed using getrlimit and setrlimit. Unfortunately, kqueue will probably still fail for users with huge libraries. It's possible to increase the limit even further by tuning kernel limits, but that is an operation the user will have to do manually.

However, since I am on Mac OS X, I have access to an alternative API, FSEvents, which does not use file descriptors. It's easy to use, and works well. You can see my FSEvents patch.

stesser commented 10 years ago

Am 23.01.2014 18:17, schrieb Sorin Ionescu:

I have solved the number of open file descriptors problem in my patch to your kqueue patch https://gist.github.com/sorin-ionescu/8571344 by requesting the maximum allowed using |getrlimit| and |setrlimit|.

There are two problems with many watched directories:

1) You are increasing the number of file descriptors allowed, to avoid hitting the resource limit.

2) But with more than 1024 file descriptors used for kqueue(), select() fails for HTTP sockets due to the FD_SETSIZE limit.

These two limits are completely unrelated. The select() limit can be increased by defining FD_SETSIZE to cover sufficient range. But that makes select() very inefficient. The poll changes deal with this inefficiency that is inherent to select() if a process has so many file descriptors to cover (the HTTP sockets will typically have file descriptors above those used for kqueue() ...).

Unfortunately, |kqueue| will probably still fail for users with huge libraries. It's possible to increase the limit even further by tuning kernel limits http://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html, but that is an operation the user will have to do manually.

This is a problem on systems with little RAM, due to auto-sizing of kernel tables. The un-tuned value of openfiles is 225000 on my system (which is unreasonably high as a default), kern.maxfiles is 250000, which is also much higher than I'll ever need.

However, since I am on Mac OS X, I have access to an alternative API, FSEvents https://developer.apple.com/library/Mac/documentation/Darwin/Reference/FSEvents_Ref/Reference/reference.html, which does not use file descriptors. It's easy to use, and works well. You can see my FSEvents patch https://gist.github.com/sorin-ionescu/8571424.

The FSEvents interface is much higher level than kqueue() and it does in fact avoid the high number of file descriptors problem. But since I do not have any MacOS systems, it does not solve my problem (which also affects other FreeBSD users with a high number of media directories - minidlna crashes for them if "inotify=yes").

I haven't had time to look at the patch mentioned in your other mail, yet - will do so as soon as possible.

Regards, STefan

hedger / minidlna-kqueue

MiniDLNA refuses to quit #5

include

include

include

include "config.h"

define sqlite3_threadsafe() 0

endif

endif

ifdef ENABLE_NLS

- LIST_INIT(&upnphttphead);

addpollfd(sbeacon, POLLIN, POLLTYPE_BEACON, NULL); +#endif /* main loop */ while (!quitting) { @@ -1118,155 +1256,137 @@ updateID++; } }

- FD_ZERO(&readset);

- }

- }

ifdef DEBUG

endif

ifdef TIVO_SUPPORT

endif

include <arpa/inet.h>

include

include

include "upnpevents.h"

include "minidlnapath.h"

if 0 /* Just let it time out instead of explicitly removing the subscriber _/

endif

}

ifdef USE_MINIUPNPDCTL

ifdef TIVO_SUPPORT

endif

endif

endif