UDP functionality required

[GoogleCodeExporter]

What steps will reproduce the problem?
1. none

What is the expected output? What do you see instead?
I'd like mongoose to work over UDP.

What version of the product are you using? On what operating system?
HEAD

Please provide any additional information below.
My current requirement is to get mongoose to run over UDP on all supported 
OSes. The UDP protocol has unique firewall-punching capabilities. Adding such 
patches to mongoose would allow people behind firewalls to serve content (files 
and media streams, such as from a camera) reliably to clients beyond the 
firewall. I wonder if the author is interested in these patches. The mechanism 
to achieve this functionality involves placing TCP packets into plain UDP 
packets, there would be minimal changes to mongoose code, the feature would be 
activated by defining a certain macro. Additional code would need to be 
compiled into mongoose, if the feature were enabled. The end result would be 
the user ability to surf the web via udp, making mongoose a one of a kind web 
server.

Original issue reported on code.google.com by janez...@gmail.com on 6 Mar 2012 at 9:33

[GoogleCodeExporter]

I'm pleased to report that mongoose works very well over UDP:

The key lies with the UDT project, which provides a socket reliability layer 
over UDP:

http://udt.sourceforge.net/

You also need to write a hack header file, currently:

#ifndef UDTMONGOOSE_HPP
# define UDTMONGOOSE_HPP

#include "../udt/udt.h"

#define socket(a, b, c) UDT::socket(a, b, c)
#define bind(a, b, c) UDT::bind(a, b, c)
#define listen(a, b) UDT::listen(a, b)
#define connect(a, b, c) UDT::connect(a, b, c)
#define close(a) UDT::close(a)
#define getpeername(a, b, c) UDT::getpeername(a, b, c)
#define getsockname(a, b, c) UDT::getsockname(a, b, c)
#define getsockopt(a, b, c, d, e) UDT::getsockopt(a, b, c, d, e)
#define setsockopt(a, b, c, d, e) UDT::setsockopt(a, b, c, d, e)
#define select(a, b, c, d, e) UDT::select(a, b, c, d, e)

#define fd_set ud_set

#undef FD_CLR
#define FD_CLR(u, uset) UD_CLR(u, uset)
#undef FD_ISSET
#define FD_ISSET(u, uset) UD_ISSET(u, uset)
#undef FD_SET
#define FD_SET(u, uset) UD_SET(u, uset)
#undef FD_ZERO
#define FD_ZERO(uset) UD_ZERO(uset)

#undef SO_REUSEADDR
#define SO_REUSEADDR UDT_REUSEADDR
#undef SO_KEEPALIVE
#define SO_KEEPALIVE UDT_REUSEADDR
#undef SO_LINGER
#define SO_LINGER UDT_LINGER

#endif // UDTMONGOOSE_HPP

include somewhere inside mongoose.c, compile with g++ and voila:

netstat -l                                                                      

Active Internet connections (only servers)                                      

Proto Recv-Q Send-Q Local Address           Foreign Address         State       

tcp        0      0 *:ssh                   *:*                     LISTEN      

tcp        0      0 localhost:ipp           *:*                     LISTEN      

tcp6       0      0 localhost:ipp           [::]:*                  LISTEN      

udp        0      0 *:ipp                   *:*                                 

udp        0      0 *:http-alt              *:*

The process listening on http-alt is mongoose running over UDP.

Original comment by janez...@gmail.com on 6 Mar 2012 at 5:58

[GoogleCodeExporter]

I've forgotten to add some more defines above:

#define send(a, b, c, d) UDT::send(a, b, c, d)
#define recv(a, b, c, d) UDT::recv(a, b, c, d)
#define sendfile(a, b, c, d) UDT::sendfile(a, b, c, d)
#define recvfile(a, b, c, d) UDT::recvfile(a, b, c, d)
#define sendmsg(a, b, c) UDT::sendmsg(a, b, c)
#define recvmsg(a, b, c) UDT::recvmsg(a, b, c)

Original comment by janez...@gmail.com on 6 Mar 2012 at 6:08

[GoogleCodeExporter]

Yet more defines were needed, but now mongoose works over UDP reliably:

#ifndef UDTMONGOOSE_HPP
# define UDTMONGOOSE_HPP

#include "udt/udt.h"

#define socket(a, b, c) UDT::socket(a, b, c)
#define accept(a, b, c) UDT::accept(a, b, (int*)c)
#define bind(a, b, c) UDT::bind(a, b, c)
#define listen(a, b) UDT::listen(a, b)
#define connect(a, b, c) UDT::connect(a, b, c)
#define getpeername(a, b, c) UDT::getpeername(a, b, c)
#define getsockname(a, b, c) UDT::getsockname(a, b, c)
#define getsockopt(a, b, c, d, e) UDT::getsockopt(a, b, c, d, e)
#define setsockopt(a, b, c, d, e) UDT::setsockopt(a, b, c, d, e)
#define select(a, b, c, d, e) UDT::select(a, b, c, d, e)
#define shutdown(a, b) closesocket(a)

#define send(a, b, c, d) UDT::send(a, b, c, d)
#define recv(a, b, c, d) UDT::recv(a, b, c, d)
#define sendfile(a, b, c, d) UDT::sendfile(a, b, c, d)
#define recvfile(a, b, c, d) UDT::recvfile(a, b, c, d)
#define sendmsg(a, b, c) UDT::sendmsg(a, b, c)
#define recvmsg(a, b, c) UDT::recvmsg(a, b, c)

#undef closesocket
#define closesocket(a) UDT::close(a)

#define fd_set ud_set

#undef FD_CLR
#define FD_CLR(a, b) UD_CLR(a, b)
#undef FD_ISSET
#define FD_ISSET(a, b) UD_ISSET(a, b)
#undef FD_SET
#define FD_SET(a, b) UD_SET(a, b)
#undef FD_ZERO
#define FD_ZERO(a) UD_ZERO(a)

#undef SO_REUSEADDR
#define SO_REUSEADDR UDT_REUSEADDR
#undef SO_KEEPALIVE
#define SO_KEEPALIVE UDT_REUSEADDR
#undef SO_LINGER
#define SO_LINGER UDT_LINGER

#endif // UDTMONGOOSE_HPP

Original comment by janez...@gmail.com on 7 Mar 2012 at 10:35

[GoogleCodeExporter]

Here's a small client for testing mongoose running over UDP:

#include <cstdlib>

#include <cstring>

#include <iostream>

#include <stdexcept>

#include <unistd.h>

#include <netdb.h>

#include "../udt/udt.h"

//////////////////////////////////////////////////////////////////////////////
int main(int argc, char* argv[])
{
  struct addrinfo hints = {};
  struct addrinfo* ai_ptr;

  hints.ai_family = AF_UNSPEC;
  hints.ai_socktype = SOCK_STREAM;

  std::cout << "resolving" << std::endl;

  if (getaddrinfo(argv[1], argv[2], &hints, &ai_ptr))
  {
    std::cout << "error resolving" << std::endl;

    return EXIT_FAILURE;
  }
  // else do nothing

  std::cout << "connecting" << std::endl;

  UDTSOCKET sockfd(UDT::socket(ai_ptr->ai_addr->sa_family,
    ai_ptr->ai_socktype, 0));

  if (-1 == UDT::connect(sockfd, ai_ptr->ai_addr, ai_ptr->ai_addrlen))
  {
    freeaddrinfo(ai_ptr);

    std::cout << "error connecting" << std::endl;

    return EXIT_FAILURE;
  }
  // else do nothing

  freeaddrinfo(ai_ptr);

  std::cout << "sending" << std::endl;

  std::string query(std::string("GET ") + argv[3] + " HTTP/1.1\r\n\r\n");

  if (query.size() != UDT::send(sockfd, query.c_str(), query.size(), 0))
  {
    std::cout << "error sending" << std::endl;
  }
  // else do nothing

  std::cout << "receiving" << std::endl;

  ssize_t len(-1);

  char buffer[512];

  while ((len = UDT::recv(sockfd, buffer, sizeof(buffer), 0)) >= 0)
  {
    std::cout << std::string(buffer, len);
  }

  close(sockfd);

  return EXIT_SUCCESS;
}

Original comment by janez...@gmail.com on 7 Mar 2012 at 5:05

[GoogleCodeExporter]

> allow people behind firewalls to serve 
> content (...) reliably to clients 
> beyond the firewall

May I ask how this actually works?
UDT is just a protocol above UDP so UDP
must work in order to use UDT.
However, also for UDP you need to open
a port in the firewall in order to
receive any data - just as for HTTP
where you need to open TCP port 80.

Original comment by bel2...@gmail.com on 14 Mar 2012 at 10:19

[GoogleCodeExporter]

UDT is indeed a protocol above UDP, but the firewall does not know it.
It treats UDP differently than TCP. Here's more:

http://en.wikipedia.org/wiki/Hole_punching
http://en.wikipedia.org/wiki/UDP_hole_punching

But TCP hole punching is also possible, but the technique is different
and less reliable.

Original comment by janez...@gmail.com on 15 Mar 2012 at 4:56

[GoogleCodeExporter]

This technique requires an additional well known server with special software. 
Furthermore you cannot use standard web browsers but will also need special 
client software, maybe different apps. You cannot use a standard server behind 
the firewall either, because it would need to periodically contact the well 
known server to check for clients willing to connect. So you would need to make 
3 pieces of non standard software and maintain a dedicated root server, correct?
Still you cannot bypass all professional firewalls, since they tend to only 
allow an outgoing tcp port 80 connection, and in larger networks only if they 
come from a dedicated company proxy server (VoIP does not work there either). 
It will work for simple firewalls that block all incoming TCP connections and 
allow all outgoing traffic – like a file sharing service for home users.

Original comment by bel2...@gmail.com on 15 Mar 2012 at 8:45

[GoogleCodeExporter]

No technique for firewall bypassing works 100%. Apps don't need to be
special. Just as one can download web pages off FTP servers, for
example, one can write a plug-in for Webkit (or some other html
rendering engine), that provides the HTML to the engine, via a
protocol, other than HTTP (even though the protocol remains HTTP in
this case, only running over UDP, for example, udt://myhost.com). The
need for an external server, that connects a pair of clients through a
firewall remains, though (I wrote one), but there are other techniques
for punching holes, such as:

http://en.wikipedia.org/wiki/NAT_Port_Mapping_Protocol
http://en.wikipedia.org/wiki/Upnp

that don't require the external server. These work for TCP just as
well though. In short, the added bonus to mongoose is not great,
except for the bragging rights of being the only HTTP server that has
the ability to run over UDP, or perhaps if the benchmarks show that
mongoose over UDT transfers files faster than over TCP.

There is, for example, this post referencing an exam question of
whether HTTP can run over UDP, people seem to think it is impossible
(his answer got nixed):

http://forums.devshed.com/c-programming-42/implementing-http-over-udp-561697.htm
l
http://stackoverflow.com/questions/323351/does-http-use-udp

Original comment by janez...@gmail.com on 15 Mar 2012 at 9:09

[GoogleCodeExporter]

Creating a WebKit based application with your own NetworkAccessManager and 
building it for all platforms is exactly what I would call special ;-) ... but 
certainly it can be done. 

As far as I know UPNP requires UDP multicasts, which also do not pass every 
firewall. So for me the solutions with a well known server seems to be more 
reliable.

I do not deny that HTTP can be implemented on top of something that adds 
reliability to UDP (one could implement it on top of smoke signals if they are 
reliable).

In case mongoose acts as the udt server, one has to make it contact the well 
known server periodically. Is the source of the well known server actually 
available?

Original comment by bel2...@gmail.com on 15 Mar 2012 at 11:55

[GoogleCodeExporter]

WebKit is bundled with Qt, so if one is using Qt, building WebKit will
probably be unnecessary. Qt would make for a quick way to surf the web
using UDT: there is even a web browser example bundled with it.

The third-party server I am writing is proprietary and C++, but I
believe there are some free implementations:

http://en.wikipedia.org/wiki/STUN
http://en.wikipedia.org/wiki/Interactive_Connectivity_Establishment
http://nice.freedesktop.org/wiki/

Original comment by janez...@gmail.com on 15 Mar 2012 at 12:17

[GoogleCodeExporter]

Thanks janezz,
I think I'll abstain from adding UDP functionality to mongoose, for the low 
benefit/complexity ratio.

Original comment by valenok on 22 Sep 2012 at 3:01

• Changed state: WontFix