"Unexpected status line:" each second time uploading a scan to an HTTP server

[l3u]

Hi!

First of all thanks a lot for this outstanding piece of software :-)

Today, I noticed that Binary Eye could forward each scanned code to an HTTP server using a GET query. I tried to implement a minimalist TCP server that would process such a request, to use it as an interface to Binary Eye as an alternative for an hardware barcode scanner.

I'm not 100 % sure if this is a bug in Binary Eye or in the stuff I hacked together, so this is 50 % a bug report and 50 % a call for support ;-)

I uploaded the quite unpolished but overseeable code I wrote to [Edit: That repo has been deleted meanwhile. Cf. below for a working version!]

It does not do much: When a connection comes in, it waits for data to be ready to be read. Then, it extracts the header, until an empty line is detected. Then, it searches for a GET request, extracts the content= part and unescapes the data. Then, an HTTP answer is sent to the connection and the connection is closed. That's all.

However, when I use it with Binary Eye, everything works the first time I scan a code. But when I go back from the data view page and scan the next one, I get Unexpected status line: (without further information) and no connection to my server is established. When I do the same again, the code is passed again. Next time, it fails. And so on: Works – doesn't work – works – doesn't work etc. Closing the app and opening it again also makes it work again – for the first scan. Then, the second one again doesn't work and so on.

I tried to track this down by using a browser and also a telnet session. No problem in both cases.

A respective telnet session would look like this (using the header data sent by Binary Eye, just to be sure):

$ telnet 192.168.178.21 8080
Trying 192.168.178.21...
Connected to 192.168.178.21.
Escape character is '^]'.
GET /?content=Some+fly+content HTTP/1.1
User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; moto g42 Build/T2SES33.73-23-2-2)
Host: 192.168.178.21:8080
Connection: Keep-Alive
Accept-Encoding: gzip

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Content-Length: 10

Read code!
Connection closed by foreign host.

I can do this as often as I want, no problem.

So … what could cause this? Thanks for all help, and of course for fixing this if it's a Binary Eye issue!

[markusfisch]

Hm, using Python's built-in HTTP server for a quick check seems to work fine. If I run:

$ python3 -m http.server 8080

And point Binary Eye at my IP with port 8080, every request works. Now, since this returns a rather useless HTML page, I wrote a quick server in Ruby (because that's probably the shortest code I can think of):

#!/usr/bin/env ruby

require 'socket'

port = 8080
address = "http://127.0.0.1:#{port}/"
server = TCPServer.new port

while socket = server.accept
  request = socket.gets
  _, full_path = request.split(' ')
  _, query = full_path.split('?')
  _, content = full_path.split('=')
  puts content
  socket.puts %{HTTP/1.1 200 OK\r
Content-Type: text/plain\r
\r
#{content}
}
  socket.close
end

This echoes whatever Binary Eye send after ?content=, and every request works as expected 🤔

Now, I'm tempted to install Qt and try your code… 😉

[l3u]

Thanks for the fast reply :-) Beware: The IP address and port is hard-coded, and no check is done if the server actually could start up (yet).

What I also found is that if I reply with my HTTP header immediately and disconnect, I don't get an error message.

But if I wait until I can read the request header, read it out, answer then and disconnect, the works – doesn't work pattern appears. But I do have to read the header of course …

[l3u]

Apparently, providing the content length was too much ;-)

After stripping down the HTTP answer to

QTextStream stream(connection);
stream << "HTTP/1.1 200 OK\r\n"
       << "Content-Type: text/plain\r\n"
       << "\r\n"
       << message.toUtf8() << "\r\n"
       << "\r\n";

it works now. So this was a problem in my code, not in Binary Eye. Thanks for the help to the self-help ;-)

[l3u]

PS: No, there weren't enough barcode scanners. We definitely needed this one. Keep on rocking :-D

[markusfisch]

Apparently, providing the content length was too much ;-)

Actually, it's the trailing \r\n after the content here that was causing the issue. I finally had some time to try it out this morning and it just wouldn't leave me alone 😉

The content should not be terminated with a CR/LF - only the header (and it's fields).

So the server was sending two bytes after the content, which were kept in the network buffer and processed for the next response, which results in the Unexpected status line: exception. Note that this message just ends with : because of the empty line it encountered. Tricky one 😉

Removing the Content-Length header solved the problem because the trailing CR/LF were just read as content.

Also, I would use QHostAddress::Any for the listen call here to make the server work on any machine.

Here are the changes I made:

diff --git a/Scanner.cpp b/Scanner.cpp
index 0ebd963..586e4eb 100644
--- a/Scanner.cpp
+++ b/Scanner.cpp
@@ -8,7 +8,7 @@
 Scanner::Scanner()
 {
     m_server = new QTcpServer(this);
-    m_server->listen(QHostAddress(QStringLiteral("192.168.178.21")), 8080);
+    m_server->listen(QHostAddress::Any, 8080);
     connect(m_server, &QTcpServer::newConnection, this, &Scanner::newConnection);
 }

@@ -80,7 +80,7 @@ void Scanner::parseRequest(QTcpSocket *connection)
     answer.append("Content-Type: text/html; charset=UTF-8\r\n");
     answer.append(QStringLiteral("Content-Length: %1\r\n").arg(message.count()).toUtf8());
     answer.append("\r\n");
-    answer.append(QStringLiteral("%1\r\n").arg(message).toUtf8());
+    answer.append(QStringLiteral("%1").arg(message).toUtf8());

     connection->write(answer);
     connection->disconnectFromHost();

[l3u]

Hey, thanks a lot for the explanation! Obviously, I spent not enough time on researching how HTTP communication works … I now use a QTextStream to write data to the socket (which is much more convenient than decoding all the stuff by hand to byte arrays), and send the content length – the correct one ;-) Well, now, the server should behave correctly, and I also don't get any errors from Binary Eye.

Just FYI: Of course, the startup of the server only uses makeshift code. I'll include this into another project, Muckturnier.org, and use a more advanced way of configuring the addresses. I'll use the SocketAddressWidget I also use for other stuff that takes care of pre-choosing the correct IP address (IPv4 or IPv6, whatever the user configures) etc.

Just to say it again: This really rocks :-D Here, different FLOSS projects act in concert to build cool stuff! And even more: I learned about HTTP low level communication, directly from the guy I blamed for a problem that was only caused by an error in my code, due to lack of knowledge. Thanks again for that insight, and for the support!

[markusfisch]

My pleasure! And I think that's the beauty of Open Source: we have always someone to blame and we're all likely to learn something new from whatever the problem was 😉

So thank you for filing an issue (surely someone will run into the same problem some day) and, of course, for using Binary Eye!

Also, Muckturnier.org looks quite interesting, too! 👍

[l3u]

Oh my, that code had other flaws. That's why people don't mess with C++, one has to take care of everything ;-) It's possible that the header doesn't arrive in one packet, so we have to deal with fragmentation. At least, TCP guarantees that everything arrives finally, and that the packets are in the correct order. Well, I think I could fix it, and I hope it's decently implemented now.

Just to leave this here, in case anybody wants to implement a QTcpServer to whom Binary Eye can talk: My (now working ;-) implementation can be found at the Muckturnier.org repo, esp. WlanCodeScannerServer.cpp and WlanCodeScannerRequest.cpp. Oh my. People know why they don't implement their own HTTP server ;-)

However: Maybe not even a corner-case application?!