website-serve infinite loop

[zwieblum]

I just realized that the "website" module does not work any more on linux. Minimal example:

(website-serve #f 8080)

The code does nothing but to start the local server. It should do nothing but to recat to http requests. But it results in one process occupying one core 100%. That process does not generate any output not does it response to any request.

[mgorges]

Sorry, I can't seem to reproduce this easily; this is from a fresh clone of the current main branch:

mgorges@ECEM01:~/lambdanative$ uname -a
Linux ECEM01 5.xxxxx-generic #xxxxx-Ubuntu SMP xxxxx x86_64 x86_64 x86_64 GNU/Linux
mgorges@ECEM01:~/lambdanative$ git log --pretty=format:'%h' -n 1
d06be02
mgorges@ECEM01:~/lambdanative$ git diff apps/DemoConsole/
diff --git a/apps/DemoConsole/MODULES b/apps/DemoConsole/MODULES
index 668f264..457c15e 100644
--- a/apps/DemoConsole/MODULES
+++ b/apps/DemoConsole/MODULES
@@ -1 +1 @@
-config ln_core 
+config ln_core generalized-arrays website 
diff --git a/apps/DemoConsole/main.scm b/apps/DemoConsole/main.scm
index c0250af..aa55fc8 100644
--- a/apps/DemoConsole/main.scm
+++ b/apps/DemoConsole/main.scm
@@ -38,6 +38,10 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 (display "DemoConsole\n")

+(define db (website->table "example-website"))
+(website-addhook db "/index.html" (lambda (x) "AAA\n"))
+(website-serve db 8888)
+
 (let loop () 
   (with-exception-catcher (lambda (e) 
     (for-each display (list (exception->string e) "\n")) #f) 
mgorges@ECEM01:~/lambdanative$ make; make install
=== using profile PART BC Children's [/home/mgorges/lambdanative/PROFILE]..
=== configured to build DemoConsole version 1.0 for linux on linux in normal mode

==> checking for required tools..
==> checking for required libraries..
==> checking for lambdanative tools..
==> creating libraries needed for DemoConsole..
 => liblambdanative..
 => cleaning up..
 => exploding library liblambdanative..
==> updating embedded files for DemoConsole..
==> updating web assets for DemoConsole..
 => compiling scheme payload..
    /home/mgorges/lambdanative/apps/DemoConsole/main.scm ..
 => done compiling scheme payload
 => generating hook..
 => assembling payload..
 == /home/mgorges/.cache/lambdanative/linux/lib/libpayload.a
==> creating linux loader needed for DemoConsole..
 => processing sounds needed for DemoConsole..
 => compiling application..
 => cleaning up..
=== /home/mgorges/.cache/lambdanative/linux/DemoConsole
==> making package..
 => making generic zip archive /home/mgorges/.cache/lambdanative/packages/DemoConsole-1.0-linux.zip..
=== /home/mgorges/.cache/lambdanative/packages/DemoConsole-1.0-linux.zip
=== using profile PART BC Children's [/home/mgorges/lambdanative/PROFILE]..
=== configured to build DemoConsole version 1.0 for linux on linux in normal mode

==> attempting to install linux application DemoConsole to local desktop..
==> attempting to copy .desktop file to /home/mgorges/.local/share/applications/
cp: cannot create regular file '/home/mgorges/.local/share/applications/': Not a directory
==> Starting application..
DemoConsole
>

[zwieblum]

Here is a complete testproject that shows the problem - on linux, that is, I don't know if it happens on other OS. DemoWebTemplate.zip

[zwieblum]

I dug a bit deeper: The server hangs at the read in modules/website/website.scm line 129 ( https://github.com/part-cw/lambdanative/blob/b9f302a3bdc6479cc1b422de9f78854e88ad2667/modules/website/website.scm#L129 )

Wireshark says:

• GET / is sent to :8080
• :8080 responds with TCP Keeep-alive
• firefox answer with TCP Keeep-alive This TCP Keep-Alive ping-pong repeats ad infinitum. Well, not eactly, after ~ 30 minutes sometimes the read finishes.

Don't know what to do about it.

[mgorges]

Your test project example runs fine when built against libgambit, but I can also make it fail on Gentoo with libgambc. Interestingly, if I comment the line with (website-addsubstitution "[PROC]" (let ((a 0)) (lambda () (set! a (+ a 1)) (number->string a)))) it oftentimes runs and sometimes fails. I originally thought it might have something to do with (glgui-timings-at-10msec!), but that doesn't fully fix it, which makes me suspicious that the problem is with the lack of render-loop calls in your hybrid app.

If I try apps/DemoHybridApp/main.scm and don't give focus to the small loader window, it also doesn't do anything; yet when I give it focus, it renders the website. Applying the same idea to your app, it doesn't render the website, but once I click the black box in the 'DemoWebTemplate' app window the website renders. I can't quite point to where exactly this happens but it is lack of redraw events coming from OpenGL and is a known problem (see https://github.com/part-cw/lambdanative/issues/86#issuecomment-162815588).

[zwieblum]

Found it!

loaders/x11/x11_microgl.c line 253 says: https://github.com/part-cw/lambdanative/blob/b9f302a3bdc6479cc1b422de9f78854e88ad2667/loaders/x11/x11_microgl.c#L253

Now this removes all Expose Events, as nowadays event.xexpose.width and event.xexpose.height are always 0.

Bugfix: Replace the line with this:

expose = 1;

... as an afterthought: if the application is only a webserver without GUI, wouldn't it be nice to remove the opengl window? Maybe a target called console like android? For my usecase the code would run as a small webserver without GUI, so I'd need to figure out how that would work anyway.

[mgorges]

The loaders/x11/x11_microgl.c#L253 code used to be that way since inception until https://github.com/part-cw/lambdanative/commit/b0a16b38709b931a6c6a593ee594f4a57facad5a, and I can't recall why we changed it specifically? - Added: It was part of pull request #417 which we partially implemented; I'd be happy to revert that change but I didn't author the original change.

Nothing requires you to use a web server with a GUI (my earliest example was a ConsoleApp). - It is solely that you used modules/hybridapp/hybridapp.scm#L68 which has the GUI requirement so people can use it to test applications that run in a mobile device - if you look at modules/hybridapp/MODULES it either applies an eventloop approach on mobile platforms or a ln_glgui one everywhere else?

[zwieblum]

The patch is flaged as "avoid some of the needless redraw events" - that might have been true in earlier days, but while they might end in eventloop.scm but do nothing. A redraw is explictly done when EVENT_REDRAW is sent down the line, so no problem there.

@hybridapp: Yes, I missed that part. I was unter the impression that hybridapp works the same on X11 as on iOS/Android - till I realised that the browser part is totally independent from the scheme part.

[mgorges]

Before I undo https://github.com/part-cw/lambdanative/commit/b0a16b38709b931a6c6a593ee594f4a57facad5a could you test and confirm that this fixes your problem completely; I recall from earlier data collection applications we had built that processing stops after a few seconds when the focus is not on the app - this is a Linux-specific problem, as Windows has a timer that triggers it at least every second (https://github.com/part-cw/lambdanative/commit/fdc3626e334c2fa380d1ef0a481b2c5b7732edf8), and similar heartbeat things exist on iOS and Android.

[zwieblum]

At least on my system it works, even when the dummy window is never exposed (= it's hidden behind forground windows all the time) and never get the focus. I don't think the timer is needed on linux - at least on Xorg, things might be different on Wayland. I could add a timer if you find that processing stops after some seconds - alarm(1) will most likely do.