Release tags for lwan

[eirnym]

I want to maintain lwan port for FreeBSD, but is has to use git do download fixed commits to download source and apply patches to.

Could you create tags/releases to make versioning much easer.

[lpereira]

That's great news, @eirnym!

Are you able to run Lwan on FreeBSD? I've tried running on 10.2 and 10.3 and it crashes; I wasn't able to fix it.

There are no released versions of Lwan, but I can push tags every now and then to make your life easier.

[eirnym]

@lpereira It successfully running on my host (FreeBSD-Current).

Which cmake options do you pass to build and which installed ports related to lwan (lua and sqlite3 at least) do you have?

[eirnym]

I use lua5.1 and sqlite3. It builds, works, but some tests has failed.

Also function get_or_create_cache in lua code always drops coredump.

SIGBUS place shown with lldb for testrunner executable:

* thread #17: tid = 101156, 0x00000000004c0012 testrunner`get_or_create_cache(priv=0x0000000801c6e140) + 562 at lwan-lua.c:258, stop reason = step over
    frame #0: 0x00000000004c0012 testrunner`get_or_create_cache(priv=0x0000000801c6e140) + 562 at lwan-lua.c:258
   255              lwan_status_error("Could not create cache");
   256          /* FIXME: This cache instance leaks: store it somewhere and
   257           * free it on module shutdown */
-> 258          pthread_setspecific(priv->cache_key, cache);
   259      }
   260      return cache;
   261  }
(lldb) n
Process 77814 stopped
* thread #17: tid = 101156, 0x00000000004c00c8 testrunner`get_or_create_cache(priv=0x0000000801c6e140) + 744 at lwan-lua.c:260, stop reason = step over
    frame #0: 0x00000000004c00c8 testrunner`get_or_create_cache(priv=0x0000000801c6e140) + 744 at lwan-lua.c:260
   257           * free it on module shutdown */
   258          pthread_setspecific(priv->cache_key, cache);
   259      }
-> 260      return cache;
   261  }
   262 
   263  static void unref_thread(void *data1, void *data2)
(lldb) n
Process 77814 stopped
* thread #17: tid = 101156, 0x00000008016f64a3 libc.so.7`memset + 83, stop reason = signal SIGBUS: hardware error
    frame #0: 0x00000008016f64a3 libc.so.7`memset + 83
libc.so.7`memset:
->  0x8016f64a3 <+83>: retq   
    0x8016f64a4:       nop    
    0x8016f64a5:       nop    
    0x8016f64a6:       nop 

[eirnym]

Simple build script for FreeBSD (without using ports) is something like following:

cmake -DCMAKE_PREFIX_PATH:STRING=/usr/local -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_C_FLAGS:STRING="-I/usr/local/include -L/usr/local/lib"
make

[lpereira]

I just call cmake ~/git/lwan, sometimes passing -DCMAKE_BUILD_TYPE for either release or debug builds. Sometimes I use a newer Clang installed through ports in order to get ASan or UBSan.

All dependencies are installed through ports, so CMake finds them. It builds fine and, although it runs, it crashes after attending a few requests. If you build it in debug mode, you'll see in the logs that some requests will be made with the correct method, but will have a trailing request with an UNKNOWN method. I suspect that's a symptom of the problem.

I spent a few evenings trying to sort out the problem, but decided to set it aside for a while.

By fooling around with dtruss, I suspect my epoll implementation on top of kqueue is to blame here; the main loop in Lwan is too tied to how epoll behaves and when coroutines are created/resumed. And coroutines in C often result in weird, undebuggable behavior, which is exactly what's happening on my box. (It seems this also affects OS X, for which I don't have access to anymore.)

If it's not crashing on 11.0-CURRENT, though, that's good news! I'll try upgrading my 10.3-RELEASE box and see what happens.

[eirnym]

@lpereira my cmake can't find sqlite3 and lua5.1 from /usr/local, so freegeoip fails to build and testrunner fails to run.

I have UNKNOWN method output too, and I also have random freezes on any types of requests.

[eirnym]

I hope that extracting epoll/kqueue methods to other files or using libevent which wraps them will be better.

[lpereira]

So @LampmanYao fixed the UNKNOWN methods, and I just pushed a commit that coalesces kevent events for a single file descriptor into a single epoll event (not doing this in the most efficient way yet, will bother optimizing whenever it's working properly).

This makes Lwan happier, but now instead of aborting due to a failed assertion, it's segfaulting. Not sure if we're going somewhere, but this looks like some progress.

[jirutka]

I’d like to make a package for Alpine Linux, but I have the same problem – I cannot make a package without a version number… Please use semver, it doesn’t matter that the API is not stable yet etc., this is exactly what versions starting with 0 (as a major ver) are for.

[lpereira]

@jirutka Nice! I've tried Lwan on Alpine Linux a few months ago and I wasn't able to make it work with Musl; is this the case now?

In related news, yeah, I think it's time to start tagging Lwan versions. I'll use semver, and start with a 0.1.0 soon. I'll close this issue as soon as the release is properly tagged.

[lpereira]

@eirnym By the way, Lwan is now working on FreeBSD (at least I can test it on my 10.3-RELEASE-p5 box). @koobs even provided a few build bots.

Whenever I have version tags I'll let you all know; just let me know if a FreeBSD port becomes available so I can update the README file accordingly.

[eirnym]

@lpereira It's great news, I will test it on my BSD box and update my port Makefile.

[eirnym]

@lpereira Can I ask you to add release tags once again?

[lpereira]

@eirnym Sure. I'll make a 0.0.1 release this week.

[eirnym]

Zero releases so far

[dyu]

1 week = 3 months in lpereira land (:

[lpereira]

Sorry about that. The past few months have been chaotic. I'm in a pile of stress, and I've been postponing almost anything that's not essential. Things should settle down in the next few weeks.

[eirnym]

@lpereira I hope so

[eirnym]

up

[Dr-Terrible]

Any update? I wish to package LWAN for Gentoo/Linux too.

[lpereira]

Please bear with me a little bit more. I've been making cleanup commits before I tag 0.1. There are still some sharp corners I'm taking care of, and some of the testing infrastructure is now gone (working on getting that replaced): no FreeBSD or OS X machines building it at the moment.

[Dr-Terrible]

@lpereira thank you for the report; meanwhile I have packaged a git snapshot of LWAN for Gentoo/Linux: https://github.com/Dr-Terrible/ineluctable-overlay/tree/master/www-servers/lwan

p.s.: I had to use some workarounds because you make assumptions about paths which are not LHS compliant.

[eirnym]

It haven't yet done so far. Why not to pull the trigger and involve more people? Yes, it's beta, experimental, and may fail.

[eirnym]

No news so far

[lpereira]

Thanks for the ping, @eirnym. I've just released v0.1. I apologize for the delay in getting this done; and I appreciate you sticking around. Enjoy!

(Please let me know of any problems while packaging it for FreeBSD. Feel free to join on #lwan at Freenode.)

[Dr-Terrible]

Thank you, @lpereira. I have packaged v0.1 for Gentoo/Linux too: https://github.com/Dr-Terrible/ineluctable-overlay/tree/master/www-servers/lwan