sysbuild does not build a working version

[angrave]

Headline: We need a stable point to start development from.

We're trying to build a new version of sys but we cannot build a working version from source.

For example, vmlinux.bin.bz2 IS NOT the same as the one in production (/sys)!

Is bower really pullling from the correct branch?

Neeblah/Ed - Please try a fresh version and see if you can reproduce this issue.

Also, it seems that grunt serve does not serve a decompressable vmlinux.bin.bz2 file - at least when jor1k is running under a firefox !? Either the data is bad or the js is bad. Strangely, under OSX, when running under Chrome jor1k's javascript is able to decompress the file successfully.

Thanks.

[neelabhg]

I downloaded a fresh copy and tried to reproduce the issues.

• grunt build works fine and builds the application correctly (apart from some fonts, separate issue).
• The file vmlinux.bin.bz2 pulled by bower and served by grunt is identical to the one in production (I diffed the files).
• Bower is indeed pulling the correct branch
• I was able to reproduce the issue (I'm using Windows, BTW) with grunt serve serving a bad vmlinux.bin.bz2 file to Firefox. Chrome, like you said, is not affected. Continue reading to know why the issue happened. I'll make a pull-request with a fix for it now.

TL;DR

Only when using grunt serve and Firefox, vmlinux.bin.bz2 seems corrupted (a giveaway is that the Content-Length header is bigger than normal in this case). After much debugging, I have found the culprit.

grunt serve uses the grunt-contrib-connect plugin, which uses the connect server framework. connect uses "middlewares" to process requests and responses in some kind of pipeline. We explicitly use the static middleware to serve static files. However, to support reloading the browser automatically upon making changes, we have the livereload option enabled. Because of this, connect injects the livereload middleware before the middlewares used by us.

The livereload middleware is causing the issue. Specifically, this middleware injects a script into any HTML body to support making websocket calls to the server. Even though it might not be injecting any script into vmlinux.bin.bz2, the middleware still processes the data (converts to a string, etc) which causes the file to become corrupt. connect-livereload has an ignore list to avoid processing some binary files, but .bz2 is not in the list. It also has a function to check if the request should be processed. In particular, it tests whether the request has an Accept header and whether it contains the string html in it.

When Firefox makes the request for vmlinux.bin.bz2, it sends the following Accept header: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 This header contains the string html in it, which causes connect-livereload to process the request, which then corrupts the file.

This problem doesn't happen in Chrome, because it sends the following Accept header when making a request for vmlinux.bin.bz2: Accept: */* This header does not contain the string html in it, which causes connect-livereload to ignore and not process the request, which then leaves the raw file data untouched.

The temporary fix is to disable livereload (which is what I'll do in the pull-request). A better solution would be to either fix connect-livereload, or use some other tool for livereloading. Yeoman's generator-webapp (which we had used to scaffold the project) has actually replaced grunt-contrib-connect with grunt-browser-sync (probably not for this reason). However, I'm in the process of making some changes to the project structure, which will affect these build tools, so I don't think it is worth integrating grunt-browser-sync or something else into our project yet.