Image-builder produces stable 2.0 image

[RespawnDespair]

Fix all the known problems in the image builder so we have a stable and usable 1.6 image.

https://github.com/RespawnDespair/wifibroadcast-image-builder/projects/1

[pilotnbr1]

@RespawnDespair it’s looking good! Only a few more todo items left! How is it going? I can probably help do some testing soon.

I was just curious, and I am not second guessing you, why the choice of Raspian for the buildroot? Was it just to make it easier to get buildroot working? I believe Rodizio was hoping for some flavor of Linux. I am just trying to understand the pros and cons.

[RespawnDespair]

It's looking good now, as i said on my repository, i had some difficulties with the OSD. These seem to be fixed now. I'll put a new image up sometime today or tomorrow.

[bortek]

@pilotnbr1 about buildroot, I guess it must be easiest to build on the same platform where you going to use the image thus raspbian. Building on Ubuntu or other Linux probably needs all the to make the build working.

[RespawnDespair]

@pilotnbr1 @bortek I try to avoid the term 'buildroot', this is a separate system which i tried first. 😄

I use Raspbian LITE, and then add all the components we need for the system to work. This leaves some extra's we don't need, but less than when using normal Raspbian.

I choose this route because the entire system depends heavily on some things raspbian-specific.

[pilotnbr1]

That makes sense. Are you using yocto? Just curious.. ;)

And all this is an image that does not support the pi3b+ yet?

[RespawnDespair]

No yocto, that would mean first recreating raspbian and then adding on top. Maybe in the future 😄

And no, i need to create the kernel8 image in the buildscript to support the 64 bit Pi3B+. Which i will do once all bugs are sorted. Also i don't have a 3B+ for testing so i would have to rely on others to do that for me.

[pilotnbr1]

@RespawnDespair Jelle thanks for the effort! Also thanks for making it easy to follow your progress on your repo, very organized!

[Yes21]

@pilotnbr1 : I have already build many times the image using @RespawnDespair script. It works perfectly with Ubuntu 18.04 following the instructions of his github. For me it took 26 minutes, but I have a big pc (Core I7-8700K / 32 Gb ram)

[dposea]

I had one issue with the raspbian lite image. Also, it's hard to restart the build if it fails. That will just take some tweaking. If I can offer a suggestion write a last step completed message to a file, and use a restart flag. If restart, get last step and start at the next one.

[dposea]

I think I need a new image. I am having many problems with the scripts. These may already be fixed so I'm waiting on a new image to test and then I can create issues.

[dposea]

If we don't already have one I can build a Mavlink 2.0 library.

[Yes21]

@ALL

@RespawnDespair is making hard work with his script to make the new 1.6 working. We need help to test/debug the last images.

Many things had been fixed now, but I can't have video/telemetry with CSL300 (rt5572) on the airPi. Could anyone confirm that ? Could you try with other boards ?

[htcohio]

@Yes21 I was unable to get the ground side to boot with respawndespairs latest highly experimental image with pi3b+.

[Yes21]

@ALL @RespawnDespair did make big progresses :

• mavlink2
• rtl8812au
• RPI 3 B+

Also read the new Readme.

He will release a new test image a.s.a.p.

I am building myself the new image ...

[careyer]

Awesome work! Can't wait to test :-)

Von meinem iPhone gesendet

Am 30.10.2018 um 18:51 schrieb Yes21 notifications@github.com:

@ALL @RespawnDespair did make big progresses :

mavlink2 rtl8812au RPI 3 B+ He will release a new test image a.s.a.p.

I am building myself the new image ...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[RespawnDespair]

@Yes21 Please use the staged branch of the image-builder. Havent tested it on a Pi yet. Baking a new image now for testing.

[Yes21]

@RespawnDespair Of course I used :

git clone -b staged https://github.com/RespawnDespair/wifibroadcast-image-builder.git

[RespawnDespair]

Okay, very good, please let me know the results!

[Yes21]

Ok, will tell you. My racing Ubuntu PC is working hard ;-)

[RespawnDespair]

It takes nearly two hours for a full run on my Hyper-V (don't have a dedicated Ubuntu machine yet). If you read the readme you'll know that a full rebuild will hopefully be something of the past 😄

[pilotnbr1]

Nice work! Wish I could try.. I have a brand new 3b+ at home but i am travelling next 3 weeks :/

[RespawnDespair]

I'm getting a Pi3B+ myself so i'm not relying on others to test. By the time you get back we'll be working on 1.7 😄

[Yes21]

Yes, I understand now why you did "cut" the script into small feet. Great idea !! Bravo ! smile

[Yes21]

@RespawnDespair : the work/build.log is also a great thing !!!

[RespawnDespair]

It does how the amount of time each step took nicely. I'll probably have enough to fix in the coming days, but i'm really happy with the way it turned out. Hopefully it will allow me to make fixes easily so we can get to 1.6 in a short while. I'm also working on a test 'document' that will specify all things we did and didn't test with the release of a new image, but that's another discussion. It's still compiling the kernel here, so it'll be a while yet before i can test the image...

[RespawnDespair]

@Yes21 Be aware that i haven't implemented the last step. It does not copy the image to the deploy folder. When it's done you can take the IMAGE.img from the work/04-Wifibroadcast folder.

[Yes21]

I just had a blue screen for "wireshark". I answered "Yes", and then an error message :

Setting capabilities for dumpcap failed
The attempt to use Linux capabilities to grant packet-capturing
privileges to the dumpcap binary failed. Instead, it has had the
set-user-id bit set.

Can't select "Ok". I'm blocked !

I will try to restart from "stages/03-Packages/00-run-chroot.sh" which is the last step from the log.

[RespawnDespair]

Yeah, i always select 'No', still haven't found a way around that stupid screen yet. stage 00, 01 and 02 will already have the skip file, so if you run .build.sh again it will retry from step 3 onward.

[Yes21]

I killed the process, and restart it with :

stages/03-Packages/00-run-chroot.sh

[Yes21]

When it's done you can take the IMAGE.img from the work/04-Wifibroadcast folder.

Good to know.

[RespawnDespair]

no, don't run the scripts directly. Kill the process and just run .build.sh again, it will re-run the entire step 3 and then continue to 4.

[Yes21]

Ok

[Yes21]

I'm now on "stages/04-Wifibroadcast/01-run-chroot.sh".

[Yes21]

The process is now finished ! I got the " work/04-Wifibroadcast/IMAGE.img".

Now I will redo it with 5.2.20 ... , before to test the image.

[RespawnDespair]

OSD doesn't work because of missing symlink to /tmp/osdconfig.txt. Fixing that now

[Yes21]

I've understood the system of SKIP files in the stages folders. One needs to rm them in order to redo a stage ?

[RespawnDespair]

If you remove the stages/04-wifibroadcast/SKIP file, and run build.sh again it will rerun from stage 4. The fix is in the wifibroadcast-osd-orig repo, but stage 4 will get the latest automatically. This last stage takes a little while due to the mavlink compiles, but it will be done a LOT faster than genereating from scratch.

And here the new build system already pays off!

I tested with the new image just as Ground, old 1.6 as Air. Video works and the OSD shows, don't have my Pixhawk connected so i haven't tested telemetry.

But the basics seem okay.

[RespawnDespair]

I'm off to bed in a bit, workday tomorrow. I will hook up the Pixhawk later this week to do some real testing. If anybody can validate the Mavlink 2 library works that would be nice. @Yes21, perhaps once you have an image and validated some stuff you can share it with others. I have no time to upload it anymore...

[Yes21]

@RespawnDespair : Have a good night ! You did a fantastic job, and your new staged script is awesome. Bravo, Bravo !!

I will share tomorrow my latest built image (with OSD working), after having testing it ...

[bortek]

It's amazing to see a great progress. Well done! 👍👍👍

[careyer]

Awesome work. Will be happy to test the MavLink stuff. Just lemme know where to download a ready compiled image ;-)

Von meinem iPhone gesendet

Am 30.10.2018 um 22:17 schrieb Bortek notifications@github.com:

It's amazing to see a great progress. Well done! 👍👍👍

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[RespawnDespair]

@Yes21 I have merged the staged branch to master so you can checkout master again. The basics seem to work great, no need to keep it experimental anymore.

[careyer]

@RespawnDespair : Curently trying to build the image with your latest image builder script on my Kali-Linux VM. =) It's doing it's job... (currently downloading all the prerequisites - which is awfully slow... think this is due to my connection) will report back if the image was successfully created

[RespawnDespair]

@careyer i hope it works, but it's only meant to work on Ubuntu. So IF it works, great, but no promises 😄

[careyer]

I'll just give it a try.... I have no other VM at hand at work =D

[RespawnDespair]

It would be great to confirm it works on other Linux distros as well. But i think there are a lot of Ubuntu specific calls in there. I'm rebuilding as well, from scratch because i made some changes in the kernel stage (which you should have, changes were from last night). So if it completes i will upload it as well. It would be untested, but the image from yesterday worked on my Pi, so this one should work as well.

[Yes21]

I'm back to my office. Will try now yesterday night image, while rebuilding a new one from latest script version.

[careyer]

It is still heavily compiling.... I spotted some error messages referring to QEMU ... otherwise it is still bashing bits and bytes. Does it make sense to grab the console output and write it to a text.file so @RespawnDespair can see what things probably went wrong on a non-Ubuntu System?

[RespawnDespair]

The QEMU errors are normal. And the console output would surely help finding any issues. Although i won't spend too much time on making it working on Linux Distro XXX. My main focus now is making sure it works reliably on Ubuntu and to fix any issues in the image 😄

[careyer]

@RespawnDespair : Sure thing! Script is finished now.... however no /deploy/ folder was generated. I think i might have failed. Trying to extract the txt file with the console output now. BTW: Which version of Ubuntu are you running on? Will try to setup a new VM at home then. Where can I upload the console output?