Image-builder produces stable 2.0 image

RespawnDespair commented 5 years ago

Fix all the known problems in the image builder so we have a stable and usable 1.6 image.

https://github.com/RespawnDespair/wifibroadcast-image-builder/projects/1

RespawnDespair commented 5 years ago

No deploy is generated, that still needs to be implemented. If all was succesfull there will be an IMAGE.img under work/04-Wifibroadcast. Try to write this to and SD card and boot.

Ubuntu 18 Xenial is what i use...

RespawnDespair commented 5 years ago

New image Just generated, not tested yet.

careyer commented 5 years ago

Ahh okay! =D... Yes there is a IMAGE.img in that folder..... I will check it then! Here is the output: https://www.ip-networx.de/content/GitHub/wifibroadcast_image_builder.txt

Maybe you can have a short glimpse if that looks okay?

Yes21 commented 5 years ago

@careyer : I'm sure it's Ok for you. You got the IMAGE.img and the script ended exactly like for me on Ubuntu 18.04.

Yes21 commented 5 years ago

I've just tested the image from last night. But no OSD. Perhaps I didn't have you last correction for OSD ? The last image is being built. These testings are needing a lot of patience and time ... ;-)

RespawnDespair commented 5 years ago

I did the correction on the image itself (missing symlink), then added it in the source repo, it is possible i made a mistake. After dinner tonight i will check the image i posted a little while back, that should have the fix. If it doesn't work i'll update the repo and make a new image (takes about 15 minutes running only step 4).

RespawnDespair commented 5 years ago

@careyer Your log look fine, apparently Kali is also able to build the images 😄 Just take the IMAGE.img fro work/04-Wifibroadcast and write it to the SD, i think it should work. The OSD might be broken still, i thought i fixed that, but without a Pi at work i cannot check. The fix for that would be to alt+f12 with a keyboard attached on the pi and then create a symlink like this:

cd /home/pi/wifibroadcast-osd
ln -s /tmp/osdconfig.txt ./osdconfig.h

Yes21 commented 5 years ago

Last build just finished : 60 min with "-j 12" !!

RespawnDespair commented 5 years ago

That's two times faster than for me! (I could put -j 64 in, but i only have 2 virtual cores). Now if somebody is able to verify the OSD is fixed or not...

Yes21 commented 5 years ago

The OSD might be broken still, i thought i fixed that, but without a Pi at work i cannot check. The fix for that would be to alt+f12 with a keyboard attached on the pi and then create a symlink like this:

cd /home/pi/wifibroadcast-osd
ln -s /tmp/osdconfig.txt ./osdconfig.h

This correction doesn't correct OSD not working on the last night image ...

Will now burn my latest build and test it.

RespawnDespair commented 5 years ago

Weird...What does it say on tty2? alt+f2? Any compile errors?

Yes21 commented 5 years ago

Test with my latest built image :

I can see OSD for 1 sec, and then it disappears. No splashscreen, nor "EZ-..." text screen.

alt+f12 : login invite, tried to create the symlink but it already exists.
alt+f2 : "waiting until video is running ...."

On the airPi I can see the led of my AWUS036ACH blinking. It's perhaps working ?

Will now test with CSL300.

Yes21 commented 5 years ago

Nothing new with CSL300.

Yes21 commented 5 years ago

I can add that my AWUS1900 on groundPi is set into monitor mode, but no wifi hotspot with the internal chipset :

eth0      no wireless extensions.

b827eb3fb20d  IEEE 802.11  ESSID:off/any  
          Mode:Managed  Access Point: Not-Associated   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:on

lo        no wireless extensions.

24050fe57cb2  IEEE 802.11  Mode:Monitor  Frequency:5.5 GHz  Tx-Power=26 dBm   
          Retry short  long limit:2   RTS thr:off   Fragment thr:off
          Power Management:off

Yes21 commented 5 years ago

On airPi, my AWUS036ACH is also set up in monitor mode :

eth0      no wireless extensions.

00c0ca953bac  IEEE 802.11  Mode:Monitor  Frequency:5.5 GHz  Tx-Power=18 dBm   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:off

lo        no wireless extensions.

RespawnDespair commented 5 years ago

So it decided it does not have a camera and acts as ground. I will test later this evening. Entertaining the kid now 😁

RespawnDespair commented 5 years ago

Sorry, there's something wrong with the core code now, i merged dino.de's changes, but i must have missed something because it is not starting several programs. No working images until i fix this. Also getting errors regarding the wifi driver... I think i'll roll back some changes untill it starts working again.

htcohio commented 5 years ago

@respawndespair,

Here is user1321's image it has some bug fixes as well as mavlink/pi3b+ and wfb comm/settings.

I'm not sure if this would be helpful as something else to look at or compare to.

Everything appears to work perfectly with the new firmware as well. Chris Dropbox link contains the new firmware, the modified app and image.

https://www.dropbox.com/sh/c62...AABattqiTLfO5Ruazk8lnMpCa?dl=0 https://www.dropbox.com/sh/c629iz8antzc70s/AABattqiTLfO5Ruazk8lnMpCa?dl=0

On Wed, Oct 31, 2018, 3:34 PM Jelle Tigchelaar <notifications@github.com wrote:

Sorry, there's something wrong with the core code now, i merged dino.de's changes, but i must have missed something because it is not starting several programs. No working images until i fix this. Also getting errors regarding the wifi driver... I think i'll roll back some changes untill it starts working again.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bortek/EZ-WifiBroadcast/issues/166#issuecomment-434816375, or mute the thread https://github.com/notifications/unsubscribe-auth/AcSp0t_37kW4ckjterlH9KCfYpHprOMDks5uqftAgaJpZM4XtML8 .

RespawnDespair commented 5 years ago

@htcohio Thanks, i will try to do a diff between this image and mine to see what's up. It would be more helpful if @user1321 could shed some light on the Pi3B+ modifications, looking for minute differences takes a lot of time...

@Yes21 It appears my latest image does not have a modified kernel... Something went wrong during the build, i'm looking into it.

Yes21 commented 5 years ago

@RespawnDespair : Take your time, and go not too late in bed ! You've already did big steps.

RespawnDespair commented 5 years ago

Thanks for your kind words, i can confirm the image i made yesterday DOES have the kernel, so i'm running in a clean install now to find out what went wrong.

@careyer Your build also failed in the kernel stage, 'bc not found'.

I'm hoping to replicate this...

careyer commented 5 years ago

Take your time my friend. Nobody is perfect and it was huge fun for me seeing my crappy work HP laptop sweat like hell under compiling the image in that Kali Linux VM ;-). You are doing a great job man. In can only add to the words of Yes21: Don't go to bed too late ;-)

Von meinem iPhone gesendet

Am 31.10.2018 um 21:18 schrieb Jelle Tigchelaar notifications@github.com:

Thanks for your kind words, i can confirm the image i made yesterday DOES have the kernel, so i'm running in a clean install now to find out what went wrong.

@careyer Your build also failed in the kernel stage, 'bc not found'.

I'm hoping to replicate this...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

RespawnDespair commented 5 years ago

@careyer did you install all the dependencies in the README of the image-builder?

BC appears not to be installed (it says so in the error).

sudo apt-get install bc might fix it, but i think your missing more packages.

Building the kernel takes a loooooooooooooooooooooong time, so if stage 2 finishes quickly, something went wrong. I'll try to throw an exception instead of just continuing.

Still working on tracing the issue, on my home pc it builds fine, but at work it appears to fail the kernel build as well...

careyer commented 5 years ago

Yes I think so... In the console output you should also see me installing the dependencies (right at the beginning ;-)... I did so exactly as the read me says:

apt-get install qemu qemu-user-static binfmt-support build-essential gcc-arm*

Von meinem iPhone gesendet

Am 31.10.2018 um 21:42 schrieb Jelle Tigchelaar notifications@github.com:

@careyer did you install all the dependencies in the README of the image-builder?

BC appears not to be installed (it says so in the error).

sudo apt-get install bc might fix it, but i think your missing more packages.

Building the kernel takes a loooooooooooooooooooooong time, so if stage 2 finishes quickly, something went wrong. I'll try to throw an exception instead of just continuing.

Still working on tracing the issue, on my home pc it builds fine, but at work it appears to fail the kernel build as well...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Yes21 commented 5 years ago

I'm sure that the latest image I've build this afternoon has compiled the good kernel (with 5.2.20 8812au driver), and all wbc modules.

careyer commented 5 years ago

@RespawnDespair : which version of Ubuntu do you run on? I want to setup a new VM for compiling the image. Just to make sure that I reduce possible problems and missing components. - thanks

RespawnDespair commented 5 years ago

@careyer 18.04 is the Ubuntu i use. I think bc is installed by default on Ubuntu...

careyer commented 5 years ago

Alright! 18.04 LTS ... This is exactly what I am currently installing as a new VM! Great! And yes... bc is part of the default software !=)

BTW: @dposea wanted to reach out to you and send you some bits and pieces via email. Maybe you can drop him a message? It would also be good to have your email if you don't mind. BTW2: I hope my explanation about the FLIR thingy was understandable?

RespawnDespair commented 5 years ago

The image-builder is broken on the kernel stage, the 5.5.2 rtl8812au driver package does not compile, this causes the kernel not to be built. I have locally reverted to the first version instead of a specific branche, testing now. So please wait until i fixed this or your builds will be broken...

RespawnDespair commented 5 years ago

If you git pull the latest image builder i believe the kernel build step now works again.

careyer commented 5 years ago

trying it out now.... (my VM setup just finished..... good timing! =)

careyer commented 5 years ago

All CPUs maxxed out!! Looks good so far.... it is compiling the kernel now!

RespawnDespair commented 5 years ago

Yeah, the kernel compile stages eats your CPU yeah 😄 Once stage 2 completes you might want to check if the work/02-Kernel folder contains a kernel1.img and a kernel7.img. That would be a sign the kernel compiled correctly.

Yes21 commented 5 years ago

Once stage 2 completes you might want to check if the work/02-Kernel folder contains a kernel1.img and a kernel7.img. That would be a sign the kernel compiled correctly.

In my latest build from yesterday afternoon, I have no kernel*.img in work/02-Kernel.

But I can't understand why the 5.2.20 rtl driver doesn't want to compile. I've did it compiled with @seeul8er procedure.

careyer commented 5 years ago

Alright... kernel1.img and kernel7.img are there...... all stages finished.... writing the image to an SD card now

careyer commented 5 years ago

Okay... here are the results:

GroundPI:

Image is booting - CHECK!
No splash screen showing
bootup sequence differs from original image (e.g. no message for setting up Wifi Cards shows)
video showing correctly - latency is ok - CHECK!
OSD showing - CHECK!
Telemetry (e.g. artificial horizion) does not work (however EZ_WBC internal telemetry eg. AirPi CPU and Temp working)
Hotspot Showing - CHECK!
Hotspot connects successfully (Message "Sendondary display connect shows)
Connection with MissionPlanner to TCP 192.168.2.1 not working

AirPi: Runnning a stock 1.6rc6 right now!

Update: Installes the Image to AirPi as well now...

AirPi:

Image is booting - CHECK!
Camera does not turn on
Wifi card (036NHA does not turn on - no light)
HDMI shows as last line: "Welcome to WiFi-Broadcast 1.6 (TX) - Press Enter to login"

RespawnDespair commented 5 years ago

I see a lot of CHECK's there, that's positive.

The splash screen was a test, so that will be fixed shortly.
If you do alt+f1 during boot it will show TTY1 where the wifi cards are being setup, by default it shows TTY12 now.
Connection with missionplanner is sad, can we be sure it is the right IP? I was really hopefull for this with all the Mavlink v2 changes i did to the source (from dino.de). I don't have a working plane with Pixhawk right now, i will try to make a bench test setup, but that will take time.

Also we need to see if it works as an AirPi image as well. I am working on a document which we can use as a guide to check everything when we release a new image. But first let's try to get the basics working. Good progress i would say!

My new Pi3B+ should arrive tomorrow so i can test compatibility as well. What are you running it on?

careyer commented 5 years ago

@RespawnDespair : Totally agree... this is making good progress. As for the MAVlink stuff you should maybe really contact dino.de. He had trouble to get this working as well. Somewhere there was a buffer that was too small and needed to be resized and that resulted in a corrupted checksum calulation ... bottomline was that MAVlink was not working at all.

As for the IP (Mission Planner telemetry). My PC gets assigned 192.168.2.2 (via DHCP from the Raspberry).... also I can find the Raspberry (wbc) at 192.168.2.1 with a network scan..... however what is strange: I can't ping 192.168.2.1.... This worked with the genuie 1.6rc6 image though. Maybe that is a good indication why Mission Planner connect doen't work? However since MavLink telemetry from the AirPi is not reflected on the OSD as well there also seems to be something wrong with the MavLink part as well

RespawnDespair commented 5 years ago

weird, i'll try to replicate the behaviour. I have seen the modifications from dino.de in the buffer:

https://github.com/RespawnDespair/wifibroadcast-base/commit/23f655f6886e30750507d28b580a09fc087d6f60#diff-9c18fe61ce296b99fa72aa8b44a41018R302

That is included, so it must be something more low-level, like something in the scripts not being right. If i can replicate it, i can try to find the cause.

careyer commented 5 years ago

Alright! Just lemme know if I can help somehow.... Keep ut the excellent work my friend! ### BIG THUMBS UP

RespawnDespair commented 5 years ago

Can you perhaps check your etc/network/interfaces file. I overwrite this in the build script, but when looking in dino.de's image i see a different content. Could you change the contect to this:

# interfaces(5) file used by ifup(8) and ifdown(8)

# Please note that this file is written to be used with dhcpcd
# For static IP, consult /etc/dhcpcd.conf and 'man dhcpcd.conf'

# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

auto lo
iface lo inet loopback

#auto wlan0
#iface wlan0 inet static
#     address 192.168.254.111
#     netmask 255.255.255.0
#     gateway 192.168.254.1
#     wpa-ssid xxx
#     wpa-psk xxx

The comment lines could be omitted offcourse. And then see if you are able to connect Mission planner?

RespawnDespair commented 5 years ago

@careyer thinking about this makes me wonder whether the problem is more with the fact that the airpi was not the latest image. Mission planner connects to the FC trough the gnd -> air pi. The telemetry was modified heavily, so it might not be compatible with an earlier image...

So testing with both airpi and gndpi with the latest image would verify whether this is the case...

careyer commented 5 years ago

@RespawnDespair : I edited the interfaces file and now it works! Also the OSD telemetry (art. horizion) works now... I can Ping the GroundPi and also connect via Mission Planner. All works well. :+1:

I cannot test with the AirPi from your image because it does not boot up correctly. (See above).

AirPi:

Image is booting - CHECK!

Camera does not turn on

Wifi card (036NHA does not turn on - no light)

HDMI shows as last line: "Welcome to WiFi-Broadcast 1.6 (TX) - Press Enter to login"

to compensate for the non-working AirPi image I use the dino.de image now as the AirPi. It works! :-)

This is what the AirPi (Buildscript version) outputs via HDMI. (no camera led and no Wifi led turns on - hence it is not transmitting anything):

RespawnDespair commented 5 years ago

Can you connect a keyboard to the airpi and swtich to tty1? alt+f1, the first tty shows the main sequence.

And great success! I'm sure the airpi not working is something minor in the scripts...(or at least i hope so!)

Yes21 commented 5 years ago

My testings with this morning build (on air and ground) :

my AWUS1900 (8814) isn't detected,
CSL300 is working on groundPi,
AWUS036ACH (8812) is working on the airPi,
CSL300 is not transmitting video, but is set up on monitor mode,
ethernet connections (192.168.0.18/19) ok on ground and air,
I can see OSD with telemetry,
I can connect QGC via wifi hotspot,
in QGC I can view the video, and telemetry, but it can't download parameters from the PiwHawk, and no bidirectional telemetry,

ps : i've perhaps an idea why my 8814 is not working ...

Yes21 commented 5 years ago

ps : i've perhaps an idea why my 8814 is not working ...


For building & installing the RTL8812AU driver with 'make' use

make
make install

and for building & installing the RTL8814AU driver with 'make' use

make RTL8814=1
make install RTL8814=1

With 5.2.20 and 5.3.4 the first make compiles the 2 drivers

careyer commented 5 years ago

@RespawnDespair : Sorry I cannot test this right now... my familiy has occupied the TV set for the next few hours

RespawnDespair commented 5 years ago

@Yes21 So to recap:

The latest image works for you on AIR and GND?
QGC works, but no download/upload, but there is telemetry
RTL8814 doesn't work (which is strange, i enabled it in the kernel config which should be used by the makefile)

5.2.20 appears incompatible with the latest kernel from github.

Yes21 commented 5 years ago

I'm building a new image with rtl8812au v5.3.4. The stage 02-kernel is finished, and I have the 2 kernel*.img. So the driver should have be compiled.

careyer commented 5 years ago

@RespawnDespair : Just an idea.... is it possible to include a unique versionnumber to a build, e.g. 1.6.xxx "My testings with this morning build" is a bit vague since you did many changes this morning (do not want to offend you Yes21 - so don't get me wrong).

Since my GroundPi image wasn't reachable at all (no ping possible but IP present) we might be talking about diffrent images here

rodizio1 / EZ-WifiBroadcast

Image-builder produces stable 2.0 image #166