breakage / errors due to obsolete code from buster packages still in use

BlackNet commented 1 year ago

What happened

Updated to v4.0.2-0-gb0c011b
Camera is using a ribbon cable and is recognized by the system and previous functional.
Acknowledgments that version 4 of Crowsnest does NOT support buster ..Debian 10 (buster) has reached end of life. Therefor crowsnest has also end support for this. We are sorry for inconvinience...

However, this is NOT the issue I have.

Camera fails with error crowsnest: Dependency: 'camera-streamer' not found. Exiting!

-- troubleshooting steps --

via ssh I ran make install, yeilds errors. `Reading package lists... Done Building dependency tree... Done Reading state information... Done Package liblivemedia-dev is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source

E: Package 'liblivemedia-dev' has no installation candidate Install dependencies ... [FAILED] Something went wrong! `

ran the following grep liblivemedia * -r

results here: tools/libs/pkglist-rpi.sh:PKGLIST="${PKGLIST} liblivemedia-dev pkg-config xxd build-essential cmake libssl-dev"

Verified this is the correct, upto date from github repo here https://github.com/mainsail-crew/crowsnest/blob/master/tools/libs/pkglist-rpi.sh

dup up info on the package in question to find out where I can get it .....https://packages.debian.org/search?keywords=liblivemedia

This is a BUSTER ONLY package.

So can someone please tell me WHY Buster support is dropped and still insist on using Buster ONLY packages?

What did you expect to happen

smooth transition to ver 4.

How to reproduce

see above for details.

Additional information

No response

BlackNet commented 1 year ago

Worth noting, this also breaks new clean installs as well.

sdooweloc commented 1 year ago

I have also encountered this issue. when I tried to update.

However, when I went into the pi and manually did sudo make install it worked without issue. Though something is definitely funky now with my camera's resolution as it is now taller than it is wide but thats a topic for another day ahah.

meteyou commented 1 year ago

@BlackNet https://github.com/mainsail-crew/crowsnest/issues/85

BlackNet commented 1 year ago

That is false. The issue I brought up was not addressed in that thread at all.

meteyou commented 1 year ago

ah sry. i read it again. if you execute sudo make install, the file will be updated/changed. check out the install script: https://github.com/mainsail-crew/crowsnest/blob/master/tools/libs/core.sh#L45

BlackNet commented 1 year ago

This has nothing to do with buster. I am fine with dropping buster. However it is not ok to force using a package that is ONLY in buster.

There is NO package for livemedia in Debian bullseye repo. It is ONLY in old stable(buster).

Also, the make install throws 2 errors with ffmjpg and will not compile, will have to post the two errors today sometimes.

New clean installs fail with any system not using buster linked repos. Which means you say you dropped support for buster but you really are not dropping it.

meteyou commented 1 year ago

Pls read my answer again! The file should be changed during the install process!

BlackNet commented 1 year ago

The install failure is with dependency checks. Perhaps the install script needs to know about the renaming before that takes place.

meteyou commented 1 year ago

You should not see it any time, because the pgklist should be changed and then installed. I reopened it, that @KwadFan can double check it.

BlackNet commented 1 year ago

This is the compile errors AFTER I removed the buster dependency file.

make[1]: Leaving directory '/home/pi/crowsnest/bin/camera-streamer' make: *** [Makefile:57: all] Error 2 make: *** [Makefile:17: install] Error 2

going in scrollback here is the errors.

make[1]: Leaving directory '/home/pi/crowsnest/bin/ustreamer' Build 'ustreamer' ... [SUCCESS] Build 'camera-streamer' using 4 Cores ... fatal: No names found, cannot describe anything. echo "#define GIT_VERSION \"\"\n#define GIT_REVISION \"f0634f9\"" > version.h.tmp; \ diff -u version.h version.h.tmp || mv version.h.tmp version.h; \ rm -f version.h.tmp diff: version.h: No such file or directory

cc -std=gnu17 -MMD -Werror -Wall -g -I/home/pi/crowsnest/bin/camera-streamer -D_GNU_SOURCE -DUSE_FFMPEG -DUSE_LIBCAMERA -I/usr/include/libcamera -DUSE_LIBDATACHANNEL -Ithird_party/libdatachannel/include -Ithird_party/libdatachannel/deps/json/include -c -o util/http/http_methods.o util/http/http_methods.c util/ffmpeg/remuxer.c: In function ‘ffmpeg_remuxer_open’: util/ffmpeg/remuxer.c:97:33: error: initialization discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers] 97 | AVInputFormat *input_format = av_find_input_format(remuxer->input_format); | ^~~~~~~~~~~~~~~~~~~~

the ~~~~~~~ is under av_find_input_format fyi.

meteyou commented 1 year ago

Bullseye have other dependencies... Only remove the single one will not fix it.

BlackNet commented 1 year ago

which ones am I missing then?

aeli292 commented 1 year ago

camera-streamer

KwadFan commented 1 year ago

I am missing a crucial information in that whole topic, which OS exactly is used?

I am asking because we build lately new OS Images and they are based on RaspiOS bullseye. So, if there where no installation candidate for it, it would fail in our buildchain and not producing a usable Image.

SUCCESS] Build 'camera-streamer' using 4 Cores ... fatal: No names found, cannot describe anything. echo "#define GIT_VERSION \"\"\n#define GIT_REVISION \"f0634f9\"" > version.h.tmp; \ diff -u version.h version.h.tmp || mv version.h.tmp version.h; \ rm -f version.h.tmp diff: version.h: No such file or directory

This one tells me that the machine you used wasnt connected to the internet or has issues to reach github ...

I will take the latest RaspiOS Bullseye 64bit and 32bit lite variant from rpi-imager. And test myself.

aeli292 commented 1 year ago

OS is bullseye 64bit on RPI 4 it seem that the sources of camera-streamer are not pulled and compiled on the update from v3.

do:

cd ~/crowsnest
make clean
make buildclean
cd bin
./build.sh -r
cd ~/crowsnest
make

fixes the build.

sudo make install .... ignores the existence of the camera-streamer repo and pulls again. and a lot of third-party. build again ... and seems to work for me.

KwadFan commented 1 year ago

OS is bullseye 64bit on RPI 4 it seem that the sources of camera-streamer are not pulled and compiled on the update from v3.

do:
cd ~/crowsnest
make clean
make buildclean
cd bin
./build.sh -r
cd ~/crowsnest
make 
fixes the build.

sudo make install .... ignores the existence of the camera-streamer repo and pulls again. and a lot of third-party. build again ... and seems to work for me.

I am a bit shocked now, that someone had discovered how build.sh works ^^ Good Job mate!

KwadFan commented 1 year ago

@BlackNet So, after facing it with someone on Discord, I start to understand the whole thing you meant (I think :thinking: ).

So, you upgraded a buster OS due changes in apt's sources.list to bullseye, am I right?

If that is the case, I have to apologize but I am no fan of doing that! This somewhat works in x86 World but as you can see is a really really bad Idea to do this in the SBC Landscape. Those Distro made by SBC manufactures are specialized in every possible direction and not purely a "Debian" even they are based on.

So, please misunderstand me here, I am really intressted in getting it running on almost everything we use in 3D Printing World, but I have no time and to be honest no desire to cover every possible, by the user, hacked together 'frankestein's monstrosity'.

So, if you grab an RaspberryOS bullseye lite and something is not working as intended I am here and will fix it, but not those 'frankensteins' ....

Regards Kwad

BlackNet commented 1 year ago

Nope. Was never buster on this setup at all.

KwadFan commented 1 year ago

Then please tell me how to reproduce. I tested both RPI 32 and 64bit. Also works on armbian images(without camera-streamer obviously)

KwadFan commented 1 year ago

Just to show you that I dont use any buster only packages. I guess liblivemedia-dev package, which is also documented here, is a package from the RPI mirror and not a Debian one.

Bildschirmfoto_2023-05-26_18-45-25

After some quick research, take a look here: https://archive.raspberrypi.org/debian/pool/main/libl/liblivemedia/

I think that proofes it is a Raspberry Foundation package only.

Regards Kwad

BlackNet commented 1 year ago

Yes, I have come to that conclusion (it is a Raspberry Foundation package) from the first post, which was one of the reasons for the post.

I looked at several repositories and found that:

Debian OS Buster WAS listed, in OLD-STABLE.
Debian OS Bullseye was not listed.
Debian OS Bookworm was not listed.
Raspberry OS bullseye WAS listed.
Raspberry OS bookworm is not listed, this may change.

I did not check Ubuntu or other repos.

Worth note, when I say Debian Bullseye I mean Debian 11 OS. Much of the confusion above seems to be 'Bullseye' to mean Raspberry Pi OS (Bullseye). There is a difference and they are not the same.

I have tested several clean installs of Debian Bullseye, Debian Bookworm, and a few others (mostly Bookworm) and everyone had the same compile errors I posted above, and the missing package.

Also, worth noting is that Debian Bookworm is now under hard freeze and its release date (June 10) is just 15 days from this post. Soon to follow will be Raspberry OS Bookworm and likely in a few days to a few weeks after Debian. Raspberry Foundation has said they are waiting for Debian to release Bookworm and they will follow suit. Once this happens the package that was listed above will be moved to old-stable and often unreachable unless you update apt/sources.list accordingly.

I think going forward the best action to do is to get all of the needed packages to compile OUTSIDE of the Raspberry OS environment and with the right packages then it should work for all.

meteyou commented 1 year ago

@BlackNet these are all dependencies of camera-streamer. maybe you should make these request here. but camera-streamer is a streamer for raspberry pi devices...

KwadFan commented 1 year ago

I am aware, that debian has a hard freeze phase( I am the only Linux daily driver on the team), but my plan is to support Raspberry Pi OS Bullseye as long as possible, we have users that are running buster, in our scope, users are lazy af.

So, tbh, for the Milestone of MainsailOS my plan is to change over to testing to get a somewhat rolling release disto, but this has to be tested a lot, so it will take up a really long time before that happends.

Also I started to build bookworm image recently, which require only a few small changes till yet. So hope are high transition to that will be smooth.

That said, I am in contact on a daily bases with Ayufan, the creator of camera-streamer and he is aware of that issue and working on a solution. Worst case scenario would be camera-streamer looses its ability to deliver RTSP stream, which to my eyes isnt a real loss.... WEBRTC ist the way to go

Regards Kwad

BlackNet commented 1 year ago

I have poked around the install scripts and have been able to get things installed under Bookworm. The problem still has to compile errors for av_find_input_format.

cronos45 commented 1 year ago

@aeli292 this is what I get after I enter make, is this the output I should expect?

`anthonycruz@mainsailos:~/crowsnest $ make crowsnest - A webcam Service for multiple Cams and Stream Services. Usage:

install Install crowsnest (needs leading sudo) uninstall Uninstall crowsnest build Compile backends / streamer buildclean Clean backends / streamer (for rebuilding) clean Clean .config config Configure crowsnest installer help Shows this help update Update crowsnest (fetches and pulls repository changes) report Generate report.txt `

mmaravillo commented 1 year ago

I have poked around the install scripts and have been able to get things installed under Bookworm. The problem still has to compile errors for av_find_input_format.

While crowsnest is being compiled on raspbian bookworm, I had to quickly edit remuxer.c before gcc reaches it, as a workaround on the av_find_input_format error.

diff --git a/util/ffmpeg/remuxer.c b/util/ffmpeg/remuxer.c
index fb1b141..a7dbc13 100644
--- a/util/ffmpeg/remuxer.c
+++ b/util/ffmpeg/remuxer.c
@@ -94,7 +94,7 @@ int ffmpeg_remuxer_open(ffmpeg_remuxer_t *remuxer)
   if (remuxer->packet)
     return 0;

-  AVInputFormat *input_format = av_find_input_format(remuxer->input_format);
+  const AVInputFormat *input_format = av_find_input_format(remuxer->input_format);
   if (!input_format)
     return AVERROR(EINVAL);

Looks like this is fixed upstream:

https://github.com/ayufan/camera-streamer/commit/f04e9311ab4f05ed86b2e2a6abceb03d87be04c6#diff-0d70bb7820c5135ca6767a8380aeed8130b8cccc7e77ad203ca7f6d6be544552****

shoikan commented 1 year ago

Looks like this is fixed upstream:

ayufan/camera-streamer@f04e931#diff-0d70bb7820c5135ca6767a8380aeed8130b8cccc7e77ad203ca7f6d6be544552****

Can confirm that this diff fixed the compile and install for me. Also did it 'on the fly' while it was compiling.

mainsail-crew / crowsnest