Closed joknjokn closed 2 years ago
Hi, @gitalm @joknjokn --
I hope this finds you both doing quite well.
I wanted to share with you the news that I received a Raspberry Pi 0W2 as a gift yesterday and plan to test out ways that BirdNET-Pi might be able to run on that hardware. It's not an RPi3, but it's Broadcom chipset has the same core type, Cortex-A53 64-bit, but with a much lower clock speed (1.4GHz for RPi3, 1GHz for RPi0W2) and significantly less RAM.
I should be able to mitigate the memory constrains with well-configured swap space, but will let you know how I fare with the other obstacles.
My best regards to you both, Patrick
Interesting topic, since I would like to use one of my older 3B (not 3B+) for birdpinet. I think I would use an aluminium case which is a cooler, too. But I don't like these small fans since there is always trouble with them. Downside of a aluminium cooler case is that I would have to add an external wifi antenna.
Try to not mount the top of the mounting box. It must be the best way to maximize both airflow and WiFi signal. I have done so on my Raspberry Pi 4B because it is inside another box.
If a aluminium cooler case is being used, the top must be mounted for optimal cooling. That's why an external antenna is necessary. But that is not a big problem, since ready-to-use 'pigtail' RP-SMA cables were available.
@gitalm @oliverk71 @joknjokn @Svardsten53 Howdy, gang -- just wanted to let you know that I've gotten my hands on a Raspberry Pi 3B+ that I will test with today. I will let you all know my results once I've had enough time to tinker with settings.
My best!
The testing branch, forms
, runs on RaspiOS-Lite-ARM64, no overclocking, no fan (but it does have an aluminum, passive cooling/heat sinking case) recording length must be set to 30+ seconds. This means that it is about twice as slow, but within 1 minute of live audio!! Not too shabby!
If you want to test it out before I pull it into the main branch, you can do the following. (You can also wait until "version-0.11" is released as the main
branch for the repo.)
Download the RaspiOS-Lite-ARM64 image (get that here)
Use the Raspberry Pi Imager (get that here)
Utilize "Ctrl+Shift+X" to:
Connect to pi@raspberrypi.local via SSH (Linux and macOS guide here, Windows guide here)
Once connected, issue:
curl -s https://raw.githubusercontent.com/mcguirepr89/BirdNET-Pi/forms/newinstaller.sh | bash && sudo reboot
That will take care of all updates and will perform the installation.
After the system reboots, go to http://birdnetpi.local. On the left under "System Links" choose "Tools." On the "Tools" page, select "Settings" and then click the "Advanced Settings" button at the bottom. http://birdnetpi.local "Tools" "Settings" "Advanced Settings"
Change the "Recording Length" value to something greater than 30 in seconds -- 30 works well. Then click "Update Settings" and should get back "Success!"
After that, the new changes will be automatically applied. You can watch the analysis log using the other link found under "System Links" entitled "BirdNET-Lite Log."
Let me know if you encounter any trouble and I will be more than happy to help!
My best regards, Patrick
Patrick, thats really excellent! First trial result with my RPi3B+:
Excellent work!!!
UPDATE:
@DD4WH Yes it is working hard -- ironically it takes about 20~ seconds to build the model, and then about 2-5 seconds to process the 30 second clip :) -- I'm sure there are even better results that can be had, but these are pretty great for such an inexpensive computer with only 2G RAM and no additional swapping (it doesn't seem to need it at all?)
Awesome! So, I just realized my RPi3 is 1GB.
I have it up and running-ish. Installed fine (hours I guess), and the website works. But: The "Currently analyzing" spectrogram states a timestamp that is 25 minutes late or so. I guess it's struggling to catchup with a queue of recordings, or something like that?
Any suggestions to what settings I should try? I don't expect it to actually end up working well with 1GB, but I think it's worth the try now it's actually running.
@joknjokn I use 2 GB zRAM, moderate overclocking and 30 s audio on my 3B. Seems to work. You can set zRAM use in birdnet-pi-config, overclocking in boot.conf and audio recording time in the advanced settings of the birdnet-pi's local website.
Thanks, @oliverk71
@joknjokn , could you compare what you see on your "System Links" > "System Info" page around where it looks likes this and let me know what you see?
@oliverk71 , thanks - it seems to be working better now. But not sure it actually uses the zRAM? (see attachment)
@mcguirepr89 So, I enabled 2GB of zRAM, set recording length to 30 sec and channels to 1 (mono sound card... So this should be 1, right?)
It runs and detects now! Amazing. The website is really slow at times (as expected). I'm wondering why zRAM says 0% used?
Can I set it to delete recordings (and related data) for recordings that gave 0 results? Just to clean it up a bit. Or how is it handled if it fills up the memory card?
My zRAM is not being used, either. I did this before I read about 30 s audio recordings and I am still using a few days old testing version of birdnet. Now, I will first burn a new image and reinstall, then without using zRAM or overclocking and see what happens.
I think mine is starting to struggle again. The "currently analyzing" is now 2 hours behind time.
@mcguirepr89 Should I do anything else to make zRAM work? I enabled it and rebooted through the birdnet-pi-config.
And would lower recording time be better for performance, or worse?
@joknjokn
So, I enabled 2GB of zRAM, set recording length to 30 sec and channels to 1 (mono sound card... So this should be 1, right?)
Yes, use 1 channel there if you'd like, though pulseaudio can very likely keep things with 2 channels, it will simply be 2 channels of the same mono. ( I don't have mics to test with, but my understanding of pulseaudio is that it abstracts the audio enough to provide multiple (virtual) channels. )
@joknjokn It runs and detects now! Amazing. The website is really slow at times (as expected). I'm wondering why zRAM says 0% used? @oliverk71 My zRAM is not being used, either.
zRAM and swap space in general will only ever be utilized by the system if and when it needs to offload memory (RAM not storage) when it is full. Based on the picture you've shared, the memory offloaded about 2MB to the first swap space and has not needed any of the zRAM swapping. Again, in my testing, where everything is still running strong, I have not allocated any additional swap space and have not overclocked the system.
@joknjokn Can I set it to delete recordings (and related data) for recordings that gave 0 results? Just to clean it up a bit. Or how is it handled if it fills up the memory card?
The system does not keep any data (recordings nor analysis results) if no species are detected. When the disk space reaches 95%, the oldest day's worth of recordings are removed (but the database is untouched). If after clearing the oldest day's worth of recordings still leaves the disk space at 95%, all of the data in the "Processed" directory is removed. This keeps the installation running indefinitely, removing the oldest data as needed. There is also a setting in "Advanced Settings" called "Full Disk Behavior" that defaults to "0". "0" does what I've described above. If you wanted to set that to "1" it would tell the system that you want to keep all of your data when the disk gets full, at which point it will trigger a script that will stop all of the data collection and analysis services.
@joknjokn I think mine is starting to struggle again. The "currently analyzing" is now 2 hours behind time.
This could be a web-browser caching issue. Try deleting your browser cache and see if that image updates.
@joknjokn And would lower recording time be better for performance, or worse?
From my testing, the RPi3B takes approximately 23 seconds to build the prediction model -- after that, it takes only a few seconds to analyze the audio. With that said, if the recording time is less than 23 seconds + the time it takes to process the audio, you will have a "queue" of audio that will start getting backed up. This is why I recommend 30+ seconds. 30+ seconds allows the analysis service the 23~ seconds or so it needs to build the model, a few seconds to process the audio, then it is ready to start over. That timing prevents audio from "queuing" and getting backed up.
If you wanted to try to get better performance, you could try a 45 second recording time (try to keep recording times divisible by 3 since BirdNET-Lite divides the audio into 3 second chunks).
I am very curious, though, why your system is not performing the way mine and @DD4WH's are? He's seeing the exact same results I see. I will be interested, @oliverk71, to learn how your installation runs after a clean install.
Thank you all so much for sticking with me as I struggled to get my hands on an RPi3B for testing. I very happy with the results so far. I am hopeful we'll be able to get this working for everyone on this thread.
My best!
@mcguirepr89 Thank you so much for the detailed response!
So, for the RAM/SWAP/zRAM, it looks like it has changed a lot since the last screenshot:
Regarding the delayed time, I don't think it's a caching issue. When I watch the BirdNET-Lite log, it's also analyzing old recordings. I will try with 45 sec.
Regarding performance: I actually thought your RPi3's were 2GB. Could it be an overheat issue? I only have a small heatsink at the moment. If you want me to check temperature or whatever, let me know (but I would probably need info on what commands to run... Not the most experienced with linux.).
Checking temp is a good idea -- you should just be able to scroll down on that System Info page to see the CPU temperature in Celsius -- if for whatever reason it is not available there, I can guide you through getting that info another way
CPU says 65 degrees, so I guess that's not the issue.
That is a good 20 degrees hotter than mine -- though it's not a dangerous level
@joknjokn Did you use the lite version (no desktop, command line only) of raspianOS bullseye to flash it on your SD card?
@oliverk71 I used the RaspiOS-Lite-ARM64 image as suggested by @mcguirepr89 yesterday.
My microSD is this one. Should be fine, right?
That looks good to me. :thinking: I'm going to run through the installation again right now and see if I am forgetting anything, if anything comes to mind, or if I get different results that might help me replicate your issues. I will let you know if I find anything. :crossed_fingers:
Not sure I understand exactly how the analysis work, but does the system rebuild the prediction model for every recording? Isn't is possible to just do it once and then feed it the data when new recordings come in?
Not sure whether we all use the same configuration:
I use RPi 3B+ (NOT RPi 3B) and the original setup without any zRAM swapping etc.
So, my info screen looks different: There is NO swap zram in my case . . .
Looks like @DD4WH and @joknjokn have RPi3B+ (ver 1.3), while mine is a RPi3B (version 1.2) -- I would suspect that mine would be the one with trouble :thinking: ???
Is there a way to check if mine has issues with power supply/throttling ?
There is, but it is in the command line -- open the web terminal and issue vcgencmd get_throttled
and let me know what the results are.
The difference with @joknjokn could be the additional graph in the info: 2GB zRAM ? BTW: it works without problems with recording length 30sec, but also with 60sec
@mcguirepr89 It printed: throttled=0x80008
Interesting - if google is telling the truth, it appears that RPi3+ is throttling at 60 degrees while RPi3 throttles at 70 degrees.
I just put a handheld fan on top of it. What an insane difference. Web-ui is lightning fast and the time delay is gone! CPU is 45 degrees now. So I assume the issue was throttling due to CPU temperature.
throttled=0x80008 Good resource
Your vcgencmd HEX converts to these bits 10000000000000001000
Using this vcgencmd table, the following is indicated:
10000000000000001000
|||| ||||_ Under-voltage detected
|||| |||_ Arm frequency capped
|||| ||_ Currently throttled
|||| |_ Soft temperature limit active
||||_ Under-voltage has occurred since last reboot
|||_ Arm frequency capped has occurred
||_ Throttling has occurred
|_ Soft temperature limit has occurred
You have hit the soft temperature limit and that soft temperature limit is active
Not sure really what this means, but anything other than 0x0 is bad news and leads to problems. I'm looking into this a bit more as the installation continues.
@joknjokn Great work troubleshooting!!! I'm so glad you've found the issue!
11010000000000000000 now, with the lower temp
|||| ||||_ Under-voltage detected
|||| |||_ Arm frequency capped
|||| ||_ Currently throttled
|||| |_ Soft temperature limit active
||||_ **Under-voltage has occurred since last reboot**
|||_ Arm frequency capped has occurred
||_ **Throttling has occurred**
|_ **Soft temperature limit has occurred**
So I guess I'm all good now, except that the handheld fan sounds like a swarm of starlings fighting wasps.
You last comment does suggest that your power supply is inadequate -- the core is still under-powered during CPU spikes, wherein things get throttled, and the soft temperature limit was still hit. I'd say, though, that it is running better in general since it seems to only report those as having happened and are not currently happening. Be careful with undervoltage warnings as they can lead to data corruption that can be undetected for a while . . . until your database breaks due to the corruption.
@mcguirepr89 , thanks! I'm using a Xiaomi 5v3A quickcharger and the thickest micro-USB cable I had around. So It wouldn't surprise me that it's not optimal. I'll buy and try a better power adapter!
I'm also going to buy a RPi4 with the official power supply. Only 2GB is in stock. Do you know of any performance difference between RPi4 2GB and 4GB for BirdNET-Pi ?
I don't have any experience with the 2G, but I wouldn't expect it to perform drastically different for the BirdNET-Pi. If you have any plans to use the full version of RaspiOS (instead of RaspiOS-Lite), then I would encourage you to get the 4GB RAM option, as the graphical user interfaces hog RAM quickly.
Alright, that's great to hear. I will probably have fun with RaspiOS at some point, but 4G won't be in stock anywhere until April I believe. Can't wait for that ;-)
Btw., I guess a possible explanation for the initial temp-issue, then under-voltage issue, could be that until I cooled it, it never really got to consume too much power before it throttled due to temperature. Now with the lower temperature, it speeds up and hits the under-voltage. (Maybe that's what you also suggested.)
Turns out once again that temperature can cause problems and appropriate cooling is essential. Until now I am fine with the aluminium heatsink case, but let's wait for summer. I think it is important to protect the pi against the sun. But on real hot days (above 35°C) temperature may cause problems even without the sun directly shining on the pi. However, I hope there will be better cooling cases in future, which maybe combine heat pipes and large coolers. Because I really don't want to use a fan.
I'm going to try an RPi4 2G with this fan (hopefully silent) in the official case:
@joknjokn What kind of power supply do you use? Specs? Make sure it delivers enough power. Regarding fans: small fans are never silent, because they need to run faster.
I've had a lot of success using 12v desktop computer fans plugged into the 5v pins. They're very quiet at that voltage and work like a charm (and they're SUPER cheap if you can find them as old parts from a junk computer).
@joknjokn
Btw., I guess a possible explanation for the initial temp-issue, then under-voltage issue, could be that until I cooled it, it never really got to consume too much power before it throttled due to temperature. Now with the lower temperature, it speeds up and hits the under-voltage. (Maybe that's what you also suggested.)
I think that is very plausible/probable and a great observation.
I always try to exchange small fans with larger fans whenever possible. But I do not want to use ANY fan for a pi, especially not for outdoor use.
Regarding the pi 4B: there is a heatpipe cooler available! It is relatively small and it comes with a fan (to get rid of the heat piped off the CPU to the above cooler). There are two versions. Here is the image of the tower cooler:
@mcguirepr89 I just installed birdnet-pi on the 3B again. There was an error at step 3. The caddyfile was not updated with the address I entered in the pre-configuration, so I did it manually. Also, I tried to set the recording time to 30 s, but it did not work somehow (edit: works after reboot). Don't know if it possibly makes problems that I have now two birdnet-pis in my local network. Guess, I will have to update the menu.html and the caddyfile, to the IP-address or local domain birdnetpi2.local. Give me some time to check everything.
@oliverk71
There was an error at step 3
That portion implies you installed using birdnet-pi-config
, is that correct? You did not use the installation one-liner that calls newinstaller.sh
, is that correct?
I need to confirm because the new forms
branch that will ultimately get pulled into the main
branch will have birdnet-pi-config
removed. The forms
branch is my current testing ground for new ideas, and the new installation approach is now to always perform a default installation with the one-liner and perform customizations afterwards.
If I'm going to change the documentation for installation before doing the pulling, so that everything will reflect this new approach.
Of course, I'm a command line kind of guy, so there will always be a command-line way of doing things if you need/want to.
So, to install from the command line using a pre-configured birdnet.conf
, do the following:
Note: This example assumes you have only booted into a newly installed RaspiOS-Lite-ARM64 image.
git
sudo apt update && sudo apt -y install git
git clone -b forms https://github.com/mcguirepr89/BirdNET-Pi.git /home/pi/BirdNET-Pi
birdnet.conf-defaults
file called birdnet.conf
cp ~/BirdNET-Pi/birdnet.conf-defaults ~/BirdNET-Pi/birdnet.conf
birdnet.conf
to your needs. *Note: Don't use the EXTRACTIONLOG_URL
under Custom URLs for now. It is the Web Terminal now. It will be changed sometime soon.
nano ~/BirdNET-Pi/birdnet.conf
or
sudo apt -y install vim && vim ~/BirdNET-Pi/birdnet.conf
birdnet.conf
~/BirdNET-Pi/scripts/install_birdnet.sh
I hope that helps out! Best regards, Patrick
Also, if you are concerned about there being two Raspberry Pis with the same hostname, you do a few things:
Note: These steps would take place just after making a copy of birdnetpi.conf-defaults
~/BirdNET-Pi/install_services.sh
before running the installer, and after installing, changing your ~/BirdNET-Pi/scripts/update_birdnet.sh
install_services.sh
to replace birdnetpi
with the hostname of your choosing:
...
set_hostname() {
if [ "$(hostname)" != "birdnetpi" ];then
echo "Setting hostname to 'birdnetpi'"
hostnamectl set-hostname birdnetpi
sed -i 's/raspberrypi/birdnetpi/g' /etc/hosts
fi
...
birdnetpi.conf
to your needs, probably putting http://mynewhostname.local
as the Custom URL for BIRDNETPI_URL
. If you do want that, over say and IP address, then you should also issue sudo systemctl enable --now avahi-alias@mynewhostname.local.service
. That will send the mDNS
announcement to the network that your new installation is mynewhostname.local
now (just like birdnetpi.local
is for a default installation).
}Bonus: If you like working in the terminal, I have put together a very informative tmux.conf
that can be used to monitor things.
sudo apt -y install tmux && sudo ln -sf ~/BirdNET-Pi/templates/tmux.conf /etc/tmux.conf && tmux
If you've never used tmux before, the "prefix" to send (most) commands to tmux
is CTRL+B
, then you press whatever you need. The best place to start exploring with that is CTRL+B
, then ?
. Have fun!
@oliverk71
There was an error at step 3
That portion implies you installed using
birdnet-pi-config
, is that correct? You did not use the installation one-liner that callsnewinstaller.sh
, is that correct?
Oh, now I realize what happened. I used the newinstaller.sh but I then used birdnet-pi-config, too!
I will try again.
@mcguirepr89 Works perfect! I just discovered the web terminal. Until now I always used ssh -l pi IP for login and had to enter my password, which I cannot remember and had to look up every time. Of course, I could have used a client, but I like the web terminal very much. :) And even more the idea to use it over the internet. This will make it much easier to remote access birdnet-pis without necessarity of open SSH port. I also had a quick look into the file manager and I think this can be useful. But, if am not wrong again, the user changes: a file owned by pi and copied is then owned by caddy. For the file permissions I personally prefer ugo, because I cannot remember the numbers, lol. But maybe I can get used to it or look it up. Ah, I see, when I click on it, I can set permissions easily. Nice! Would it be possible to copy files not only locally but over the internet?
I think about to replace the pi 4B in my garden with the 3B, since everything works nice. But I would like to copy the database from the 4B to the 3B to not lose any bird recognitions. Of course, I have to be careful and set it up with the exactly same GPS coordinates and ID for birdweather etc.
@oliverk71 Thanks for the kind words :)
I like the web terminal very much. :) And even more the idea to use it over the internet.
Thanks again, but that is thanks to @yudai'sGoTTY
I still need to lock down that service through my little caddy
reverse_proxy localhost:8080
trick via another mDNS and need to replace the EXTRACTIONLOG_URL
with WEBTERMINAL_URL
and default that the way birdnetpi.local
is the default mDNS for the BIRDNETPI_URL
. I've tested this and it works great. I'm also considering utilizing caddy
's tls internal
as a default so that everything can be served over HTTPS, even locally. That's all to say, you probably shouldn't make it internet accessible yet, unless you lock it down in some other fashion that prevents tampering with your system. Just a tester's warning :)
But I would like to copy the database from the 4B to the 3B to not lose any bird recognitions.
Unless you're planning on changing out SD cards for some reason, I'm 90% sure you can just put the SD card you're using into the RPi3B and it shoud just work, as they say. If not, there is a command line tool I put together called rebuild_db.sh
, but there is one line that needs to be corrected (you can probably git pull
to get this change). If you need to do a clean install on the RPi3B, you can just copy over the BirdDB.txt
file from one system into the new installation's /home/pi/BirdNET-Pi
directory, and then run rebuild_db.sh
and it will do what it promises. Hope that helps!
Hi,
I'm wondering if you think this could run on RPi 3? If so, would it require difficult changes?
Thanks!