Closed isZumpo closed 3 months ago
I like idea of having images for each species, that has been on my mind to implement, but I don't want it to be dependent on any online service. Using external resources is often against Terms of Service, and with BirdWeather there is also availability issues.
So I am all in to have thumbnails for each species but they must be locally served, and for that we need to find a royalty free source for photos or have them done using generative AI like Midjourney or Dall-E.
Yes those are some really good points! I am also quite curious about them. Especially the Terms of Service aspect... I suppose using the birdweather API to post things is different from utilizing their images. I can see that birdnet-pi is grabbing their images from flickr so that might be a more viable solution?
However, does everything really need to be served locally? That mostly increases the size of the binary, as well as the bandwidth requirements. Sure you get better availability since there is less external dependencies, and have an easier time in air-gapped environments, but is it always necessary? Perhaps some sort of hybrid approach is also a viable option?
Regarding ToS issues, perhaps wikidata could be a potential source?
This!
However, does everything really need to be served locally? That mostly increases the size of the binary, as well as the bandwidth requirements. Sure you get better availability since there is less external dependencies, and have an easier time in air-gapped environments, but is it always necessary? Perhaps some sort of hybrid approach is also a viable option?
To be honest, I really enjoy the "simple" view right now. This birdnet-Program is really lightweight and draws much less energy then for example the birdnet-pi. Which makes it suitable for battery powered applications (I know there are better solutions, like the AudioMoth or other diy ARU)
Me personally would prefer pictures from an online source, so it's possible to turn this feature off and to save storage. I know how most birds in my area look like- (I get it why pictures are great tho!) -
If needed I can provide some Bird-Pictures (photography brought me to birdnet ;) ) As long as it's not commercially used. (And this feature can be turned off before it downloads XX-hundreds of pictures)
Obviously I don't have ALL birds on my camera (If you're interested let me know, even when you don't want to implement it right now)
Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."
Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."
That is nice! In theory we could then create our own set of pictures from the same source and either bundle in the git repository itself or host somewhere. Though, I would prefer to use birdweather directly since their api providers some really cool features. Not sure how we should move forward on this topic...
Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."
That is nice! In theory we could then create our own set of pictures from the same source and either bundle in the git repository itself or host somewhere. Though, I would prefer to use birdweather directly since their api providers some really cool features. Not sure how we should move forward on this topic...
I created an issue as a feature tracker for stuff from birdweather. Probs discuss stuff there and see what other people think too.
Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."
That is nice! In theory we could then create our own set of pictures from the same source and either bundle in the git repository itself or host somewhere. Though, I would prefer to use birdweather directly since their api providers some really cool features. Not sure how we should move forward on this topic...
I created an issue as a feature tracker for stuff from birdweather. Probs discuss stuff there and see what other people think too.
Awesome :) Do you mind linking to the issue?
Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."
That is nice! In theory we could then create our own set of pictures from the same source and either bundle in the git repository itself or host somewhere. Though, I would prefer to use birdweather directly since their api providers some really cool features. Not sure how we should move forward on this topic...
I created an issue as a feature tracker for stuff from birdweather. Probs discuss stuff there and see what other people think too.
Awesome :) Do you mind linking to the issue?
The recent updates encompass significant enhancements to the system's image handling and caching functionalities. Key additions include methods to fetch and cache bird thumbnails, improved database retrieval functions, and new image API integrations. These updates enhance the user interface by displaying bird images and attribution details in tables, leveraging asynchronous cache initialization for efficient image handling.
Files/Groups | Summary |
---|---|
internal/datastore/interfaces.go , internal/datastore/... |
Added new methods for retrieving detected species and expanding bird data retrieval. |
internal/httpcontroller/routes.go , internal/httpcontroller/utils.go |
Introduced image caching, Wikimedia querying, and thumbnail display functions, along with route and utility updates for these features. |
views/fragments/birdsTableHTML.html , views/fragments/recentDetections.html |
Added thumbnail images and attribution details to bird species tables. |
go.mod |
Updated dependencies to support new features. |
internal/httpcontroller/imageprovider/... |
Implemented image fetching, caching mechanisms, and Wikipedia interactions for bird images. |
internal/httpcontroller/init.go |
Introduced bird image cache initialization in the server setup. |
sequenceDiagram
participant User
participant Server
participant Datastore
participant WikimediaAPI
User->>Server: Request bird species data
Server->>Datastore: Query for detected species
Datastore-->>Server: Return detected species list
Server->>WikimediaAPI: Request bird thumbnail for species
WikimediaAPI-->>Server: Return bird thumbnail and attribution
Server->>User: Display bird species with thumbnails
Whiskers twitch in coder's delight,
Bird thumbnails now take their flight,
Cached and loaded, swift as breeze,
With attribution notes, sure to please.
Datastore hums, steady and true,
Images fetched, a splendid view,
In rabbit's realm, code dreams anew.
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Regarding of internet dependency I think it would be OK to have as an option. whoBird has a toggle that pulls down images from Macaulay Library https://www.birds.cornell.edu/home/ebird-api-terms-of-use/ Looks to permit use but way I am reading it could not be packaged inside of the container, could have same setup as birdweather.
I am sorry but I am not going to accept this PR, I am not okay with consuming any resources from BirdWeather.com without prior agreement with Tim & Co., and even then I don't see any point of accessing resources from them if original source is publicly available at somewhere else.
I am sorry but I am not going to accept this PR, I am not okay with consuming any resources from BirdWeather.com without prior agreement with Tim & Co., and even then I don't see any point of accessing resources from them if original source is publicly available at somewhere else.
I am completely on board with this thought too.
Was rebasing the thumbnail branch yesterday and planning on soon replacing the birdweather api calls for wikimedia ones. Though it might make more sense to close this PR and make a new for that?
Ok finally got around to creating some sort of PoC of the wikimedia api. Thanks to @PovilasID for making me aware of the query feature in the wikipedia api in the discussion thread yesterday. Was quite lost on how to get the correct image for a given bird in some automatic way otherwise :)
Image proof that it still works as expected:
However, I am still not quite sure how we are actually supposed to give the correct credit for the images. I can see that the birdweather graphql api returns these fields along with the image:
imageCredit - Credited author of image
imageLicense - Name of image license
imageLicenseUrl - URL to image license page
I have never used birdweather directly myself, so I don't know how these fields are displayed inside the application itself? Do you have any idea @matthew73210
@isZumpo np. I do not think we can really get good way to get author info I think for now you can probably just split the string of the image url on the dash after 300px and replace everything before it with https://en.m.wikipedia.org/wiki/File:
aka https://en.m.wikipedia.org/wiki/File:Blackcap_(Sylvia_atricapilla)_male.jpg and stick in hover over or meta data or in full page with Source Wikipedia
macaulaylibrary.org has en embed view https://macaulaylibrary.org/asset/211672961/embed to get pics from them you need to know image ID... whobird just made 3 60k arrays... to be as lookup table lol. So Sylvia atricapilla_Eurasian Blackcap
in https://github.com/woheller69/whoBIRD/blob/master/app/src/main/assets/labels_en.txt corresponds to blackc1
in https://github.com/woheller69/whoBIRD/blob/master/app/src/main/assets/taxo_code.txt and 211672961
in https://github.com/woheller69/whoBIRD/blob/master/app/src/main/assets/assets.txt this permits them build a quick image URL: https://cdn.download.ams.birds.cornell.edu/api/v1/asset/211672961/900 This is a very fragile way of doing things but there is probe a way to update it and this way is very fragile but I have to admit I do prefer their pictures.
Ok finally got around to creating some sort of PoC of the wikimedia api. Thanks to @PovilasID for making me aware of the query feature in the wikipedia api in the discussion thread yesterday. Was quite lost on how to get the correct image for a given bird in some automatic way otherwise :)
Image proof that it still works as expected:
However, I am still not quite sure how we are actually supposed to give the correct credit for the images. I can see that the birdweather graphql api returns these fields along with the image:
imageCredit - Credited author of image imageLicense - Name of image license imageLicenseUrl - URL to image license page
I have never used birdweather directly myself, so I don't know how these fields are displayed inside the application itself? Do you have any idea @matthew73210
Re credit, probs the best way is to have it mentioned on the github page and under options (if that's where you put the toggle). If memory serves they can be part of the metadata of the image itself. Not sure if this helps.
I have never used birdweather directly myself, so I don't know how these fields are displayed inside the application itself?
On the birdweather website clicking on the 🛈 icon next to the bird thumbnail takes you to a page with information about the species, a larger image and image credit link at the bottom of the page. e.g. https://app.birdweather.com/species/golden-whistler
In the birdnet iOS app clicking on the thumbnail takes you to the Wikipedia article.
I have never used birdweather directly myself, so I don't know how these fields are displayed inside the application itself?
On the birdweather website clicking on the 🛈 icon next to the bird thumbnail takes you to a page with information about the species, a larger image and image credit link at the bottom of the page. e.g. https://app.birdweather.com/species/golden-whistler
In the birdnet iOS app clicking on the thumbnail takes you to the Wikipedia article.
Ok that looks great! Did some work today and managed to parse out the owner of the image and license itself:
will have to do some more code improvement and management of edge cases before I update the pr :)
looks good
Finally managed to get something kind of ready. Let me tell you, getting the author information out of Wikipedia is very tricky since everyone is using different formats for the author block. Some just have their names, others have URLs, and then there are even some crazy people with tables and lots of links... I believe what I have now covers most of the different scenarios quite well!
Recent detections view
Daily summary view
I am a bit unsure about the daily summary view since the credits do not really fit into the table itself. Any suggestions on how we could format it in a nicer way there?
Anyways for those of you who want to try it out, feel free to either build this PR yourself. Or grab the image itself from my fork:
docker run ghcr.io/iszumpo/birdnet-go:f68715d97dddf310e05c525d4eba281f52f06dec ...
Is this ready for merge? We can tweak things further later
Is this ready for merge? We can tweak things further later
Yeah, you are right let's not try to get everything perfect on the first try. There are observed issues for certain birds where Wikipedia does not return any "machine-readable" data. I believe that should be fixable without too much trouble in a future PR. Furthermore, it does not cause much more trouble than a missing image.
Since there are mentions of birdweather etc in the previous commits of this PR. Perhaps it is best to merge it by squashing?
"A picture is worth a thousand words"
I though it could be nice to have some images of the detected birds since I am really bad at bird names... I saw that the BirdWeather GraphQL API offers endpoints to receive thumbnail for a given bird. In this pull request, I have started on a very crude implementation for fetching these thumbnails and adding them in some of the html views.
Daily summary view:
Recent detections view:
Of course the images could be placed better inside the html view, I just did not spend too much time on it.
In addition to only fetching the urls, a in memory caching mechanism is utilized. The cache saves the result for each bird after each request, such that the next request for that bird will be received from the cache instead. In addition, during startup, the cache will be initialized with every detected bird in the database.
I am a bit unsure of where to place these functions. Maybe it would be better to put these inside the birdweather internal/birdweather folder instead? Please give me some advice, and I will do my best to update the pull request accordingly.