A stateful, distributable, real-time (aside from some mandated polling),
highly observable data-streaming web service that continuously analyzes
r/WhatsThisBird activity. munin
's primary responsibility within the overall
FileTheseBirds
application is to generate community-vouched deductions to be
persisted to a database, but it also doubles as a Reddit bot that leaves a
comment trail of its
internal state.
munin
's runtime is a Swim server, and we
rely entirely on the Swim platform to achieve the italicized properties. The
source code and documentation here is mostly driver- and "business
logic"-focused; refer to Swim's guides if you wish to learn more about the
runtime.
A public instance of munin
is available at munin.swim.services
over both
the secure warps
and the warp
protocols.
src/main/resources/reddit-config.properties
file that looks like:
clientId=p-jcoLKBynTLew
clientSecret=gko_LXELoV07ZBNUXrvWZfzE3aI
redditUser=reddit_bot
redditPass=snoo
userAgent=FileTheseBirds/0.1.0 by reddit_bot
All credentials above are fake, copied from the linked tutorial.
src/main/resources/ebird-taxa.csv
file, where each row's first
column indicates an eBird taxonomy code, and the second column is the desired
common name for that code. If we use US-locale English common names, then the
first several rows will look like:
ostric2,Common Ostrich
ostric3,Somali Ostrich
y00934,Common/Somali Ostrich
grerhe1,Greater Rhea
lesrhe2,Lesser Rhea
src/main/resources/reviewers.txt
file containing the lowercase
usernames of reviewer-privileged Reddit accountssrc/main/resources/nonparticipants.txt
file containing the lowercase
usernames of accounts whose posts/comments should never be analyzedTODO:
readonly
mode that hits www.reddit.com
instead of
oauth.reddit.com
. Such a mode doesn't require prerequisites 2 and 3 but has
two restrictions:
Gradle 7+ is required to build munin
from source. You
may use the provided wrapper scripts to circumvent manually installing Gradle
on your machine, e.g.
./gradlew build
on Unix-like systems, and
.\gradlew.bat build
on default Windows shells.
gradle run
or ./gradlew run
/ .\gradlew.bat run
if using the wrapper scripts. Incurs
overhead due to management by a Gradle daemon.
gradle build
with command line runnerPrerequisite: a way to either untar or unzip an archive
build
commandsbuild/distributions
, (either the
tarball or the .zip)$UNPACK-DIR/bin
.Prerequisites: dpkg
, service
gradle packageDeb
(or a wrapper script variant)sudo dpkg -i build/distributions/munin-$VERSION_all.deb
sudo service munin start
TODO as this project matures
munin
exposes several streaming endpoints via its Web Agents. Below, we list
the ones that you are most likely to find useful. munin
delivers only the last
36 hours worth of r/WhatsThisBird activity in all of these APIs, but will be
processing far more than that under the hood.
We use swim-cli
commands in the following examples for simplicity; these may
just as easily be translated into downlinks. Be sure to replace
warps://munin.swim.services
with the host under which your munin
instance
runs, e.g. warp://localhost:9001
.
In the context of this API, the "status" of a submission is defined to be a flattened union of its info and (if it's nonempty) in-progress answer.
% swim-cli sync -h warps://munin.swim.services -n /submissions -l statuses
@update(key:"/submission/xwgwh6")@status{id:xwgwh6,title:"What species are these? In eastern washingon and I saw one jump",location:"north america",thumbnail:"https://b.thumbs.redditmedia.com/svwmIdeX0vhhgUT_sYSXc1SJ-jMizyMUJEr7hg9HFGE.jpg",createdUtc:1664991193,karma:8,commentCount:5,taxa:{bkbmag1},reviewers:{brohitbrose}}
@update(key:"/submission/xwgwj0")@status{id:xwgwj0,title:"Anyone know what this feather could belong to? West Michigan, USA.",location:"north america",thumbnail:"https://a.thumbs.redditmedia.com/_gmhGQJpMM7R04UigBVFsVkXY7h7fh0dPhT_CtnvHo8.jpg",createdUtc:1664991196,karma:14,commentCount:5,taxa:{wiltur},reviewers:{tinylongwing}}
@update(key:"/submission/xwgz1y")@status{id:xwgz1y,title:"Bird found in Tom Green County",location:"north america",thumbnail:"https://a.thumbs.redditmedia.com/C8f_G4ZEs2pDdWQRA8MmKOjnTb24kYRUi5M6E2vJ2w8.jpg",createdUtc:1664991356,karma:8,commentCount:3,taxa:{sora},reviewers:}
...
% swim-cli sync -h warps://munin.swim.services -n /submissions -l unanswered
@update(key:xwaj1m)@status{id:xwaj1m,title:"Can anyone ID this strange bird sound?",location:unknown,thumbnail:"https://b.thumbs.redditmedia.com/xhMyBWKfdQzhWKITLRHHYlLF_wQAqHkFGPBtHwSVvjo.jpg",createdUtc:1664976070,karma:7,commentCount:3,taxa:,reviewers:}
@update(key:xwxh4k)@status{id:xwxh4k,title:"Not a usual bird we see in the city. Location: Philippines",location:"southeast asia",thumbnail:"https://b.thumbs.redditmedia.com/zpt1ueqL9S1N6Pb-e2x0ImGNUrh8f-Z5wzVp-mb2z0A.jpg",createdUtc:1665035125,karma:13,commentCount:11,taxa:,reviewers:}
@update(key:xx19q7)@status{id:xx19q7,title:"ID request: Eastern Alps in Switzerland, dropped out of the sky",location:europe,thumbnail:"https://b.thumbs.redditmedia.com/LG8Rr7xkdLEiFg5FwGTNtU2hV2OQ5IRbZcH6_IlKlcw.jpg",createdUtc:1665049854,karma:15,commentCount:0,taxa:,reviewers:}
...
The answered
, unreviewed
, and reviewed
lanes are similarly available.
% swim-cli sync -h warps://munin.swim.services -n /submissions -l answers
@update(key:"/submission/xxkv53")@answer{taxa:{comyel}}
@update(key:"/submission/xxkwcq")
@update(key:"/submission/xxldl3")@answer{taxa:{grbher,bcnher,greegr}}
@update(key:"/submission/xxlwlz")@answer{taxa:{babwar},reviewers:{broken_faaace}}
...
NOTE: All subsequent commands will return nothing if the submission being inspected is over 36 hours old.
% swim-cli sync -h warps://munin.swim.services -n /submission/y0drr3 -l info
@submission{id:y0drr3,title:"Bird ID help! Northern Illinois",location:"north america",thumbnail:"https://b.thumbs.redditmedia.com/dlK6Ls1MWwjBIYsfby4T4vXbw7cVj0oXAbJW9EkO2Ac.jpg",createdUtc:1665405563,karma:32,commentCount:8}
% swim-cli sync -h warps://munin.swim.services -n /submission/y0drr3 -l answer
@answer{taxa:{eursta,rebwoo}}
% swim-cli sync -h warps://munin.swim.services -n /submission/y0drr3 -l status
@status{id:y0drr3,title:"Bird ID help! Northern Illinois",location:"north america",thumbnail:"https://b.thumbs.redditmedia.com/dlK6Ls1MWwjBIYsfby4T4vXbw7cVj0oXAbJW9EkO2Ac.jpg",createdUtc:1665405563,karma:31,commentCount:8,taxa:{eursta,rebwoo},reviewers:}
% swim-cli sync -h warps://munin.swim.services -n /submission/xvtxqc -l motions
@update(key:{1664927079,ir33kwk})@review(brohitbrose){plusTaxa:{leasan,y00496}}
@update(key:{1664997555,ir6sjgg})@review(haematopuspalliatus)
@update(key:{1664997652,ir6ssw9})@review(brohitbrose)
@update(key:{1665007491,ir7i1d4})@review(haematopuspalliatus){overrideTaxa:{leasan,semsan}}