search

cosmir / openmic-annotator

Annotation framework for annotating data for OpenMIC
MIT License
56 stars 1 forks source link

AudioNet initiative #21

Closed informusica closed 7 years ago

informusica commented 7 years ago

Hi all,

Sharing some intelligence from the Multimedia community (with the ACM Multimedia conference starting in a few days, from Oct 15-19): the Multimedia Commons people, who have been annotating the YFCC100m dataset, now want to focus on audio annotations. They're calling this sub-initiative 'AudioNet'...I guess it's clear what example they've been thinking of here. :)

Even though I'm not quite sure how many items in YFCC100m will feature musical instruments, it might be worthwhile to bring up Open-MIC and some wishlist items related to the initiative (and, well, with 800,000 videos in that dataset...there might at least be a few hits). YFCC100m also strongly had departed from the wish to have open and web-scale data for evaluation and benchmarking, which aligns nicely to COSMIR.

First way to give input on where AudioNet should head is through the survey (see http://mmcommons.org/audionet). They also will do on-site informal brainstorming over coffee at ACM Multimedia on Tuesday. It looks like @superbock will be on-site, as madmom is one of the Open Source Software Competition entries (congrats! :))...Sebastian, if you're free on Tuesday morning, maybe you could join there?

As for me, I'm on the organization of the MM conference and a participant in the MediaEval benchmarking workshop right afterwards, where @julian-urbano will be. Due to parallel ongoing teaching obligations, I won't manage joining the AudioNet coffee brainstorm session myself...but for the rest, I'd be happy to represent any Open-MIC/broader COSMIC wishes in person where I can, if you can help me with an update on the current status quo and what to stress (haven't been as active myself yet here...and will remain quite a bit 'in service' in the coming days, so may not yet manage catching up fully to all discussions that were ongoing).

Of course, I'd also be happy to also connect Julián to any Multimedia Commons colleagues that still will be around at MediaEval. Just let me know!

superbock commented 7 years ago

I will join the AudioNet coffee at ACM Multimedia, so if you would like me to bring some specific things/topics up there, just let me know.

ejhumphrey commented 7 years ago

​Thanks for circulating! Also relevant to the conversation is the SONYC project [https://wp.nyu.edu/sonyc/] from NYU and some work led by Emilia at MTG with YouTube annotations ... I think this is the right link: https://portal.upf.edu/web/mdm-dtic/projects/-/asset_publisher/Ef1was9TxNY4/content/id/4035220#.V_-QzZMrJE4

As for an update, I feel like we're approaching a good point for some kind of broad update / newsletter, so I'll see if I can't put something together in the next day or two.

julian-urbano commented 7 years ago

I won't be at ACM MM; only at MediaEval. It'd be nice to know what kind of tags they'll look for. I guess there're lots of people playing in front of the camera, so maybe instrument tags will be there. In any case, yes to everything. Yes, and please :smiley:

ejhumphrey commented 7 years ago

@superbock, i realize it's been a while, but was there any updates re: audionet worth reporting here?

related, I wonder if we could define a done state for this issue? summarize their goals, similarities, differences?

superbock commented 7 years ago

Yes, it's been a while and I am still waiting for a response/mailing of them...

To summarise so far: it was a loose meeting with a bunch of people, lead by Julia Bernd (at ICSI Berkely). She was mainly asking what people would need or aim for if such a corpus was available.

The Multimedia community is not so much multimedia, I'd call it rather single/dual-media. It seemed to me they are only interested in moving/still images and things you can extract from these (e.g. objects, people, emotions, etc.). If there was audio involved it was mostly audio scene classification and the like. It was rather hard to spot any music related stuff and people.

That said, most people are interested in stuff like audio scene classification. I added a couple of things of interest to us, such as information about music, instrumentation, voice, etc. But I fear they will only be available on a clip-level. Still, everyone can take their survey here.

They struggle with /discuss the same technical questions as we do here. Maybe we (i.e. you guys doing an excellent job!) made even some more progress, they could benefit from.

TL;DR, the project is in a similar state, aims for a much broader audience focussing more on video and images rather then audio and -- to an even lesser extent -- music.

ejhumphrey commented 7 years ago

awesome, thx for the comprehensive update! sounds like we should try to start / maintain a dialog with them as things progress here