search

mvcisback / SSLVC

Sound Source Localization using Visual Cues
4 stars 1 forks source link

Project Phase 1 #14

Closed ghost closed 9 years ago

ghost commented 9 years ago

For next week meeting, I think this should be doable.

1- Detect Voice (High energy for each frame in the spectrogram, might have to do something smoothing) 2- Assign Frame(s) to your detected voice ( map audio frames to video frame, assuming synchronization) 3- Find pixels in those(that) frames that corresponds to the voice (frame subtraction and thresholding) 4- Try different colormaps 5- Try PCA and different number of dimensions on the frames 6- Spatial audio for detected voice ( how the person is talking and where is he in the original frame).

How does this sound to you guys?

ghost commented 9 years ago

I was trying to notify you, but apparently that's not what assigning is for.

mvcisback commented 9 years ago

@ramili, use the @ + user name to notify me or @ffaghri1

mvcisback commented 9 years ago

Also, sounds good to me. I'll try playing around with it once I finish the homework

ghost commented 9 years ago

@ffaghri1 let me know if this sounds doable to you as well

ghost commented 9 years ago

I tried all the colormaps, you can only get the mouth movement if you zoom in to those pixels. Threshholding might help, like if the pixels are too far apart ignore them or sth

ffaghri1 commented 9 years ago

It all sounds doable. @ramili, do you mean you just went through all steps for next week?

On Wed, Oct 29, 2014 at 10:18 PM, ramili notifications@github.com wrote:

Already tried for all cases, you can only get the mouth movement if yo zoom in to those pixels. Threshholding might help.

Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61040600.

ghost commented 9 years ago

@ffaghri1 yes :p I know @mvcisback is on an interview, but if you got sometime tomorrow, I can show it to you. I didn't try PCA for reducing dimensions, it didn't look like it would be of any use. I just try different colormaps

ffaghri1 commented 9 years ago

@ramili nice job, I am exciting to see how they look. Would 11 AM tomorrow work for you?

ghost commented 9 years ago

Sure, same place 11. I put some explanation about the data format somewhere under the issues, resize video 1 ....

On Wednesday, October 29, 2014, Faraz Faghri notifications@github.com wrote:

@ramili https://github.com/ramili nice job, I am exciting to see how they look. Would 11 AM tomorrow work for you?

— Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61042990.

Thanks, Best Regards, Ramin

ffaghri1 commented 9 years ago

Could we meet at 12 instead? On Oct 29, 2014 11:58 PM, "ramili" notifications@github.com wrote:

Sure, same place 11. I put some explanation about the data format somewhere under the issues, resize video 1 ....

On Wednesday, October 29, 2014, Faraz Faghri notifications@github.com wrote:

@ramili https://github.com/ramili nice job, I am exciting to see how they look. Would 11 AM tomorrow work for you?

Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61042990.

Thanks, Best Regards, Ramin

Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61045857.

ffaghri1 commented 9 years ago

Sorry, there is a talk then I want to attend. Are you free anytime after 3 pm? On Oct 30, 2014 10:18 AM, "Faraz Faghri" faraz.faghri@gmail.com wrote:

Could we meet at 12 instead? On Oct 29, 2014 11:58 PM, "ramili" notifications@github.com wrote:

Sure, same place 11. I put some explanation about the data format somewhere under the issues, resize video 1 ....

On Wednesday, October 29, 2014, Faraz Faghri notifications@github.com wrote:

@ramili https://github.com/ramili nice job, I am exciting to see how they look. Would 11 AM tomorrow work for you?

Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61042990.

Thanks, Best Regards, Ramin

Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61045857.

ghost commented 9 years ago

sure, I should be free at 3 for about half an hour or so.

On Thu, Oct 30, 2014 at 10:21 AM, Faraz Faghri notifications@github.com wrote:

Sorry, there is a talk then I want to attend. Are you free anytime after 3 pm?

On Oct 30, 2014 10:18 AM, "Faraz Faghri" faraz.faghri@gmail.com wrote:

Could we meet at 12 instead? On Oct 29, 2014 11:58 PM, "ramili" notifications@github.com wrote:

Sure, same place 11. I put some explanation about the data format somewhere under the issues, resize video 1 ....

On Wednesday, October 29, 2014, Faraz Faghri notifications@github.com wrote:

@ramili https://github.com/ramili nice job, I am exciting to see how they look. Would 11 AM tomorrow work for you?

Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61042990.

Thanks, Best Regards, Ramin

Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61045857.

— Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61110725.

Thanks, Best Regards, Ramin

mvcisback commented 9 years ago

Hey if you guys met, could you give a quick summary for me?

Also, does one of you mind taking notes in lecture for me? His slides are good, but he often leaves out intuitions and factoids

ghost commented 9 years ago

I'm gonna be late for class so I'm planning to ask someone to record the session for me, if that happened I can give you my recordings. Let me write up a draft of what we discussed and post it later on the weekend. Meanwhile Fraz took a picture of what we discussed that he can share with you.

On Friday, October 31, 2014, Marcell Vazquez-Chanlatte < notifications@github.com> wrote:

Hey if you guys met, could you give a quick summary for me?

Also, does one of you mind taking notes in lecture for me? His slides are good, but he often leaves out intuitions and factoids

— Reply to this email directly or view it on GitHub https://github.com/mvcisback/SSLVC/issues/14#issuecomment-61263719.

Thanks, Best Regards, Ramin

mvcisback commented 9 years ago

So from #16. @ramili said that phase 1 was a failure. Could we get you to expand on that here?

mvcisback commented 9 years ago

We don't have enough resolution to track lip movement. Instead moving to black box face detection system for phase 2