CNMAT / CNMAT-MMJ-Depot

CNMAT's expanding library of Max/MSP/Jitter patches
90 stars 17 forks source link

music 29 lab 8: frequency domain model builder #55

Closed EdmundCampion closed 5 years ago

EdmundCampion commented 9 years ago

This is assignment to build a Spectral Modeling patch/app for Music 29 SP2015. The due date is no later than February 20th, 2015.

Here are basic details, but please sit down with Ed to go over everything before starting.

Description:

Load a soundfile of a single instrumental sample playing one note (e.g. a Sather Bell strike or a Saxophone playing G etc..). Playback of the sample should be on separate gain, have loop/no loop, have mute/unmute button.

Prefader gain from sample is sent to sonogram analysis onboard with patch.

Using sonogram analysis and probing sinusoid locate specific frequencies inside the sample. (This will entail going back to frequency domain analysis patch in MUSIC 29 and working out some better and more refined way to capture probed frequencies)

Once a frequency is located it gets added to an individual frequency control track that can post the frequency selected, play the frequency constantly, have full gain control, and have a mute/unmute button. Output should be mono.

Once many frequency are located, all the frequencies should appear as a mixing board, allowing for fine tuning of amplitudes and control of every frequency found. (maybe an upper limit of 20 frequencies?).

The mixed output of all frequencies and the original source sample should be sent to a dual master mono output, one for sinusoids, one for sample. The normal default of the master mono outpust gain is a slider output for each. The sinusoid master output will have a button that selects an ADSR feature. When the ADSR component is checked, then a window is enable to create a specific envelope that will auto match to the length of the currently loaded soundfile and will trigger each time the loaded soundfile begins.

adrianfreed commented 9 years ago

According to some sources sonogram is the old world for spectrogram.

<research related aside that could be a distraction: Also note that old lore from Helmholtz suggests relative phase of partials is not audible. This was debunked in 1947 and regularly ever since. I am working with Meyer Sound on this because it is of great interest to John Meyer and David set them on an interesting path towards research on it. From the musical perspective this phase thing is interesting because one of the things that happens when you change the relative phase of partials is that you modulate the roughness of the perceived sound. This may have consequences for inharmonic sounds such as bells.

analyzer~ is buggy and I have no ETA for a fix so try to avoid it for the analysis side.

It sounds like the model being fitted here supposes that the original signal has fixed frequency partials.

My approach would be as follows:

Giant FFT on the whole file Identify the 20-30 largest peaks.

Feed the sound BACKWARDS into a bank of resonators tuned to the frequencies of those peaks with relatively short decay rates. Estimate the energy out of each filter using RMS or something similar. Reverse the resulting tracks back to the right temporal order.

You can fit the ADSR to these tracks, turn them into SDIF or whatever.

The reason for doing this in reverse is that it avoids ringing of the resonators messing up the attack.

Lots of us have had a hand at this sort of thing (Miller, Ali, John MacCallum, MichaelZ etc.) so there may be some value in collecting up the various approaches.

On Dec 14, 2014, at 1:33 PM, Edmund Campion notifications@github.com wrote:

This is assignment to build a Spectral Modeling patch/app for Music 29 SP2015. The due date is no later than February 20, 2015.

Here are basic details, but please sit down with Ed to go over everything before starting.

Description:

Load a soundfile of a single instrumental sample playing one note (e.g. a Sather Bell strike or a Saxophone playing G etc..). Playback of the sample should be on separate gain, have loop/no loop, have mute/unmute button.

Prefader gain from sample is sent to sonogram analysis onboard with patch.

Using sonogram analysis and probing sinusoid locate specific frequencies inside the sample. (This will entail going back to frequency domain analysis patch in MUSIC 29 and working out some better and more refined way to capture probed frequencies)

Once a frequency is located it gets added to an individual frequency control track that can post the frequency selected, play the frequency constantly, have full gain control, and have a mute/unmute button. Output should be mono.

Once many frequency are located, all the frequencies should appear as a mixing board, allowing for fine tuning of amplitudes and control of every frequency found. (maybe an upper limit of 20 frequencies?).

The mixed output of all frequencies and the original source sample should be sent to a dual master mono output, one for sinusoids, one for sample. The normal default of the master mono outpust gain is a slider output for each. The sinusoid master output will have a button that selects an ADSR feature. When the ADSR component is checked, then a window is enable to create a specific envelope that will auto match to the length of the currently loaded soundfile and will trigger each time the loaded soundfile begins.

— Reply to this email directly or view it on GitHub.

EdmundCampion commented 9 years ago

This brings us back to a need to discuss further the Depot software aesthetic. Adrian, let's plan some discussion about this sort of Music 29 assignment on Wednesday...

This one is about musicianship -- the students need to match frequencies in the source sample. It is not really an analysis-based modeling problem.

We already have a number of tools in the MU29 suite that fit together for this assignment and serve the need with little additional work required. Mainly a GUI thing.

equilet commented 9 years ago

I think the markermanager might be of some assistance here. This could function as an overlay for the spectroscope~ object. This user can watch the peak amplitudes of the sound as played back, click on the relevant portion of the spectroscope~, and as they click to add / subtract frequencies they are spawned within a poly~ instance. I'll meet with you about this in the upcoming week, as I want to make sure I have a clear picture of what you are asking.

https://github.com/CNMAT/CNMAT-MMJ-Depot/blob/master/javascript/markermanager.maxhelp https://github.com/CNMAT/CNMAT-MMJ-Depot/blob/master/javascript/markermanager2.maxhelp

equilet commented 9 years ago

note: consider freq domain analysis as a plunder-space for this.

equilet commented 9 years ago

OK, this is in the depot now, and included in the overview. @mattschumaker @EdmundCampion can you please test and let me know how you find the functionality. I'm certain that it will need labels, etc.

Don't we all. :)

mattschumaker commented 9 years ago

Hi Jeff,

This is what I am getting so far. I had two crashes on the standalone. Pls. see below.

Program looks beautiful. Loading and playing back sounds works. Playback looping works. Directions are clear.

While following directions, standalone program crashed on me twice. Before crashing, gave me these errors

Error: playlist~: couldn't find /Users/jml/Documents/CNMAT/repositories/CNMAT-Production/campion/natural_frequencies/lib/audio/SatherBells2014_Samples-ord/2_02bell.aif

errors when trying to press range area: "address is unbound: length: address /markers is unbound error evaluating expression: "

errors when moving a partial id: address is unbound: scale: address /tempmarks is unbound error evaluating expression:

Recommend labels (as you suspected…however with a couple of labels would be very functional for class next week bk gsis can explain.) label for range area label for partial ids & sliders possible labels for sound file volume and model volumes--but these are explained in the help file.

equilet commented 9 years ago

Please provide a list of steps to reproduce the crash. Thanks!

mattschumaker commented 9 years ago

Started app again. These steps led me to a crash:

--Open application

~~--see error for missing sound on max console: playlist~: couldn't find /Users/jml/Documents/CNMAT/repositories/CNMAT-Production/campion/natural_frequencies/lib/audio/SatherBells2014_Samples-ord/2_02bell.aif~~

--open subpatcher quickinfo

--drag sound file z14 bell to playlist area. (This is an .aiff file)

--turn on audio

--turn up sound file volume slider

--hit loop sound.

--working on following further directions when the app crashed. The short sound looped approximately five times. In the past, I had been able to do more, like select a range button on the right, before app crashes.

equilet commented 9 years ago

I can't get this to crash. Will follow up with you in person.

equilet commented 9 years ago

Matt and I tested this, got the crash to happen. I isolated it down to either a crash on cycling's part, or a memleak that odot exhibited a while ago. Either way, the crash doesn't occur any more... I'll follow up for my own edification, but for now this is sorted out and the model builder will be deployed to the lab. Rev 1 is finished and annotated with badge/banner/comments/labels. Rev 2 or any further work will be another issue.

EdmundCampion commented 9 years ago

Looking over the Freq Model builder standalone. Thanks Jeff for some very nice improvements in GUI on that. There are a few issues to deal with.

Please change the name of the 29 standalone and Depot patch to: Freq_Model_Builder

• always include a link to the DSP Settings • and always include a background default message for those settings.

• For the "audio button" make sure there is some evidence that the audio is running as we have been doing.

Both of these problems above were solved with the Audio I/O. Right now, the audio button is initiated as "green" which to me means "on".

• if possible implement a way to jump the start point of the audio playback. • implement a "pause" "resume" soundfile playback with keyboard space bar • if possible implement a loop feature so that a particular section can be selected and looped. • As mentioned before, we will need to add for each of the sinusoids. That should be posted right next to the sinusoid numbering that is so nicely implemented in the tool.
• add a "write" to save out the data to text file. See if you can make it so that you have two lists -- one with and a second that is formatted for direct input into sinusoids~

• if possible, the resizing of the sonogram should also reposition the sinusoid freq/gain pots to the right position.

• As mentioned, we need the "nudge" tool for finegrain adjustment of sinusoids

• please make sure that stereo files are being summed and not just playing L or R.

If the user changes the number of sinusoids then they all burst onto the audio scene really loudly. Should mute everything if this button is touched.

mattschumaker commented 9 years ago
  1. Rafael pointed out one repeatable bug which will not affect labs this week: In the soundfile window, if you press "clear" while the sound is looping then standalone will crash. Easy to avoid this for the moment.
  2. leds or metering for audio outputs on top right
  3. change audio on/off colorization to be more clear
  4. when you change the spectrum range, should the window size stay the same, allowing the partials to more clearly shift? examine
  5. there is a strange thing happening whereby you can drag in multiple mono files into the playlist... see if there is a way to lock user out of that. if not, just revert to using the waveform~ object
  6. more contrast between partial marker color and spectrum below
equilet commented 9 years ago

background dsp initialization, nudging, define start point, and loop were already implemented.