search

mixxxdj / mixxx

Mixxx is Free DJ software that gives you everything you need to perform live mixes.
http://mixxx.org
Other
4.46k stars 1.27k forks source link

Make waveform scratchable / drag like a PDF #5090

Closed mixxxbot closed 2 years ago

mixxxbot commented 2 years ago

Reported by: depicco Date: 2009-02-04T16:25:35Z Status: Fix Released Importance: Wishlist Launchpad Issue: lp325377 Tags: waveform Attachments: waveform_scratch.patch, [Apply this patch after RJ's patch](https://bugs.launchpad.net/bugs/325377/+attachment/2023743/+files/Apply this patch after RJ's patch), mouseScratch.patch, mouseScratch.patch, mouse_scratch2.patch, [Patch 1](https://bugs.launchpad.net/bugs/325377/+attachment/2090625/+files/Patch 1), [patch 2, as discussed in post mixxxdj/mixxx#4929](https://bugs.launchpad.net/bugs/325377/+attachment/2090636/+files/patch 2, as discussed in post mixxxdj/mixxx#4929), mouseScratch_zoomableWavefrom2.patch, mouseScratch2.patch, [Patch to return old drag to change rate/speed, to the right mouse button](https://bugs.launchpad.net/bugs/325377/+attachment/2207576/+files/Patch to return old drag to change rate/speed, to the right mouse button), [pitch-bend on right-click drag on waveform](https://bugs.launchpad.net/bugs/325377/+attachment/2632186/+files/pitch-bend on right-click drag on waveform)

(once again, this is a feature request) at first i found the drag wave method to be a little annoying, but now i find it useful; but i find more natural to drag the vae in the wave display like it was something i actually drag (not a slider to change the speed). it would be more like dragging a pdf document. i click somewhere on the wave, then i drag this wave according to the displacement of the mouse pointer, it does not keep moving if i am not moving the pointer, and the drag distance of the wave is the same as the x movement of the mouse, it would move in the same direction (unlike now, if i move the pointer forward the track moves forward, but the wave picture moves the other way). if i want to go somewhere fast, i use the full wave display. (it would be a 'drag mode option', not to replace the method used now but to have it as an option)

tnx

mixxxbot commented 2 years ago

Commented by: Pegasus-RPG Date: 2009-03-17T12:52:26Z

I would prefer this too. It would make scratching with the mouse MUCH more intuitive.

mixxxbot commented 2 years ago

Commented by: smashuu Date: 2010-01-11T18:41:23Z

Perhaps the left mouse button could work as it does currently, and the right button (which doesn't seem to currently affect the waveforms) could drag the wave directly as described above?

mixxxbot commented 2 years ago

Commented by: rryan Date: 2010-11-25T01:14:05Z

Skye has already done a lot of this work in her branch.

Currently it uses the 'scratch' control. I think 2 things need to happen for it to be part of 1.10: 1) Switch to using scratch2 2) Add filtering to reduce rate warbling.

Her work also includes the ability to 'flick' the waveform which is nice because it lets you emulate a backspin.

mixxxbot commented 2 years ago

Commented by: rryan Date: 2011-04-11T03:14:20Z Attachments: waveform_scratch.patch

Here's my patch that adds position control to the engine. MIDI controllers and GUI widgets can simply set scratch2_position to the desired position they'd like to go to, and a PID controller in RateControl will control the rate of the deck such that it quickly approaches the position.

Using my patch, hold down the right mouse button on the waveform to enable scratching on it.

Right now its pure rate control, so there are many discontinuities in the rate values. This leads to a 'popping' sound when moving the waveform slowly. I think the right thing to do to fix this is to add second-order control -- make the PID controller control the deck acceleration only instead of velocity. This should smooth out discontinuities, but may make the scratching less responsive.

Right now only proportional and differential control on the PID controller are used.

Another WTF, you should only load 44.1KHz tracks to test this out. There are some dumb constants that I haven't adjusted for different track sample rates yet.

mixxxbot commented 2 years ago

Commented by: esbrandt Date: 2011-04-11T16:01:47Z

Hi RJ, thanks for working on this, some comments about your latest patch.

1 - Change the direction Currently, if you drag the mouse in the playing direction the track goes backwards. This is counterintuitive and contradicts the behavior of a vinyl on a turntable.

2 - Change the resolution Currently, 100 pixel mouse movement translate into 1 second of music at default waveform zoom level which is way to sensitive. A competitor uses 50 pixel per second, while another uses 25 pixel. Personally i found 25 pixel works best for scratching with mouse/pen/trackpad.

3 - Make it default I vote the scratch behavior to be the new default while the current click-and-drag-to-control-playback goes right-click option. This would make Mixxx running on touchpad device behave more natural. Moreover, it is useful for trackpads and similar input devices.

mixxxbot commented 2 years ago

Commented by: esbrandt Date: 2011-04-11T16:07:14Z

Correction for bullet point 2 - i mixed up second/tenth of a second.

2 - Change the resolution Currently, 100 pixel mouse movement translate into 1 second of music at default waveform zoom level which is way to sensitive. A competitor uses 500 pixel per second, while another uses 250 pixel. Personally i found 250 pixel works best for scratching with mouse/pen/trackpad.

mixxxbot commented 2 years ago

Commented by: rryan Date: 2011-04-11T16:35:08Z

Thanks jus,

1) Yup, sounds good.

2) Interesting, I think the initial goal was to make mouse-movement / waveform be 1 to 1. I can easily make it less sensitive.

3) I'm fine w/ this. I think the normal dragging is not very useful too.

Also, Sean gave this a shot on his SCS1d and mentioned it isn't very good for scratching there yet. I think the problem there is the discontinuous change in velocity. I need to experiment more w/ my controllers to see figure out a better solution.

RJ

On Mon, Apr 11, 2011 at 12:07 PM, jus

Correction for bullet point 2 - i mixed up second/tenth of a second.

2 - Change the resolution Currently, 100 pixel mouse movement translate into 1 second of music at default waveform zoom level which is way to sensitive. A competitor uses 500 pixel per second, while another uses 250 pixel. Personally i found 250 pixel works best for scratching with mouse/pen/trackpad.

-- You received this bug notification because you are a member of Mixxx Development Team, which is subscribed to Mixxx. https://bugs.launchpad.net/bugs/325377

Title: Make waveform scratchable / drag like a PDF

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-04-12T11:34:35Z Attachments: [Apply this patch after RJ's patch](https://bugs.launchpad.net/mixxx/+bug/325377/+attachment/2023743/+files/Apply this patch after RJ's patch)

WOW RJ! you are my hero ;-)

Using my approach engine tends to drop some values when mouse changes direction of motion very fast. But your patch works just flawlessly!

Btw, I managed to add some inertia, correct the direction (point 1 of Jus) and handle the rate as zoom changes with pitch

I have attached my patch.

Thanks a lot :-) shanx

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-04-13T01:12:27Z Attachments: mouseScratch.patch

This patch is against trunk.

New features:

  1. Improved inertia In my last patch, if track is paused, waveform stopped with a jerk Now if play button is not pressed waveform doesn't stop with a jerk

  2. Some fancy stuff

    • A hand appears while mouse scratching
    • Mouse motion (in X direction) is restricted to waveform display only

What other things need to be addressed before mouse scratching really makes it to trunk?

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-04-13T01:14:11Z Attachments: mouseScratch.patch

This patch is against trunk

  1. Improved inertia In my last patch, if track was paused, waveform stopped with a jerk, now if play button is not pressed waveform doesn't stop with a jerk

  2. Some fancy stuff

    • A hand appears while mouse scratching
    • Mouse motion (in X direction) is restricted to waveform display only

What other things need to be addressed before mouse scratching really makes it to trunk?

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-04-13T01:15:22Z

sorry for double posts

mixxxbot commented 2 years ago

Commented by: esbrandt Date: 2011-04-14T20:04:20Z

@shanxS Had some problems applying your latest patch with trunk http://bazaar.launchpad.net/~mixxxdevelopers/mixxx/trunk/revision/2748

localhost:mixxx user$ patch -p0 < mouseScratch.patch
patching file ./src/engine/enginebuffer.cpp
Hunk mixxxdj/mixxx#4910 succeeded at 462 (offset 7 lines).
patching file ./src/engine/enginebufferscalelinear.cpp
patching file ./src/engine/enginechannel.cpp
Hunk mixxxdj/mixxx#4910 FAILED at 69.
1 out of 1 hunk FAILED -- saving rejects to file ./src/engine/enginechannel.cpp.rej
patching file ./src/engine/ratecontrol.cpp
patching file ./src/engine/ratecontrol.h
patching file ./src/widget/wglwaveformviewer.cpp
patching file ./src/widget/wglwaveformviewer.h
localhost:mixxx user$

But somehow the patch worked anyway.

Some feedback: I like that changed direction and the little hand, nice touch :-)

Restricting the mouse x-range to to waveform display`s size is not practical. While it might work ok on large parallel waveform displays it does not work well on small displays/ separated waveform display and is a potential source of unpredictable behavior.

It is difficult to do precise scratches. I have the impression that the waveform drifts because the inertia is alway "on". Waveform should strictly follow mouse movement according to an modified resolution ( see post #⁠5).

Inertia would be more realistic if only active when "throwing" the waveform in one direction.

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-04-18T17:49:55Z Attachments: mouse_scratch2.patch

Hey Jus, Thanks for feedback!

Since I am no DJ, I am relying on DJs in Mixxx community to point out short comings. :-)

This patch is against trunk #⁠2756, which by the time of this post is latest trunk. It reports 1 fuzz while applying the patch, but patch works (gotta run to class)

Restricting the mouse x-range to to waveform display`s size is not practical. While it might work ok on large parallel waveform displays it does not work well on small displays/ separated waveform display and is a potential source of unpredictable behavior.

I have scrapped off this feature.

It is difficult to do precise scratches. I have the impression that the waveform drifts because the inertia is alway "on".

that 'inertia' like effect is now taken care of, but it may cost a penalty if few milliseconds, this should not hurt, will it?

Waveform should strictly follow mouse movement according to an modified resolution ( see post #⁠5).

I have changed default resolution to 250 px per second, and motion is still one on one.

Inertia would be more realistic if only active when "throwing" the waveform in one direction.

'Inertia' nows comes in play when track is 'thrown'

please review this feature again :-)

Thanks shanx

mixxxbot commented 2 years ago

Commented by: esbrandt Date: 2011-04-19T08:11:14Z

Hi shanxS, your latest patch is a mayor improvement in timing and usability.

Precise scratches are no problem anymore, the realistic "rewind" effect when throwing the waveform backward is very nice. Also the changed default resolution allows to see the actual position more precisely. I like it, but it may not suit everybody. The best way would be to provide an option to zoom in/out the waveform: https://bugs.launchpad.net/mixxx/+bug/367151

Like said in post#⁠5 point 3, make "mouse scratching" default and swap it with the current default behavior.

One more thing: Sometimes the waveform is not in sync with the audio. E.g. you can scratch hear (and scratch) a basskick, but the waveform is several pixels ahead. Cant say if this based on your patch or a general flaw.

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-04-19T14:08:02Z

Hey Jus, It's good to know that you like my implementation :-)

make "mouse scratching" default and swap it with the current default behavior.

You mean I should swap right click with left click, consider it done!

The best way would be to provide an option to zoom in/out the waveform: https://bugs.launchpad.net/mixxx/+bug/367151

How would you like to zoom i.e., would you like to use mouse wheel, or a slider alongside of waveform (as in VirtualDJ) or some keyboard shortcut?

Sometimes the waveform is not in sync with the audio. E.g. you can scratch hear (and scratch) a basskick, but the waveform is >several pixels ahead. Cant say if this based on your patch or a general flaw.

I am looking into it, can't say for sure what is going wrong.

Thanks for reviewing! Shanx

mixxxbot commented 2 years ago

Commented by: esbrandt Date: 2011-04-19T16:14:02Z

The most practical way for waveform zoom would be a standard control that could be accessed like all the other controls by keyboard mappings, MIDI mappings and skins. http://mixxx.org/wiki/doku.php/midi_controller_mapping_file_format#ui_midi_controls_and_names A slider/stepper sounds nice, which could allow to zoom full in / full out in a few (4-5?) steps + reset to default zoom.

Once the control is in there, i`ll be happy to help updating the skins/keyboard mapping.

Thanks, jus

mixxxbot commented 2 years ago

Commented by: rryan Date: 2011-04-26T05:24:15Z

Hey all,

I cleaned up my patch and committed it to trunk. Shanx, while cleaning it up I made some reorganizations and it differs from your patch since it was based on my test one above. I ported most of the changes from your patch such as the hand-icon, the switching of directions, and I made it the default action (completely got rid of the old left-click behavior). I also hand-patched in your inertia changes and added some comments. They are in the PositionScratchController class in trunk (src/engine/positionscratchcontroller.cpp)

I think that we shouldn't add waveform zoom because it's already past feature freeze for the 1.10 branch and even if we did that would mean more work for Jus to add zoom buttons to the skins, which is probably not what he needs right now :).

Thanks, RJ

mixxxbot commented 2 years ago

Commented by: esbrandt Date: 2011-04-26T08:53:59Z

Tested trunk r2774 which included the waveform scratch feature. Compared to teh patch from comments mixxxdj/mixxx#4922 it is ( hate to say that ;-) a regression. 
Basically it is now again like in comment mixxxdj/mixxx#4921, difficult to do precise scratches. The waveform slightly drifts because it does not strictly follow mouse movement.

Also the inertia feature when throwing the waveform acts strange if used while playback is active. Play a track, do some scratches and then "throw" the waveform, the playback rate spins down to zero only to continue immediately with the original playback rate. like on a Direct Drive Turntable it would be realistic if the rate spins down to the original playback rate and continue playback from there. My gut feeling tells me the spindown period should be slightly longer then the current.

mixxxbot commented 2 years ago

Commented by: rryan Date: 2011-04-26T13:05:28Z

Oops :) I must have missed some of shanx's changes. I'll go through and double check later.

RJ

On Tue, Apr 26, 2011 at 4:53 AM, jus

Tested trunk r2774 which included the waveform scratch feature. Compared to teh patch from comments #⁠13 it is ( hate to say that ;-) a regression. Basically it is now again like in comment #⁠12, difficult to do precise scratches. The waveform slightly drifts because it does not strictly follow mouse movement.

Also the inertia feature when throwing the waveform acts strange if used while playback is active. Play a track, do some scratches and then "throw" the waveform, the playback rate spins down to zero only to continue immediately with the original playback rate. like on a Direct Drive Turntable it would be realistic if the rate spins down to the original playback rate and continue playback from there. My gut feeling tells me the spindown period should be slightly longer then the current.

-- You received this bug notification because you are a member of Mixxx Development Team, which is subscribed to Mixxx. https://bugs.launchpad.net/bugs/325377

Title: Make waveform scratchable / drag like a PDF

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-04-26T20:29:53Z Attachments: [Patch 1](https://bugs.launchpad.net/mixxx/+bug/325377/+attachment/2090625/+files/Patch 1)

Hey guys, Thanks for considering mouse scratching for Mixxx 1.10.0 :-) I am attaching 2 patches: Patch 1: mouseScratch.patch

Patch 2: mouseScratch_zoomableWaveform.patch

What I suggest is, lets use patch 2 and alter the skins for zoom buttons in Mixxx 1.11.0 or later, as situation demands. Or other way round is to use patch 1 and we can change default resolution to 250 px per second. What do you guys suggest?

Both patches are against trunk r2775 Launchpad is not allowing me to attach more than 1 patch, patch 2 is attachment of next post Regards shanx

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-04-26T20:34:20Z Attachments: [patch 2, as discussed in post mixxxdj/mixxx#4929](https://bugs.launchpad.net/mixxx/+bug/325377/+attachment/2090636/+files/patch 2, as discussed in post mixxxdj/mixxx#4929)

This is patch 2 as I discussed in post #⁠20 Just for record:

shanx

mixxxbot commented 2 years ago

Commented by: esbrandt Date: 2011-05-02T09:03:02Z

Tested again latest trunk http://bazaar.launchpad.net/~mixxxdevelopers/mixxx/trunk/revision/2783

The scratch function still behaves unnaturally - exact,real sounding scratches are not possible. How vinyl on a turntable sounds:

  1. If you move a vinyl forth and back, the sound stops exactly in the moment you stop the motion (fingers still holding the vinyl). The vinyl ,and as a result the sound coming from the speaker, follows exactly the fingers movement.
  2. If you push a vinyl forward or spin the record back, you get this typical sound we all know.

How Mixxx current implementation sounds in comparison:

  1. If you move the mouse pointer forth and back on the waveform, the sound stops NOT exactly in the moment you stop the motion (mouse still pushed down). The sound and waveform follows the mouse pointers movement delayed , inertia is activated where it has no business (see comment #⁠12). Patch from comment #⁠13 had these problems resolved.
  2. Realistic behavior, also it sounds good.

The patches: shanx patch1

Also the CPU goes crazy on full zoom out and it is noting to see from the waveform as the white beat marker lines fill up the whole space.

mixxxbot commented 2 years ago

Commented by: shanx-shashank Date: 2011-05-03T15:13:51Z Attachments: mouseScratch_zoomableWavefrom2.patch, mouseScratch2.patch

Hey Jus, Thanks for review!

How Mixxx current implementation sounds in comparison:

  1. If you move the mouse pointer forth and back on the waveform, the sound stops NOT exactly in the moment you stop the motion (mouse still pushed down). The sound and waveform follows the mouse pointers movement delayed , inertia is activated where it has no business (see comment #⁠12). Patch from comment

    ⁠13 had these problems resolved.

I have made changes, hope these change makes mouse scratching more realistic.

shanx patch2

  • If zoomable waveform gets implemented i think it is fair to scrap the simultaneous zoom on pitch change.

Agreed, I can scrap away zooming with pitch change when you say the word.

  • Dunno if i have a thinking error but should`t the range for zoom-in/-out be equal ?

Actually, 'zoom out' virtually has no limit, you can make waveform as small as you like, but 'zoom in' is limited. Since we have to use sliders (in future) for zooming, best option is to make limits symmetric. New limits are [-0.8,0.8].

If i use a slider for zoom the result is strange

oops, I missed something while making the new CO, it works better now.

Also the CPU goes crazy on full zoom out and it is noting to see from the waveform as the white beat marker lines fill up the whole space.

Reducing the 'zoom out' limits (this also makes limits symmetric) should help.

mouseScratch2.patch -> patch for mouse scratching without waveform zoom mouseScratch_zoomableWavefrom2.patch -> patch for mouse scratching with waveform zoom.

Info about new CO, "zoom_waveform":

Both patches are against trunk.

Thanks again. shanx

mixxxbot commented 2 years ago

Commented by: esbrandt Date: 2011-05-04T16:24:54Z

23 mouseScratch2.patch

As suggested in comment #⁠6 , try to set a default (@zoom_waveform=0) mouse distance/audio ratio of 250ms. The current 100? ms feel still too sensitive.

Otherwise it is fine.

⁠23 mouseScratch_zoomableWavefrom2.patch

Limiting the range makes sense and is much nicer to the CPU, but the zoom range could be extended (-1.0..1.0). E.g. being able to zoom even further in allows to set (hot)cues to precise positions when preparing tracks.

I think a continuous slider (rather than a stepper/button) for waveform zoom is only really useful if we can map it to a keyboard shortcut. In a live situation this would be the most likely way to zoom in/out. Moving by 1/10th per step should be sufficient . See crossfader_up/_down for example http://mixxx.org/wiki/doku.php/midi_controller_mapping_file_format#ui_midi_controls_and_names

Also now we as can zoom in, the waveform-out-of sync error is obvious (see comment #⁠14)

Good work, fun to test jus

mixxxbot commented 2 years ago

Commented by: rryan Date: 2011-05-27T23:43:19Z

Just applied mouseScratch2.patch -- thanks shanx! I think zooming is interesting but something we should hold off on and consider for 1.11.

mixxxbot commented 2 years ago

Commented by: wrobertson1981 Date: 2011-07-14T12:32:53Z Attachments: [Patch to return old drag to change rate/speed, to the right mouse button](https://bugs.launchpad.net/mixxx/+bug/325377/+attachment/2207576/+files/Patch to return old drag to change rate/speed, to the right mouse button)

Okay, I see this, and I missed the old drag to speedchange .. so I raise you.. a patch! Its on the right mouse button.

mixxxbot commented 2 years ago

Commented by: jonasclemens Date: 2011-12-13T19:53:49Z

I updated the patch by Waylon Robertson to work against today's trunk.

mixxxbot commented 2 years ago

Commented by: rryan Date: 2011-12-13T20:16:14Z

Hi Waylon / Jonas,

I'm sorry but since this bug was marked Fix Committed I forgot all about your patch. I ended up re-enabling the pitch bend behavior from 1.9 on right-click as you requested. It's been fixed in the 1.10 branch for a little while now. Are you guys building from the 1.10 branch?

Thanks for the patch! RJ Ryan

mixxxbot commented 2 years ago

Commented by: jonasclemens Date: 2011-12-14T00:22:48Z Attachments: [pitch-bend on right-click drag on waveform](https://bugs.launchpad.net/mixxx/+bug/325377/+attachment/2632186/+files/pitch-bend on right-click drag on waveform)

i was using the trunk branch and was missing the pitch-bend feature so i thought i'd have to patch it manually. didn't know it was implemented in 1.9 and 1.10 already. thanks for clearing things up!

mixxxbot commented 2 years ago

Issue closed with status Fix Released.