Need help with selecting custom waveform part and create subtitle.

[vivekd95]

Hi everyone,

I am working on a project in which I need to add the functionality of being able to create subtitle for a video manually. What I have is a media, a text area and below it a waveform of the audio of that particular media. So, what I need to do is to be able to click, drag and select an area of waveform and then type the subtitle in the text area.

Below is the video of what I am doing: Click here to see video

I was using wfplayer to show the waveform and my custom components to select and create as well as show the previously present subtitles. Now I want to use peaks.js and hence coming up with a issue of CSS clashing. So, lets say I have 2 components. First (lets say its name is Blocks) to show the subtitles that I have already created. Second (Lets say its name is CreateSub) to show the on the go creation and saving of subtitle over the waveform (which you can see me doing in the video above)).

Issue 1 :

Now what I need is to know that do peaks.js provide me with any way to get pixel of waveform at a particular time on which we have clicked without passing the time. Like when we click on waveform and without moving the seeker. I have checked the peaks parameter passed in Peaks.init function but I am not able to see any way. Also, because my CreateSub component is overlapping the waveform hence whenever I click on the waveform to get the details of that particular clicked moment I get currentTime as 0 because you know seeker have not moved. Hence, I am trying to get pixel of that moment without being dependent on time. Is it possible??

Issue 2 : Second issue that I am facing is how to apply the functionality of moving the subtitle. Here is the video of what is am doing using wfplayer

Now I am doing this in Blocks using mouse move event. I am measuring the difference of initial X value and then new X value. Then, I need to map this distance on waveform. const diffLength = event.pageX - this.leftStart; this.leftDiff = diffLength */+/- ??????????????; How can I calculate the distance( leftDiff ) according to pixels in peaks.js

Thanks for your time and support.

[chrisn]

I'm currently working on a similar feature within Peaks.js. I'm actually starting with making segments draggable (see https://github.com/bbc/peaks.js/issues/443). There's a working prototype in the segment-drag branch`. Each segment could represent a subtitle. After that I may consider implementing creating segments by click+drag.

Issue 1

There's a function view.timeToPixels() that converts a given time in seconds to the number of pixels at the current zoom level. The zoomview.click event contains the time position where the user clicked.

peaks.on('zoomview.click', function(event) {
  const view = peaksInstance.views.getView('zoomview');

  console.log(event.time, view.timeToPixels(event.time));
});

Issue 2

I'm not sure what units you need the distance to be in, but if you have the initial X and current X, you can use view.pixelsToTime() or view.pixelOffsetToTime() to convert pixel values to time in seconds.

Note that all these functions are implementation details and not part of the documented API, which means they may change at any time (although that's fairly unlikely in practice).

[vivekd95]

Thanks for the reply and making things a little easier. But your solution will not work in my case because to create subtitle I am using drag and select action on the waveform via my custom component which basically is overlapping waveform. Hence, I am not able to move seeker.

Also, I would like to give you an advice regarding #443. In this issue, what I can understand is that author only can move the segment. To make a segment what an user should have is start and end time for which I think you should also add the functionality of click and drag of a seeker so that user be able to select segment on their own like I was doing in the first video I attached to the original comment.

[chrisn]

Thanks, yes, #443 is about moving and resizing segments. Issue #190 is about creating segments in the way you describe.

[vivekd95]

Thanks for the help.