Log plots support - Githubissues

florian6973 commented 1 year ago

Hi!

I am trying to use plotly-resampler for log plots, but since it is a linear downsampling, it does not work (bad resolution on the left on the graph). Would you know a workaround by any chance, or if it is a planned feature?

Thanks!

jonasvdd commented 1 year ago

@florian6973, If you would use the plotly-resampler log support branch see #207, would the code snippets added in the comments resolve your issue?

Main things I did:

the pull-request fixes the zoom-resample functionality for log-axis plots
I altered an aggregation algorithm to use log-based equidistant bins to select data points.

Kind regards, Jonas.

florian6973 commented 1 year ago

Hi Jonas! Wow thanks for working on this feature so quickly! I am sorry, I should have been more precise in my first message, but I was mainly interested in plotting the both axes together (x and y) with a logarithmic scale (for a log-log Fourier Transform plot), and it seems the code snippets work for x axis only (I think it can be really useful too in any case ;)) Just out of curiosity, what does the 'NoGapHandler' class do please? Thanks again :)

jonasvdd commented 1 year ago

Hi @florian6973, no problem at all! You had an interesting use-case, and this functionality was not yet supported by plotly-resampler, so was certainly worthwhile to investigate!

Just out of curiosity, what does the 'NoGapHandler' class do please?

If you read this part of the documentation, you will see that the NoGapHandler does not introduce gaps (i.e. unconnected lines) in the plot. Side questions:

How are gaps determined: Gaps are determined through assessing the differences between consecutive x-values of the aggregated series. If this difference is above a certain threshold, which is how the MedDiffGapHandler works, a gap will be inserted.
Why is this important for log-x aggregators: Log-x aggregators such as LogLTTB will perform binning to ensure equidistant log-x bins. This will create large irregularities in the non-log x-space; making the default x-space based gap handlers introduce gaps where there shouldn't be any. Setting the gap handler to NoGapHandler is an ad-hoc solution to alleviate this unwanted behavior.

I can see that the second demo visualizes the data via a log x and log-y? So If you would use the NoGapHandler (and maybe increase the number of output samples (i.e. n_shown_samples), the visualization should be fine?

fyi: Could you send me some example data from your log-log FT plot? I think that would make a good use-case to add in the examples.

Kind regards, Jonas

florian6973 commented 1 year ago

Thank you very much for your explanations, and sorry for this late reply! Oh I see, I was not aware there was such a feature in plotly-resampler about gaps... This is quite nice indeed! It also means that later a LogGapHandler could be implemented?

Yes exactly, regarding the log-log plots, I just used your code and I added the line fr.update_yaxes(type='log'), so there already are the LogLLTB and the NoGapHandler as arguments but there are still some gaps... I increased the max_n_samples to 10000 but it does not remove them unfortunately, as you can see: 😢 newplot (14) I am not sure to know what I am doing wrong.

Sure, the idea is to plot the FFT of a single-cell recording, and to be able to see at the same time the LFP frequencies (a few Hz) and the frequencies related to spikes (10^2 to 10^3 Hz). Please find attached a sample, even if I am not sure this is the best one to see it well. This is 1-sec recording of one channel,. dat_s.txt

If you need any additional information, please let me know,

Best regards, florian6973

jonasvdd commented 1 year ago

Mhmm, I am not following you 100%; I just loaded the dat_s.txt but this does not give me the plot which you show above (even when I apply np.fft.fft, or I am doing something wrong with the data, which is very plausible 🙃 ).

Can you give me some example data and code where your log gap issue occurs, even when the nogaphandler is set?

florian6973 commented 1 year ago

Yes you're right, sorry I mixed up the two examples. I get the graph below when I plot the FT of the signal from dat_s.txt (NoGapHandler is working well in this case): figure: code:

import numpy as np
from plotly_resampler.aggregation import NoGapHandler
from plotly_resampler import FigureResampler
from plotly_resampler.aggregation.aggregation_interface import DataPointSelector
from typing import Union
import plotly.graph_objects as go

[... LogLLTB class ...]

file = "dat_s.txt"
dat_s = np.loadtxt(file)

fft_coefs = np.abs(np.fft.rfft(dat_s))
fft_freq = np.fft.rfftfreq(len(dat_s), d=1 / 30000)

fr = FigureResampler(False)
fr.add_trace(
    go.Scattergl(mode="lines+markers"),
    hf_x=fft_freq,
    hf_y=fft_coefs,
    downsampler=LogLTTB(),
    gap_handler=NoGapHandler(),
    max_n_samples=10000,
)
fr.update_xaxes(type="log")
fr.update_yaxes(type="log")
fr.update_layout(template="plotly_white", title="log axis demo")
fr.show_dash(mode="inline")

However I still have an issue when I interactively zoom in on the figure: the curve disappear, as you can see: newplot (16) Besides, I agree it is not the best looking plot, but most of data I use are confidential...

Concerning the issue about NoGapHandler, it is with this example:

[... imports and LogLTTB class ...]

n = 100_000
y = np.sin(np.arange(n) / 2_000) + np.random.randn(n) / 10

fr = FigureResampler(False)
fr.add_trace(
    go.Scattergl(mode="lines+markers", marker_color=np.abs(y) / np.max(np.abs(y))),
    hf_x=np.arange(n),
    hf_y=y,
    downsampler=LogLTTB(),
    gap_handler=NoGapHandler(),
    max_n_samples=10000,
)
fr.update_xaxes(type="log")
fr.update_yaxes(type="log")
fr.update_layout(template="plotly_white", title="log axis demo")

fr.show_dash(mode='inline')

This gives the figure of my previous message, and I am surprised there are gaps between points, even though it is not supposed to be so.

Thanks :)

jonasvdd commented 1 year ago

Hi @florian6973,

Thank you for sharing the clarifcation of the examples and sharing your code, very useful.

Regarding your zoom-resample issue: I tried your example and the zoom-resample works just fine (see GIF below). I believe this is caused by the fact that you do not use the plotly-resampler log plot code-base (#207), which resolves this issue.

log_example

Regarding the gaps: You tried taking the log of negative value, which yields -inf!

[... imports and LogLTTB class ...]

n = 100_000
y = np.sin(np.arange(n) / 2_000) + np.random.randn(n) / 10

fr = FigureResampler()
fr.add_trace(
    go.Scattergl(mode="lines+markers", marker_color=np.abs(y) / np.max(np.abs(y))),
    hf_x=np.arange(n),
    # NOTE: this y can be negative as it is a noisy sine wave
    # hf_y=y,
    hf_y=np.abs(y),
    downsampler=LogLTTB(),
    gap_handler=NoGapHandler(),
    max_n_samples=10000,
)
fr.update_xaxes(type="log")
fr.update_yaxes(type="log")
fr.update_layout(template="plotly_white", title="log axis demo")

fr.show_dash(mode='inline')

If you use np.abs(y), as in the above snippet; the result will look like this!

jonasvdd commented 1 year ago

I think this issue can be closed if (i) the log plot PR gets merged and (ii) the above answer resolved your question!

florian6973 commented 1 year ago

Thank you very much Jonas for all your explanations and help! Indeed I cloned the repo, switched to the log_support branch and tested, it works perfectly then! So I will wait for the merging of the PR and then its release to use this feature ;) Oh my bad, to be honest I did not look at the values and just applied a log axis 😅 By the way I have always thought I would get at least a numpy warning if I take the log of a negative number... Thanks again!

jonasvdd commented 1 year ago

The PR is now merged on the main branch, After I fixed another issue, I will create a new release candidate!

jvdd commented 1 year ago

@florian6973 we just created plotly-resampler==0.9.0rc3

predict-idlab / plotly-resampler

Log plots support #206