26medias / timeseries-analysis

NPM Package - Timeseries analysis, noise removal, stats, ...
237 stars 48 forks source link

Feature request: MA for time period even if data is missing #11

Open pedrosimao opened 5 years ago

pedrosimao commented 5 years ago

First of all, thanks a lot for this nice share. I see you have put some good thought on the forecast functions and so on. You library can be handy but there is one important feature missing. Let's say I have incomplete data, for example, I have one value every one minute, but for 10 minutes I have missing data. If I use the MA function, for a period of 60, the output will be wrong, because the algorithm considers that the data is complete.

Do you know a little trick to detect missing data? We would need to consider the dates in that case and not only the values...

pedrosimao commented 5 years ago

The idea would be to make the buffer slice according to a period in time (1 hour, 30m or 1 day), instead of a predefined period number. Then average the buffer length instead of the buffer period. I am just not sure how to make this slice.

pedrosimao commented 5 years ago

Ok. I think I did something acceptable. I just needed to use an external library (moment.js)

// Mean by timeframe (timeframe: "hour", "day", "isoWeek", "quarter", "month")
timeseries.prototype.meanBy = function(timeframe) {
    const momentData = _.map(this.data, item => {
        return {
            date: moment(item[0])
                .startOf(timeframe)
                .format(),
            values: parseInt(item[1])
        }
    });
    const groupedData = _.groupBy(momentData, "date")
    return _.mapValues(groupedData, item => {
        return _.meanBy(item, 'values')
    })
};
26medias commented 5 years ago

Hi,

I'm not updating this package anymore, but feel free to submit a pull request if you build it.

I have been working on a visual replacement for this package, a way to process, transform & analyze data series in a visual way with an export of the JS algorithm generated on the other end so you can use it anywhere you need it (or trigger alerts, send to ML, export to other site, ...)

I posted a preview a couple days ago, I'm opening the first private beta in about a month. https://www.facebook.com/640905346018076/videos/233273264019370/

Developers can build/fork their own logic blocks, datasources, visualization, all in Javascript, right in the browser. I got tired of Python everywhere and the iPython/Jupyter standard.