search

JuliaStats / StatsBase.jl

Basic statistics for Julia
Other
587 stars 192 forks source link

Basic computation routines #67

Closed lindahua closed 10 years ago

lindahua commented 10 years ago

In the process of removing NumericExtensions from packages, I found that I keep rewriting some simple routines everywhere (i.e. many stats-related packages).

I feel that some of the commonly used routines should be placed here.

I am considering to write a small set of functions to support basic statistical computing, so that people won't have to write them again and again.

Here is a tentative list of functions to be added:

This wouldn't make the package much larger, but it will provide a lot of convenience to statistical computing.

If there's no objection, I will take the lead to implement these. Opinions?

johnmyleswhite commented 10 years ago

I agree that we could use all of those.

simonster commented 10 years ago

inplace arithmetics (add!, subtract!, multiply!, etc) add/subtract a vector (or its scaled version) to each column/row inlace

I'm not sure we need new functions for these; broadcast! works well enough for me.

inplace of several widely used math functions (abs!, abs2!, sqrt!, exp!, and log!)

It is possible to use broadcast! for these as well, but it's awkward. It may make sense to have separate functions. (It's also possible to use map!, but there is a substantial performance cost that broadcast! avoids by effectively specializing on the function argument.)

weighted sum

I will add this to #66 as you suggested.

sum/mean of abs/square (both weighted & non-weighted)

I can handle this, too. Unweighted sum of squares is just Base.sumabs2 (maybe this should be exported from Base?), and I've implemented a variant of Base.sumabs2 that accepts weights for #53, although the PR is not quite ready. abs is similar enough that I can reuse the implementation.

maximum absolute value

Sounds good.

nalimilan commented 10 years ago

I agree these are useful, but shouldn't the first 3 go to Base instead? (I really wish there were an inplace operator so that these functions would not be needed at all, but waiting for it...)

lindahua commented 10 years ago

Using broadcast! can do inplace arithmetics, but it is cumbersome and not pretty.

simonbyrne commented 10 years ago

Certainly the last 3 would make sense here.

The first 3 aren't really stats related, but until we have proper inplace support in Base, I guess it would make sense to keep them here. Perhaps they shouldn't be exported though?

papamarkou commented 10 years ago

@lindahua, I wanted to ask if there is a roadmap to remove NumericExtensions completely. I use your package in various places and wanted to be up-to-date with the ongoing developments so as to modify my coding accordingly - what is the plan at the moment?

lindahua commented 10 years ago

@scidom NumericExtensions is still being maintained, and bugs will be fixed as needed. You don't have to change those packages at the moment.

In long term, we are trying to look for a better way to provide basic computational support.

lindahua commented 10 years ago

Close this as many of these functions have already been implemented in Base.