search

natekupp / ffx

Fast Function Extraction
http://trent.st/ffx
Other
80 stars 97 forks source link

Index out of bounds error #15

Closed doccosmos closed 10 years ago

doccosmos commented 10 years ago

I'm trying FFX with gridded data (not certain if that is an issue) with a set of approx. 3000 data points consisting of 4 variables. The traceback is:

Traceback (most recent call last): File "", line 1, in File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/ffx-1.3.4-py2.7.egg/ffx/api.py", line 4, in run return core.MultiFFXModelFactory().build(train_X, train_y, test_X, test_y, varnames, verbose) File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/ffx-1.3.4-py2.7.egg/ffx/core.py", line 443, in build next_models = FFXModelFactory().build(train_X, train_y, ss, varnames, verbose) File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/ffx-1.3.4-py2.7.egg/ffx/core.py", line 653, in build ss, varnames, bases, X, y, ss.final_max_num_bases, ss.final_target_train_nmse, verbose) File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/ffx-1.3.4-py2.7.egg/ffx/core.py", line 683, in _basesToModels max_num_bases, target_train_nmse, verbose) File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/ffx-1.3.4-py2.7.egg/ffx/core.py", line 700, in _pathwiseLearn (X_unbiased, y_unbiased, X_avgs, X_stds, y_avg, y_std) = self._unbiasedXy(X_orig_regress, y_orig) File "/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/ffx-1.3.4-py2.7.egg/ffx/core.py", line 810, in _unbiasedXy X_unbiased[i] = (Xin[i] - X_avgs[i]) IndexError: index out of bounds

jmmcd commented 10 years ago

"Gridded data" is a bit vague. Can you give some code that reproduces this? My guess is you have one of the input arrays the wrong shape, eg need to be transposed. See one of the example calls eg in the tests directory.

doccosmos commented 10 years ago

Here is some sample data that illustrates the problem:

-29.5 8.25 2.25 3.25 -1.4630270313 -29.5 8.25 2.25 7.25 -1.71697949168 -29.5 8.25 2.25 4.25 -1.69787291674 -29.5 8.25 2.25 5.25 -1.53302128151 -29.5 8.25 2.25 6.25 -1.26625153738 -29.5 8.25 3.75 3.75 4.2483692247 -29.5 8.25 3.75 4.75 4.76095406085 -29.5 8.25 3.75 5.75 5.08402880341 -29.5 8.25 3.75 6.75 5.79300879904 -29.5 8.75 1.75 1.25 -1.47987379559 -29.5 8.75 1.75 2.25 -1.71747796295 -29.5 8.75 1.75 3.25 -3.00425917554 -29.5 8.75 1.75 4.25 -2.77916000807 -29.5 8.75 1.75 5.25 -1.95375185498 -29.5 8.75 1.75 6.25 -0.63724592177 -29.5 8.75 1.75 7.25 2.08913760261 -29.5 8.75 2.75 1.25 -0.155497274147 -29.5 8.75 2.75 1.75 -0.218847240101 -29.5 8.75 2.75 2.25 -0.371720185231 -29.5 8.75 2.75 2.75 -1.35347045491 -29.5 8.75 2.75 3.25 -0.478830838358 -29.5 8.75 2.75 3.75 -1.47377076418 -29.5 8.75 2.75 4.25 0.298827373567 -29.5 8.75 2.75 4.75 -0.881919827508 -29.5 8.75 2.75 5.25 0.42159213941 -29.5 8.75 2.75 5.75 -0.246759807172 -29.5 8.75 2.75 6.25 1.48925577785 -29.5 8.75 2.75 6.75 1.31331287969 -29.5 8.75 2.75 7.25 1.6723001813 -29.5 8.75 3.25 0.75 -0.41862274541 -29.5 8.75 3.25 1.25 -0.684206334538 -29.5 8.75 3.25 1.75 -0.297069354246 -29.5 8.75 3.25 2.25 -0.380677947922 -29.5 8.75 3.25 2.75 0.853572303649 -29.5 8.75 3.25 3.25 -0.450737452874 -29.5 8.75 3.25 3.75 0.636269718857 -29.5 8.75 3.25 4.25 -0.589512684182 -29.5 8.75 3.25 4.75 1.29536062911 -29.5 8.75 3.25 5.25 -0.34709497969 -29.5 8.75 3.25 5.75 1.90536019765 -29.5 8.75 3.25 6.25 -0.944309148957 -29.5 8.75 3.25 6.75 3.56470500459 -29.5 8.75 3.25 7.25 0.057826211354 -29.5 8.75 3.75 1.75 1.06321657158 -29.5 8.75 3.75 2.25 -0.708762194742 -29.5 8.75 3.75 2.75 1.26826205141 -29.5 8.75 3.75 3.25 -0.0855678428659 -29.5 8.75 3.75 3.75 1.19536378126 -29.5 8.75 3.75 4.25 -0.0148874229793 -29.5 8.75 3.75 4.75 1.41682690051 -29.5 8.75 3.75 5.25 0.168650814895 -29.5 8.75 3.75 5.75 1.9244483625 -29.5 8.75 3.75 6.25 0.475264337511 -29.5 8.75 3.75 6.75 1.71457787299 -29.5 8.75 3.75 7.25 1.29416603539 -29.5 9.25 1.75 1.25 -2.02076789366 -29.5 9.25 1.75 1.75 -1.32620237076 -29.5 9.25 1.75 2.25 -1.30243774157 -29.5 9.25 1.75 2.75 -1.61338068424 -29.5 9.25 1.75 3.25 -1.41760489972 -29.5 9.25 1.75 3.75 -1.02918821443 -29.5 9.25 1.75 4.25 -1.60473719182 -29.5 9.25 1.75 4.75 -0.411650655813 -29.5 9.25 1.75 5.25 -0.90861248941 -29.5 9.25 1.75 5.75 0.000241932450377 -29.5 9.25 1.75 6.25 -0.0632908250508 -29.5 9.25 1.75 6.75 1.54971423354 -29.5 9.25 1.75 7.25 0.75952105709 -29.5 9.25 1.75 7.75 4.94875722855 -29.5 9.25 1.75 8.25 2.94327967867 -29.5 9.25 2.25 1.25 -1.61809940238 -29.5 9.25 2.25 1.75 -0.416867174735 -29.5 9.25 2.25 2.25 -1.3192740169 -29.5 9.25 2.25 2.75 -0.583581369466 -29.5 9.25 2.25 3.25 -1.13176978088 -29.5 9.25 2.25 3.75 -0.647840985445 -29.5 9.25 2.25 4.25 -0.428840781012 -29.5 9.25 2.25 4.75 -0.213591980912 -29.5 9.25 2.25 5.25 -0.0821204834809 -29.5 9.25 2.25 5.75 0.0426479614008 -29.5 9.25 2.25 6.25 1.33934768042 -29.5 9.25 2.25 6.75 1.28195123851 -29.5 9.25 2.25 7.25 3.43743763762 -29.5 9.25 2.25 7.75 3.49643823748 -29.5 9.25 2.75 0.75 -0.889823651214 -29.5 9.25 2.75 1.25 -0.674384953575 -29.5 9.25 2.75 1.75 -0.698879111213 -29.5 9.25 2.75 2.25 -0.496085054416 -29.5 9.25 2.75 2.75 -0.628892434135 -29.5 9.25 2.75 3.25 -0.295017379337 -29.5 9.25 2.75 3.75 -0.755296556732 -29.5 9.25 2.75 4.25 -0.24801729937 -29.5 9.25 2.75 4.75 -0.498771868989 -29.5 9.25 2.75 5.25 0.118420791373 -29.5 9.25 2.75 5.75 0.918845998494 -29.5 9.25 2.75 6.25 0.178274200918 -29.5 9.25 2.75 6.75 2.49961285924 -29.5 9.25 2.75 7.25 2.81562908779 -29.5 9.25 2.75 7.75 5.19761924026 -29.5 9.25 3.25 0.75 -0.335358124468

jmmcd commented 10 years ago

No, you need to show us the code you're using to read it in, and what you're passing to FFX.

doccosmos commented 10 years ago

data = np.genfromtxt(file) nvars = data.shape[1] - 1 X = data.T[:nvars] y = data.T[nvars] X = X.T X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size = 0.2) vars = ["x%s" % i for i in range(nvars)] models = ffx.run(X_train, y_train, X_test, y_test, vars)

jmmcd commented 10 years ago

Works for me.

doccosmos commented 10 years ago

Hi,

I'm sorry but it does not for me. This is a reproducible error on my system.

Cheers,

Matthew

On Mar 20, 2014, at 1:53 PM, James McDermott wrote:

Works for me.

— Reply to this email directly or view it on GitHub.

jmmcd commented 10 years ago

This is why "post a complete self-contained minimal example" is good advice when bug-tracking. Can you post a program which produces the error without reading any external file? Put the data in the program, as in tests/x_square_test.py.

Make sure to include all your imports etc, so that we are definitely running the same thing.

And report your versions of everything relevant -- FFX, Numpy, Sklearn, Python.

And report the behaviour in full. When does the error happen -- immediately, or after some iterations?

http://www.chiark.greenend.org.uk/~sgtatham/bugs.html