search

matteoredaelli / strategico

Automatically exported from code.google.com/p/strategico
GNU General Public License v3.0
1 stars 1 forks source link

Unit tests warnings : attempting model selection on an essentially perfect fit is nonsense... #27

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
A Unit test fails: file test_050_ltp.R

  e3 <- EvalTSString(project.name,
                     ts.string="33,244,225,201,101,103,51,40,24,44,73,120,106",
                     ## ts.periods.string="2001-1,2001-2,2002-1,2002-2,2002-3,2003-1,2003-3,2004-1,2004-2,2005-1,2005-2,2005-3,2006-1,2006-3,2007-1,2007-2,2007-3,2008-1",
                     period.start="2010-4", period.freq=12, project.config=project.config, db.channel=db.channel)
  checkEquals(
              c(5, 6, 7, 5, 6, 7, 5, 6),  # RANDOM... IT FAILS.....
              ##c(317, 298, 274, 174, 176, 124, 113, 97),
              as.vector(e3[1,])
              )

The same test can be run with

./strategico.R -n sample --cmd eval_ts 
--ts.string="33,244,225,201,101,103,51,40,24,44,73,120,106"  --ts.start=2010-4 
--ts.freq=12 -i 4343

This is forecast 2.11
INFO [2011-07-07 15:39:37] ++++++++++++++++++++++++Item.EvalData 
++++++++++++++++++++++++
INFO [2011-07-07 15:39:37] Project=sample Loading item ID=4343 KEYS= V1=NA
INFO [2011-07-07 15:39:37] TS length= 13
DEBUG [2011-07-07 15:39:37] 33
 DEBUG [2011-07-07 15:39:37] 244
 DEBUG [2011-07-07 15:39:37] 225
 DEBUG [2011-07-07 15:39:37] 201
 DEBUG [2011-07-07 15:39:37] 101
 DEBUG [2011-07-07 15:39:37] 103
 DEBUG [2011-07-07 15:39:37] 51
 DEBUG [2011-07-07 15:39:37] 40
 DEBUG [2011-07-07 15:39:37] 24
 DEBUG [2011-07-07 15:39:37] 44
 DEBUG [2011-07-07 15:39:37] 73
 DEBUG [2011-07-07 15:39:37] 120
 DEBUG [2011-07-07 15:39:37] 106
INFO [2011-07-07 15:39:37] period.start=2010-04 period.freq=12 
period.end=2011-04
DEBUG [2011-07-07 15:39:37] Param=  
n.ahead=8;range=c(-Inf,Inf);NA2value=0;n.min=10;try.models=c('mean','trend','lm'
,'es','arima','naive');naive.values='last';logtransform=FALSE;stepwise=TRUE;form
ula.right.lm='S*trend+S*trend2';rule='BestAIC';rule.noMaxOver=2;negTo0=FALSE;toI
nteger=TRUE
DEBUG [2011-07-07 15:39:37] Normalized data:
DEBUG [2011-07-07 15:39:37] list(V = c(0, 0, 0, 33, 244, 225, 201, 101, 103, 
51, 40, 24, 44, 73, 120, 106))
 DEBUG [2011-07-07 15:39:37] c(2010, 4)
WARN [2011-07-07 15:39:40] Saving Model to file 
/var/www/strategico/projects/sample/8/4343/V1/model.RData
DEBUG [2011-07-07 15:39:40] Importing data to table sample_data_norm_V1
DEBUG [2011-07-07 15:39:40] Running SQL: delete from sample_data_norm_V1 where 
item_id = 4343
DEBUG [2011-07-07 15:39:40] Saving data to table sample_data_norm_V1
DEBUG [2011-07-07 15:39:40] Importing data to table sample_summary_V1
DEBUG [2011-07-07 15:39:40] Running SQL: delete from sample_summary_V1 where id 
= 4343
DEBUG [2011-07-07 15:39:40] Saving data to table sample_summary_V1
DEBUG [2011-07-07 15:39:40] Importing data to table sample_summary_models_V1
DEBUG [2011-07-07 15:39:40] Running SQL: delete from sample_summary_models_V1 
where item_id = 4343
DEBUG [2011-07-07 15:39:40] Saving data to table sample_summary_models_V1
WARN [2011-07-07 15:39:40] Best Model is  Arima
DEBUG [2011-07-07 15:39:40] Importing data to table sample_results_V1
DEBUG [2011-07-07 15:39:40] Running SQL: delete from sample_results_V1 where 
item_id = 4343
DEBUG [2011-07-07 15:39:40] Saving data to table sample_results_V1
DEBUG [2011-07-07 15:39:43] RESULTS:
DEBUG [2011-07-07 15:39:43] 2011-05
 DEBUG [2011-07-07 15:39:43] 2011-06
 DEBUG [2011-07-07 15:39:43] 2011-07
 DEBUG [2011-07-07 15:39:43] 2011-08
 DEBUG [2011-07-07 15:39:43] 2011-09
 DEBUG [2011-07-07 15:39:43] 2011-10
 DEBUG [2011-07-07 15:39:43] 2011-11
 DEBUG [2011-07-07 15:39:43] 2011-12
DEBUG [2011-07-07 15:39:43] c(317, 298, 274, 174, 176, 124, 113, 97)
Warning messages:
1: attempting model selection on an essentially perfect fit is nonsense
2: In predict.lm(modlm, newdata = new.data, interval = "prediction",  :
  prediction from a rank-deficient fit may be misleading
3: In qt(p, df, lower.tail, log.p) : NaNs produced
4: closing unused RODBC handle 1

Original issue reported on code.google.com by matteo.redaelli on 7 Jul 2011 at 2:34

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago

Original comment by matteo.redaelli on 8 Jul 2011 at 7:30