Step 1 can fall over when discarding data.

[GoogleCodeExporter]

Part of step 1 discards data under the control of configuration files in 
config/* .  If a particular datum is marked as to-be-discarded then it will 
fail if the input data has a record for that station but the record is short, 
and does not extend up to the time of the to-be-discarded datum.

This can happen if you are playing around with the input files, but in 
particular, it can happen if you're using the v2.longrural data which is not a 
complete dataset.

$ nice python tool/run.py -p data_sources='ghcn' -s 0,1,3,5... creating 
directory result
====> STEPS 0, 1, 3, 5  ====
No ocean data; using land data only
Load GHCN records
Traceback (most recent call last):
  File "tool/run.py", line 280, in <module>
    sys.exit(main())
  File "tool/run.py", line 261, in main
    data = step_fn[step](data)
  File "tool/run.py", line 104, in run_step5
    result = step5.step5(data)
  File "/Users/drj/ccc-gistemp/code/step5.py", line 443, in step5
    return annzon(zoned_averages)
  File "/Users/drj/ccc-gistemp/code/step5.py", line 320, in annzon
    (tdata, twt) = zoned_averages.next()
  File "/Users/drj/ccc-gistemp/code/step5.py", line 183, in zonav
    boxed_data.next())
  File "/Users/drj/ccc-gistemp/tool/gio.py", line 1626, in step5_bx_output
    for record in data:
  File "/Users/drj/ccc-gistemp/code/step5.py", line 69, in SBBXtoBX
    landweight,landsub,oceansub = zip(*itertools.islice(data, nsubbox))
  File "/Users/drj/ccc-gistemp/tool/gio.py", line 1646, in step5_mask_output
    for datum in data:
  File "/Users/drj/ccc-gistemp/code/step5.py", line 415, in ensure_weight
    for _,land,ocean in data:
  File "/Users/drj/ccc-gistemp/tool/gio.py", line 1555, in step5_input
    for land, ocean in data:
  File "/Users/drj/ccc-gistemp/tool/gio.py", line 1589, in ensure_landocean
    for series in data:
  File "/Users/drj/ccc-gistemp/tool/gio.py", line 1604, in add_blank
    for this_box in data:
  File "/Users/drj/ccc-gistemp/tool/gio.py", line 1405, in step3_output
    for thing in data:
  File "/Users/drj/ccc-gistemp/code/step3.py", line 296, in step3
    for box in box_source:
  File "/Users/drj/ccc-gistemp/code/step3.py", line 162, in iter_subbox_grid
    station_records = list(station_records)
  File "/Users/drj/ccc-gistemp/tool/gio.py", line 1374, in output
    for thing in data:
  File "/Users/drj/ccc-gistemp/code/step1.py", line 503, in step1
    for record in alter_discont(without_strange):
  File "/Users/drj/ccc-gistemp/code/step1.py", line 338, in alter_discont
    for record in data:
  File "/Users/drj/ccc-gistemp/code/step1.py", line 325, in drop_strange
    series[(year-begin)*12 + x-1] = MISSING
IndexError: list assignment index out of range

Original issue reported on code.google.com by d...@ravenbrook.com on 17 Dec 2010 at 11:20

[GoogleCodeExporter]

This issue was closed by revision r647.

Original comment by d...@ravenbrook.com on 17 Dec 2010 at 11:41

• Changed state: Fixed

[GoogleCodeExporter]

Original comment by d...@ravenbrook.com on 17 Dec 2010 at 11:41

• Changed title: Step 1 can fall over when discarding data.