Find below this e-mail the steps that I now perform to make an update for IBLI based on the eMODIS data. I have just checked with Michele Meroni (also in cc), and he has no problem neither for sharing code. As for myself, we are more scripting to get something working, but the code may look a bit messy sometimes and may need to be changed slightly for each update. Nonetheless, the comments in the file should make the main processes clear.
Attached you find:
Presentation given at ILRI (note that currently IBLI uses for now still the ‘original’ approach) to have some background info
Zip-file with the IDL-programming codes (as referred to below)
Png-file with diagnostics (see explanation below)
Hopefully this is useful to get started. I am sure that there will be other questions after, so feel free to ask.
Best regards,
Anton
PS: possibly the code may work as well using GDL, but I’ve never tried…
§ NOTE: you need to allow about 2-3 days for the latest dekad image to become available. So dekad 1-10 January would become available on 12 or 13 January. I do not have hard data on it, but the updated filtered data of previous dekads comes in about 8 (?) hours after the latest dekad is added. You could monitor this a bit, but it seems that updating on midday of 3, 13, and 23 of each month should be fine.
· Run eMODIS_ORGANIZE.pro and eMODIS_ORGANIZE_INDIVIDUAL_FILES.pro
· Create one large Bil (band interleaved by line)-file: eMODIS_createBIL.pro . This takes: 265 minutes
update number of bands in header! (464)
§ NOTE: this BIL-creation is not a very efficient process now. It should be quite easy to program it differently (in relation to next step) to create per dekad a long-term average and standard deviation.
· Perform z-scoring (consider retaining old diagnostics file): ZNORMBIL_8BIT
Because intercalibration with GIMMS (AVHRR) remains needed, we take the 2001-2011 period as the base period, so:
396 corresponds to 21-31 December: so for z-scoring (avg/stdev) we take complete series up to band 396
Takes about 30 minutes
465 should be updated also for the correct number of bands in the latest stack file (with all updates)
§ NOTE: this file is creating a new diagnostics file each time (which says something about the usability of each pixel). However, in all recent updates we used for the next step always the old diagnostics file to keep results compatible with the formerly delivered data. Perhaps the easiest if I send that file somehow (about 14 MB)? See the attached PNG-file on how this more or less looks
· red=10=variability NDVI 5-95% less than 0.05
· purple=101=5% of more values lower than minimum value (mostly water)
· green=1= at least one value lower than minVal or greater than maxVal was included in the Z computation
· Now aggregate per administrative division the z-scores for each period: AGGREGATE_Z.pro
See above, with the old diagnostics file for direct compatibility: eMODIS_FEWS_Kenya_dia_oldest.img (we only use pixels with diagnostics of 0.
!!! update number of bands here!!! !!! Update also workPath!!!!
17 minutes
· CUMULATE_Z_PER_DIVISION.pro
for all periods (L:21, S:15), and also for partial (S:5 in this case)
§ NOTE: of course partial could differ: is sometimes L, sometimes S…
· run the forecast (Excel calculations) by creating a linear relationship between partial and full SRSD seasons for the complete years (separately for each admin area).
§ NOTE: check with Apurba, but this is probably not needed for the updates.
From Anton:
Find below this e-mail the steps that I now perform to make an update for IBLI based on the eMODIS data. I have just checked with Michele Meroni (also in cc), and he has no problem neither for sharing code. As for myself, we are more scripting to get something working, but the code may look a bit messy sometimes and may need to be changed slightly for each update. Nonetheless, the comments in the file should make the main processes clear.
Attached you find:
Hopefully this is useful to get started. I am sure that there will be other questions after, so feel free to ask.
Best regards,
Anton
PS: possibly the code may work as well using GDL, but I’ve never tried…
IBLI update steps (end-of-season + forecast next)
· Download new files:
§ NOTE: you need to allow about 2-3 days for the latest dekad image to become available. So dekad 1-10 January would become available on 12 or 13 January. I do not have hard data on it, but the updated filtered data of previous dekads comes in about 8 (?) hours after the latest dekad is added. You could monitor this a bit, but it seems that updating on midday of 3, 13, and 23 of each month should be fine.
· Delete older files in individual folder
· Run eMODIS_ORGANIZE.pro and eMODIS_ORGANIZE_INDIVIDUAL_FILES.pro
· Create one large Bil (band interleaved by line)-file: eMODIS_createBIL.pro . This takes: 265 minutes
§ NOTE: this BIL-creation is not a very efficient process now. It should be quite easy to program it differently (in relation to next step) to create per dekad a long-term average and standard deviation.
· Perform z-scoring (consider retaining old diagnostics file): ZNORMBIL_8BIT
§ NOTE: this file is creating a new diagnostics file each time (which says something about the usability of each pixel). However, in all recent updates we used for the next step always the old diagnostics file to keep results compatible with the formerly delivered data. Perhaps the easiest if I send that file somehow (about 14 MB)? See the attached PNG-file on how this more or less looks
· red=10=variability NDVI 5-95% less than 0.05
· purple=101=5% of more values lower than minimum value (mostly water)
· green=1= at least one value lower than minVal or greater than maxVal was included in the Z computation
· Now aggregate per administrative division the z-scores for each period: AGGREGATE_Z.pro
· CUMULATE_Z_PER_DIVISION.pro
§ NOTE: of course partial could differ: is sometimes L, sometimes S…
· run the forecast (Excel calculations) by creating a linear relationship between partial and full SRSD seasons for the complete years (separately for each admin area).
§ NOTE: check with Apurba, but this is probably not needed for the updates.