SWS-Methodology / faoswsProduction

Repository to contain the code for imputing production data in the statistical working system.
0 stars 1 forks source link

Investigate the effect of removing Estimates/Imputation #155

Closed mkao006 closed 8 years ago

mkao006 commented 8 years ago

This is to see whether the function imputeWithEstimateAndWithoutEstimates is necessary

mkao006 commented 8 years ago

After the investigation, we lose about 4280 time series if we remove all estimation/imputation, these time series compose solely of estimated/imputed values from the old system (E, e) or (I, -).

We should remove the function imputeWithEstimateAndWithoutEstimate to avoid confusion, but use processProductionDomain to determine what values should be kept and should be discarded then perform the imputation.

Solutions:

  1. Remove them altogether and accept that we don't have sufficient information to impute them.
  2. Another solution is to extend the production imputation to re-impute these time series.
  3. We accept them only in the case where they are the only values available and remove them when other information are available.
mkao006 commented 8 years ago

Apparently (I, -) is actually calculated value in the old system based on old relationship.

In someway, we can accept this as imputation in the old methodology sense. To avoid disruption, we will not remove (I, -) and since it will not be created in the future, the old assumed relationship will have lesser and lesser effect in the future.

mkao006 commented 8 years ago

So the current decision is to keep (I, -) but remove (E, e) which has minimal impact. Need to discuss with Carola + Team B/C.

If this is agreed, then we can proceed to replace the imputeWithEstimateAndWithoutEstimate with the normal imputeProduction and reduce the large computation burden.

mkao006 commented 8 years ago

function removed and updated package and module in a0721317bbe2f00d6830c9f0016ae30ebd7d9daf