jddingemanse / awtiCodeDev

Development of code for and by Arba Minch University Water Technology Institute
1 stars 2 forks source link

Filling missing data code #29

Open behafta opened 1 year ago

behafta commented 1 year ago

I validated the script file Estimation of Missing data.py. When comparing the code for the Normal Ratio method with my own data, I came upon a problem:

The previous code is:


PA=NX/3*(Data.Mirab.replace(np.nan,0)/NM+Data.Chano.replace(np.nan,0)/NC+Data.Shara.replace(np.nan,0)/NS)

PM=NX/3*(Data.Arbaminch.replace(np.nan,0)/NA+Data.Chano.replace(np.nan,0)/NC+Data.Shara.replace(np.nan,0)/NS)

PC=NX/3*(Data.Mirab.replace(np.nan,0)/NM+Data.Arbaminch.replace(np.nan,0)/NA+Data.Shara.replace(np.nan,0)/NS)

PS=NX/3*(Data.Mirab.replace(np.nan,0)/NM+Data.Chano.replace(np.nan,0)/NC+Data.Arbaminch.replace(np.nan,0)/NA)

However, instead of NX, there should be the calculated Normal Ratio for each individual station. With the following code it works:

PA=NA/3*(Data.Mirab.replace(np.nan,0)/NM+Data.Chano.replace(np.nan,0)/NC+Data.Shara.replace(np.nan,0)/NS)

PM=NM/3*(Data.Arbaminch.replace(np.nan,0)/NA+Data.Chano.replace(np.nan,0)/NC+Data.Shara.replace(np.nan,0)/NS)

PC=NC/3*(Data.Mirab.replace(np.nan,0)/NM+Data.Arbaminch.replace(np.nan,0)/NA+Data.Shara.replace(np.nan,0)/NS)

PS=NS/3*(Data.Mirab.replace(np.nan,0)/NM+Data.Chano.replace(np.nan,0)/NC+Data.Arbaminch.replace(np.nan,0)/NA)

the Inverse Distance method is works, but I changed the long formula to a short formula. like this:-

def fillingIDW(inputCols,distances):
    distances = np.array(distances)
    distSq = (distances**2).reshape((1,len(distances)))
    filled = (inputCols/distSq).sum(axis=1)/np.sum(1/distSq)
    return filled

#filling  missing data for Arbaminch station by inverse distance method 
ArbaminchInput = Data .iloc[:,[2,3,4]]
ArbaminchDists = [10,12,14]
ArbaminchFillData = fillingIDW(ArbaminchInput,ArbaminchDists)
Arbaminchfilled2 = Data.Arbaminch.fillna(fillingIDW(ArbaminchInput,ArbaminchDists))
#filling  missing data for Mirab station by inverse distance method 
MirabInput = Data.iloc[:,[1,3,4]]
MirabDists = [10,18,16]
MirabFillData = fillingIDW(MirabInput,MirabDists)
Mirabfilled2 = Data.Mirab.fillna(fillingIDW(MirabInput,MirabDists))
#filling  missing data for Chano station by inverse distance method 
ChanoInput = Data.iloc[:,[1,2,4]]
ChanoDists = [12,18,20]
ChanoFillData = fillingIDW(ChanoInput,ChanoDists)
Chanofilled2 = Data.Chano.fillna(fillingIDW(ChanoInput,ChanoDists))
#filling  missing data for Shara station by inverse distance method 
 SharaInput = Data.iloc[:,[1,2,3]]
 SharaDists = [14,16,20]
 SharaFillData = fillingIDW( SharaInput,SharaDists)
 Sharafilled2 = Data.Shara.fillna(fillingIDW( SharaInput, SharaDists))
Final=Data.fillna(value = {'Arbaminch':Arbaminchfilled2,'Mirab':Mirabfilled2,'Chano':Chanofilled2,'Shara':  Sharafilled2})
#Finally save like this
[ፓይተን.docx](https://github.com/jddingemanse/awtiCodeDev/files/11208336/default.docx)

Final.to_csv('output/FinalfillIDW.csv')
Final.to_excel('output/FinalfillIDW1.xlsx')

Inverse distance method.txt

jddingemanse commented 1 year ago

These seem good suggestions for changing. Please change the code in the file Estimation of Missing data.py, and upload the new file / save the changes.