search

gtkfi / ArcSDM

Spatial Data Modeler 6 for ArcGIS Pro
32 stars 18 forks source link

Using sk-learn module to replace sdmlr.exe. #132

Closed qiaogh97 closed 4 years ago

qiaogh97 commented 4 years ago

Hi, there! I want to rebuild logisticregression.py code using sklearn module, but I don't know what to do in sdmlr.exe. After reading this code seriously, I get input values and output values. So I want to get some help or explanation about this values in order to rebuild this code successfully.

Input values in case.dat like this:

case.dat

I have to get two output dBASE table, Output_Polynomial_Table and Output_Coefficients_Table. For Output_Polynomial_Table, there are four fields in this table. Here are the codes:

gp.AddField_management(fnNew, 'ID', 'LONG', 6)
gp.AddField_management(fnNew, 'LRPostProb', 'Double', "#", "#", "#", "LR_Posterior_Probability")
gp.AddField_management(fnNew, 'LR_Std_Dev', 'Double', "#", "#", "#", "LR_Standard_Deviation")
gp.AddField_management(fnNew, 'LRTValue', 'Double', "#", "#", "#", "LR_TValue")

For Output_Coefficients_Table, there are also four fields in this table. Here are the codes:

gp.AddField_management(fnNew2, "Theme_ID", 'Long', 6, "#", "#", "Theme_ID")
gp.AddField_management(fnNew2, "Theme", 'text', "#", "#", 256, "Evidential_Theme")
gp.AddField_management(fnNew2, "Coeff", 'double', "#", "#", "#", 'Coefficient')
gp.AddField_management(fnNew2, "LR_Std_Dev", 'double', "#", "#", "#", "LR_Standard_Deviation")

I use numPoints as X , Areas as Y and RasterValue1 as label to execute sklearn.linear_model.LogisticRegression. Also, I use RasterValue2 as label to get other model parameters. For Output_Polynomial_Table, I think ID is easy to get, it is the same as FID in case.dat. But I don't know what LR_Posterior_Probability, LR_Standard_DEviation and LR_TValue are. For Output_Coefficients_Table, I also don't know how to get Coefficient and LR_Standard_Deviation.

Thank you for your help!

qiaogh97 commented 4 years ago

Here are some output information when I try to execute origin logisticregression.py. I print more information in it.

Executing: LogisticRegressionTool C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\reclassified\rc_till_ni;C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\reclassified\rc_till_p d;d C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\rc_till_ni_CD.dbf;C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\rc_till_p_CD.dbf C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\mineral_deposits.lyr -99 1 C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\LR_logpol C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\LR_coeff C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\LR_pprb C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\LR_std C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\LR_conf
Start Time: Mon May 11 09:29:59 2020
Running script LogisticRegressionTool...
check out extension!!!
before check environment
after check environment
get input_rasters
Debug: rc_till_ni
Debug: C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\reclassified\rc_till_ni
C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\reclassified\rc_till_ni
Debug: rc_till_p
Debug: C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\reclassified\rc_till_p
C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\reclassified\rc_till_p
Input rasters: [u'C:\\Users\\BIM\\qiao\\Seafile\\qiao\\Bim\\arcgis\\demodata-master\\demodata-master\\arcsdm\\DemoData_version_files\\reclassified\\rc_till_ni', u'C:\\Users\\BIM\\qiao\\Seafile\\qiao\\Bim\\arcgis\\demodata-master\\demodata-master\\arcsdm\\DemoData_version_files\\reclassified\\rc_till_p']
Evidence_types: [u'd', u'd']
Wts_Tables: [u'C:\\Users\\BIM\\qiao\\Seafile\\qiao\\Bim\\arcgis\\demodata-master\\demodata-master\\arcsdm\\DemoData_version_files\\rc_till_ni_CD.dbf', u'C:\\Users\\BIM\\qiao\\Seafile\\qiao\\Bim\\arcgis\\demodata-master\\demodata-master\\arcsdm\\DemoData_version_files\\rc_till_p_CD.dbf']
C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\rc_till_ni_CD.dbf
C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\rc_till_p_CD.dbf
TrainPts: C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\mineral_deposits.lyr
MissingDataValue: -99
Creating Generalized Class rasters.
C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\reclassified\rc_till_ni
Output_Raster: C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\Scratch\rc_till_ni_G8
Temp_Raster: C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\Scratch\tmp_rst11
C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\reclassified\rc_till_p
Output_Raster: C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\Scratch\rc_till_p_G2
Temp_Raster: C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\Scratch\tmp_rst12
========START========
tmp_UCras8
C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\Scratch\tmp_UCras8
<geoprocessing list object object at 0x28EE6920>
1
DERasterDataset
RasterDataset
tmp_UCras8
1
0
tmp_UCras8
C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\Scratch
========END========
======thmUC_Start=========
Rowid
VALUE
COUNT
RC_TILL_NI_G8
RC_TILL_P_G2
======thmUC_End=========
evflds: ['RC_TILL_NI_G8', 'RC_TILL_P_G2']
lstsVals: [[2, 1, 2, 1], [2, 2, 1, 1]]
lstAreas: [[74380.75, 13624.5, 9881.0, 555.5], [74380.75, 13624.5, 9881.0, 555.5]]
CntsPerRasValu: {1: 218, 2: 10, 3: 10, -9999: 1}
lstPnts: [218, 10, 10, 0]
mcIndeces: [-1, -1]
catMCLists: [[[], []], [-1, -1]]
lstWA: [[1.8559554254165533], [1.8939829899407519]]
ot=[[u'C:\\Users\\BIM\\qiao\\Seafile\\qiao\\Bim\\arcgis\\demodata-master\\demodata-master\\arcsdm\\DemoData_version_files\\reclassified\\rc_till_ni, C:\\Users\\BIM\\qiao\\Seafile\\qiao\\Bim\\arcgis\\demodata-master\\demodata-master\\arcsdm\\DemoData_version_files\\rc_till_ni_CD.dbf'], [u'C:\\Users\\BIM\\qiao\\Seafile\\qiao\\Bim\\arcgis\\demodata-master\\demodata-master\\arcsdm\\DemoData_version_files\\reclassified\\rc_till_p, C:\\Users\\BIM\\qiao\\Seafile\\qiao\\Bim\\arcgis\\demodata-master\\demodata-master\\arcsdm\\DemoData_version_files\\rc_till_p_CD.dbf']]
lstWA: [[1.8559554254165533], [1.8939829899407519]]
Logistic regression output file C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\demodata-master\demodata-master\arcsdm\DemoData_version_files\Scratch\logpol.tba does not exist.
 Error in case.dat or param.dat. 
GP ERRORS:

PYTHON ERRORS:
Traceback Info:
  File "C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\ArcSDM-new\ArcSDM-master\Toolbox\arcsdm\logisticregression.py", line 503, in Execute
    raise Exception

Error Info:
    <type 'exceptions.Exception'>: 

Traceback (most recent call last):
  File "<string>", line 1552, in execute
  File "C:\Users\BIM\qiao\Seafile\qiao\Bim\arcgis\ArcSDM-new\ArcSDM-master\Toolbox\arcsdm\logisticregression.py", line 503, in Execute
    raise Exception
Exception

Error can be ignored since I would like to replace this module.