Implementing a basic knn model with weighting functions

[krisy]

Implement a basic knn model with numpy:

• iterate over the test data, for all row 'te':
• iterate through the training data, for all row 'tr':
• for all columns 'co' in 'te':
• call function {columname}Distance(tr[co], te[co])
• sum the above
• create list of sums, tr ids: 'li' ordered by sum
• use function thresholdDistance(li,te,k,t) to compute final prediction, where t is threshold (for paramscan?)

[krisy]

See: http://glowingpython.blogspot.hu/2012/04/k-nearest-neighbor-search.html for an implementation of knn in numpy. For {columnname}Distance:

• check, if there are managers, who would grant anything (use in managerDistance)
• for roleDistance should check if they are in the same ROLE_FAMILY (or ROLE_UP?)
• do same ROLE_IDs on the same RESOURCE_ID imply the same action (or more correctly, how likely is this)?

[krisy]

Also see http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.KDTree.html#scipy.spatial.KDTree for KDTree implementation in numpy

[krisy]

In the first 1000 rows, if the resource_id and role_id where equal for two rows, in only about 30 events were the outcomes (action) different. Need to check this, and that how many pairs were in total - but should be a nice start for knn (if there are a lot of similar elements in the test)