Lancern / asm2vec

An unofficial implementation of asm2vec as a standalone python package
160 stars 38 forks source link

Sigmoid Function is Incorrect #5

Open oalieno opened 3 years ago

oalieno commented 3 years ago

https://github.com/Lancern/asm2vec/blob/6e975f0b9cf573358d2f73c308aa043e26fb9a10/asm2vec/internal/training.py#L187-L188

should be

def _sigmoid(x: float) -> float:
    return 1 / (1 + np.exp(-x))
oalieno commented 3 years ago

After my calculation, if we use 1 / (1 + np.exp(x)) as sigmoid function, then the derivative in the code is correct.

https://github.com/Lancern/asm2vec/blob/6e975f0b9cf573358d2f73c308aa043e26fb9a10/asm2vec/internal/training.py#L234

But if we want to use the regular sigmoid function 1 / (1 + np.exp(-x)). Then we need to change the derivative to the following, which only differ by a minus sign comparing with original derivative.

 g = (_dot_sigmoid(delta, tk.v_pred) - _identity(tk is sp_tk)) * context.alpha() 

Both are correct and produce the same result. But I think using the regular sigmoid function is less confusing.