NervanaSystems / neon

Intel® Nervana™ reference deep learning framework committed to best performance on all hardware
http://neon.nervanasys.com/docs/latest
Apache License 2.0
3.87k stars 811 forks source link

Sigmoid function in CPU backend is numerically unstable #425

Open miketout opened 6 years ago

miketout commented 6 years ago

To prevent crashing on a regular basis, I've had to add the following file to our project. I would really prefer not to do this:

from scipy.special import expit

def fix_logistic_cpu(be): try: from neon.backends.nervanacpu import numpy_call_dict_cpu assert not numpy_call_dict_cpu.get('sig', None) is None numpy_call_dict_cpu['sig'] = lambda left: expit(left) except Exception as e: from neon.backends.nervanacpu import numpy_call_dict assert not numpy_call_dict.get('sig', None) is None numpy_call_dict['sig'] = lambda left: expit(left)

def fix_logistic(be): fix_logistic_cpu(be)