Why do we need `np.vectorize(sigmoid_double)(z)`?

[fuhaoda]

Hi,

I was trying to understand the codes in Chapter 5. Why do we need np.vectorize(sigmoid_double)(z) in the layers.py file? It seems that it is about 100x slower than directly broadcasting the function. Here is a simple comparison, please let me know if I missed something here.

import numpy as np

def sigmoid_d(x):
    return 1.0/(1.0+np.exp(-x))

def sigmoid(z):
    return np.vectorize(sigmoid_d)(z)

import time

def measureRunTime(fun):
    import time
    def wrapper(*arg, **kwargs):
        print("{} starts running".format(fun.__name__))
        time0 = time.time()
        result = fun(*arg, **kwargs)
        time1 = time.time()-time0
        print("{} ran in {} seconds".format(fun.__name__, time1))
        return result
    return wrapper

rv = np.random.RandomState(0)
x = rv.random(1000000)

@measureRunTime
def myfun1(v):
    return sigmoid(v)

@measureRunTime
def myfun2(v):
    return sigmoid_d(v)

z1 =myfun1(x)
z2 = myfun2(x)

Output

myfun1 starts running
myfun1 ran in 0.9834280014038086 seconds
myfun2 starts running
myfun2 ran in 0.009630918502807617 seconds

[maxpumperla]

@fuhaoda yeah, if you use the vectorized version on single values, it will of course be slower (although I didn't expect 100x to be fair). but the point of vectorizing is that, well, we want to apply this to a vector (the result of the feed-forward layer). Have you tried replacing the vectorized version with the standard one in the actual network training code? you should see that fail.

[fuhaoda]

Max - Thanks for the reply.

As you see from the above example, both functions can be applied to vectors (x is a vector in above example, i.e although sigmoid_d is defined on a single value, it can automatically applied to a vector). In addition, I also tried the codes as you suggested, codes run OK.

[maxpumperla]

oh, I misread your code. you're right!

now I wonder why I did that originally, probably an earlier version that got refactored that needed the vectorized version. hard to reconstruct.