MB melgan batch inference but inference speed has not improved

[xuezhongfei2008]

start_time=time.time() audio=mblemgan.inference(mels) print("use_time:",time.time()-start_time)

test1: mels.shape (1, 13040, 80) use_time: 1.916731595993042

test2: mels.shape (40, 326, 80) use_time: 1.9499073028564453

[dathudeptrai]

you need warm-up tensorflow model around 5 iterations before calculate its real inference time (tf model always slow at some first iterations)

[xuezhongfei2008]

mels = np.load(ff) num=int(40) mels=mels.repeat(num,axis=1) #feats.repeat(num)

mels = mels[None, ...] print("mels.shape",mels.shape) mels = mels.reshape([40,-1,80]) start_time=time.time() print("mels.shape",mels.shape) for i in range(50): audio=mblemgan.inference(mels) if i == 25: start_time1=time.time() print("use_time:",time.time()-start_time) print("use_time2:",time.time()-start_time1)

mels.shape (1, 326, 3200) mels.shape (40, 326, 80) use_time: 106.86395192146301 use_time2: 51.141302824020386

Front 25 iterations and last 25 iterations time consumed is the same

[TheHonestBob]

mels = np.load(ff) num=int(40) mels=mels.repeat(num,axis=1) #feats.repeat(num)

mels = mels[None, ...] print("mels.shape",mels.shape) mels = mels.reshape([40,-1,80]) start_time=time.time() print("mels.shape",mels.shape) for i in range(50): audio=mblemgan.inference(mels) if i == 25: start_time1=time.time() print("use_time:",time.time()-start_time) print("use_time2:",time.time()-start_time1)

mels.shape (1, 326, 3200) mels.shape (40, 326, 80) use_time: 106.86395192146301 use_time2: 51.141302824020386

Front 25 iterations and last 25 iterations time consumed is the same

ur code is wrong, mels.shape (1, 13040, 80) == mels.shape (40, 326, 80), batch inference is mels.shape (10, 13040, 80)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.