how to use clahe in preprocess?

bei-startdt commented 5 years ago

hey my friend

in your paper, ROBUSTNESS ENHANCEMENTS ,'Histogram Equalization. Histogram equalization successfully standardizes speech data for robust speech recognition (Torre et al., 2005; Harvilla & Stern, 2012). For images, we find that preprocessing with Contrast Limited Adaptive Histogram Equalization (Pizer et al., 1987) is quite effective. Unlike our image denoising attempt (Appendix F), CLAHE reduces the effect of some corruptions while not worsening performance on most others, thereby improving the mCE. We demonstrate CLAHE’s net improvement by taking a pre-trained ResNet-50 and fine-tuning the whole model for five epochs on images processed with CLAHE. The ResNet-50 has a 23.87% error rate, but ResNet-50 with CLAHE has an error rate of 23.55%. On nearly all corruptions, CLAHE slightly decreases the Corruption Error. The ResNet-50 without CLAHE preprocessing has an mCE of 76.7%, while with CLAHE the ResNet-50’s mCE decreases to 74.5%.'

in your repo, I can't find clahe

I wanna know, how to use it?

I tried two usage:

1.

clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
for i in range(3):
    img[:,:,i] = clahe.apply(img[:,:,i])

clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2LAB)
frame[:,:,0] = clahe.apply(frame[:,:,0])
frame = cv2.cvtColor(frame, cv2.COLOR_LAB2BGR)

for the one part, I don't know which usage is correct
for the other, I don't know which number clipLimit,tileGridSize should be set

Could you help me with the answer? Thanks

hendrycks commented 5 years ago

I used CLAHE from skimage.exposure.equalize_adapthist. That implementation has rescaling which cv2 does not have. However cv2 is much faster, so an implementation with cv2 can look like this:

def rescale(x):
    x_min, x_max = x.min(), x.max()
    return np.uint8(np.clip((x - x_min)/float(x_max - x_min + 1e-8), 0, 1) * 255)

clahe = cv2.createCLAHE(clipLimit=2 tileGridSize=(8, 8))

class HistEq(object):
    # def __init__(self):

    def __call__(self, x):
        x = cv2.cvtColor(np.array(x), cv2.COLOR_RGB2LAB)
        l = x[:,:,0]
        lmin, lmax = l.min(), l.max()
        return cv2.cvtColor(np.concatenate((clahe.apply(np.uint8((l - lmin) / (lmax - lmin + 1e-8) * 255))[...,None], x[:,:,1:]), axis=2),
                            cv2.COLOR_LAB2RGB)

I can e-mail you the weights for the fine-tuned model if you want. As it happens, I recently updated the paper to mention a more exciting technique that beats CLAHE significantly. Fine-tuning with "Stylized ImageNet" significantly outperforms fine-tuning with CLAHE applied to images.

bei-startdt commented 5 years ago

Thank U very much.

hendrycks / robustness