search

rfuruta / pixelRL

137 stars 31 forks source link

What is the moved_image used for? Can you explain the function of state.step() a bit more? #6

Open machanic opened 4 years ago

machanic commented 4 years ago

What does moved_image = self.image + move[:,np.newaxis,:,:] mean, which is located in in https://github.com/rfuruta/pixelRL/blob/master/denoise/State.py#L17

Can you explain the function of def step(self, act) a bit more to make it easier to understand? It is located in https://github.com/rfuruta/pixelRL/blob/master/denoise/State.py#L13

rfuruta commented 4 years ago

move_range indicates the number of actions that move the pixel values. e.g., when move_range=3, there are three actions: pixel_value+=1, +=0, and -=1. When move_range=5, there are five actions: pixel_value +=2, +=1, +=0, -=1, and -=2.

act is the action map whose element indicates the index of chosen action at each pixel. Let us consider the case when move_range=3. act[b, y, x] = 0, 1, ... , or 8 where b denotes the b-th image in the minibatch, and y, x denote the indices of row and col, respectively. At the pixel where act[b, y, x] == 0, the pixel value is changed 'pixel_value -= 1' (Line 14-17 in State.py ). https://github.com/rfuruta/pixelRL/blob/e0f5d35f4303724d90c1f5c6d9663c07284419e3/denoise/State.py#L14-L17 Similarly, pixel_value +=0 is performed at the pixel where act[b, y, x] == 1, and pixel_value += 1 is performed at the pixel where act[b, y, x] == 2 in Line 14-17. The variable move is divided by 255 in Line 16 because the pixel values are normalized into the range [0, 1].

act[b, y, x] == 3, 4, 5, 6, ,7 and 8 correspond to each action: Gaussian filter, bilateral filter, median filter, another Gaussian filter (with different parameters), another bilateral filter (with different parameters), and box filter, respectively. In Line 26-27, Gaussian filter is performed on the entire image if at least one act[b, y, x] == 3 exists. Similar process is performed in Line 28-37. Finally, in Line 40, the pixel value after Gaussian filter is performed is chosen at the pixel where act[b, y, x] == 3. Similar process of the other filters is performed in Line 41-45.

rgeng5 commented 2 years ago

Thank you for the detailed explanation. I hope to generate the action maps to reproduce the Fig.2 and 4 in your 2019 paper. Where would the best place to output them?

rfuruta commented 2 years ago

https://github.com/rfuruta/pixelRL/blob/e0f5d35f4303724d90c1f5c6d9663c07284419e3/denoise/test.py#L54 By adding the following code to line 54 in test.py, you can save the action maps.

a = action.astype(np.uint8)
a = np.transpose(a, (1,2,0))
cv2.imwrite(SAVE_PATH+str(i)+'_'+str(t)+'_action.png', a)

In the saved action maps, each pixel value has the index of the chosen action. 0: pixel value -= 1 1: do nothing 2: pixel value += 1 3: Gaussian filter (sigma=0.5) .... (see State.py for more details) After saving the action maps, we colorized them by (a little modified version of) Pascal VOC color map.