Error while applying custom transforms

[AmmaraRazzaq]

I am getting this error for a custom class the I have written

`File "../../../../../miniconda3/envs/ffcv5/lib/python3.8/site-packages/ffcv/transforms/module.py", line 25: def applymodule(inp, ): res = self.module(inp) ^

During: resolving callee type: type(CPUDispatcher(<function ModuleWrapper.generate_code..apply_module at 0x7f8c63dc0940>)) During: typing of call at (2)

During: resolving callee type: type(CPUDispatcher(<function ModuleWrapper.generate_code..apply_module at 0x7f8c63dc0940>)) During: typing of call at (2)

File "/nfs/users/ext_mohammed.haashir/projects/teacherstudentmodelv1.2/TMwithCheXpert/notebooks/ffcv", line 2: <source missing, REPL/exec in use?>`

def border_pad(image, long_side):
    h, w, c = image.shape

    vertical = int((long_side-h)/2)
    horizontal = int((long_side - w)/2)
    image = np.pad(image,((vertical, long_side-h-vertical),
                            (horizontal, long_side-w-horizontal), (0, 0)),
                    mode='constant',
                    constant_values=0
                    )
    return image

import torch 
import numbers
import torch.functional as F

class BorderPad(torch.nn.Module):
    def __init__(self, long_side=512):
        super().__init__()

        if not isinstance(long_side, (numbers.Number)):
            raise TypeError("Got invalid long_side argument")

        self.long_side = long_side

    def forward(self, img):
        return border_pad(img, self.long_side)

    def __repr__(self):
         return self.__class__.__name__ + '(long_side={})'.format(self.long_side) 

loaders = {}
for name in ['train', 'val']:
    label_pipeline: List[Operation] = [NDArrayDecoder()]
    image_pipeline: List[Operation] = [SimpleRGBImageDecoder()]

    if name == 'train':
        image_pipeline.extend([
            BorderPad(long_side=600)
        ])

    loaders[name] = Loader(
        paths[f'{name}_beton_path'],
        batch_size=14,
        num_workers=6,
        order=OrderOption.RANDOM,
        drop_last=(name=='train'),
        pipelines={
            'image': image_pipeline,
            'label': label_pipeline
        }
    )

[AmmaraRazzaq]

Converting the image toTensor and toTorchImage worked. However, since I have written the custom class it is not handling the whole batch with dimensions BxCxHxW, instead it is expecting one image and I am getting the following error

Exception in thread Thread-21: Traceback (most recent call last): File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/site-packages/ffcv/loader/epoch_iterator.py", line 79, in run result = self.run_pipeline(b_ix, ixes, slot, events[slot]) File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/site-packages/ffcv/loader/epoch_iterator.py", line 133, in run_pipeline result = code(args) File "", line 2, in stage_1 File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/site-packages/ffcv/transforms/module.py", line 25, in apply_module res = self.module(inp) File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(input, **kwargs) File "/tmp/ipykernel_76500/3009548602.py", line 16, in forward File "/tmp/ipykernel_76500/290271050.py", line 2, in border_pad ValueError: too many values to unpack (expected 3)

[GuillaumeLeclerc]

Hello!

Your augmentation is implemented in numpy. This is great because FFCV is able to run numpy code extremely fast. However Pytorch code on the CPU can't be optimized by FFCV and it will run really slowly (and use a single worker). You shouldn't wrap your numpy code into a torch.nn.Module but use a ffcv.pipeline.Operation instead. This way you will be able to run FFCV at full speed and use multiple workers. We have many examples in this repo. A good example would be: https://github.com/libffcv/ffcv/blob/main/ffcv/transforms/cutout.py

[AmmaraRazzaq]

Hi, Thankyou for your response.

Writing custom transformations in numpy code and wrapping it into ffcv.pipeline.Operation works. However for this particular border pad function I am getting the following error message > Use of unsupported NumPy function 'numpy.pad' or unsupported use of the function.

Exception in thread Thread-5: Traceback (most recent call last): File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/site-packages/ffcv/loader/epoch_iterator.py", line 79, in run result = self.run_pipeline(b_ix, ixes, slot, events[slot]) File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/site-packages/ffcv/loader/epoch_iterator.py", line 133, in run_pipeline result = code(*args) File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/site-packages/numba/core/dispatcher.py", line 482, in _compile_for_args error_rewrite(e, 'typing') File "/nfs/users/ext_mohammed.haashir/miniconda3/envs/ffcv5/lib/python3.8/site-packages/numba/core/dispatcher.py", line 423, in error_rewrite raise e.with_traceback(None) numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend) Failed in nopython mode pipeline (step: nopython frontend) Use of unsupported NumPy function 'numpy.pad' or unsupported use of the function. File "pyfiles/transformations.py", line 29: def border_pad(images, dst):

horizontal = int((long_side - w)/2) dst[i] = np.pad(images[i], ((vertical, long_side-h-vertical), (horizontal, long_side-w-horizontal), (0, 0)), ^

During: typing of get attribute at /nfs/users/ext_mohammed.haashir/projects/teacherstudentmodelv1.2/TMwithCheXpert/notebooks/ffcv/pyfiles/transformations.py (29)

File "pyfiles/transformations.py", line 29: def border_pad(images, dst):

            horizontal = int((long_side - w)/2)
            dst[i] = np.pad(images[i], ((vertical, long_side-h-vertical), (horizontal, long_side-w-horizontal), (0, 0)),
            ^

During: resolving callee type: type(CPUDispatcher(<function BorderPad.generate_code..border_pad at 0x7fa5ec5590d0>)) During: typing of call at (2) During: resolving callee type: type(CPUDispatcher(<function BorderPad.generate_code..border_pad at 0x7fa5ec5590d0>)) During: typing of call at (2) File "/nfs/users/ext_mohammed.haashir/projects/teacherstudentmodelv1.2/TMwithCheXpert/notebooks/ffcv", line 2: <source missing, REPL/exec in use?>

This is the class I wrote

class BorderPad(Operation):

    def __init__(self, long_side: int = 512):
        super().__init__()
        self.long_side = long_side

    def generate_code(self) -> Callable:
        my_range = Compiler.get_iterator()
        long_side = self.long_side

        def border_pad(images, dst):
            h, w, _ = images[0].shape
            print('image shape: ', images[0].shape)

            for i in my_range(images.shape[0]):
                vertical = int((long_side-h)/2)
                horizontal = int((long_side - w)/2)
                dst[i] = np.pad(images[i], ((vertical, long_side-h-vertical), (horizontal, long_side-w-horizontal), (0, 0)),
                                mode='constant',
                                constant_values=0
                                )
            return dst

        border_pad.is_parallel = True
        return border_pad

    def declare_state_and_memory(self, previous_state: State) -> Tuple[State, Optional[AllocationQuery]]:
        return (replace(previous_state, jit_mode=True),
                AllocationQuery(previous_state.shape, previous_state.dtype))

[GuillaumeLeclerc]

Hello,

You don't really need np.pad to do what you are trying to achieve:

• Fill dst with 0
• Write the source image wherever you want in dst. It will essentially be padded.

If you need the padded image to be bigger, you can make your AllocationQuery to have a bigger shape than the original image.