search

NVIDIA / DALI

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html
Apache License 2.0
5.09k stars 615 forks source link

Error when executing GPU operator GridMask encountered #2973

Closed dariodematties closed 3 years ago

dariodematties commented 3 years ago

Hi there,

I am trying to use grid mask in my pipe self.grid_mask = ops.GridMask(device='gpu') I use some external inputs to feed the operation and then inside define_graph images = self.grid_mask(images, angle=angle, shift_x=crop_pos_x, shift_y=crop_pos_y, ratio=grid_mask_ratio, tile=grid_mask_tile) I receive the following error

RuntimeError: Critical error in pipeline:
Error when executing GPU operator GridMask encountered:
[/opt/dali/dali/pipeline/data/buffer.h:114] Assert on "type_.id() == TypeTable::GetTypeID<T>()" failed: Calling type does not match buffer data type: int32 v. float

Any Idea of what is happening here?

I am using the following container

pytorch_21.04-py3.sif with mpi4py added

Thanks!

Dario

JanuszL commented 3 years ago

Hi @dariodematties,

I suspect that is about grid_mask_tile argument which expects int32 input while you provide float. By default, numpy uses float type. Can you make sure that the data provided by the external input to this argument is int32?

dariodematties commented 3 years ago

You are right @JanuszL ! Thanks so much for your patience