Open Vincouux opened 1 year ago
Hello @Vincouux ,
Thank you for reaching out.
First thing I see is that you don't need to wrap the mean and stddev in the types.Constant
:
# Normalizing (images - mean) / std.
images = dali.fn.normalize(
images,
batch=False,
mean=[0.485, 0.456, 0.406],
stddev=[0.229, 0.224, 0.225]
)
If this doesn't help, could you please post the error message, so I'd know more what's going on?
Here is the error I am facing:
Traceback (most recent call last):
File "<string>", line 5, in <module>
File "<frozen importlib._bootstrap>", line 553, in module_from_spec
AttributeError: 'NoneType' object has no attribute 'loader'
Traceback (most recent call last):
File "<string>", line 8, in <module>
File "/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/site-packages/nvidia/dali/_utils/autoserialize.py", line 72, in invoke_autoserialize
dali_pipeline().serialize(filename=filename)
File "/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/site-packages/nvidia/dali/pipeline.py", line 1471, in create_pipeline
pipe_outputs = func(*args, **fn_kwargs)
File "/models/detection_exp114_20220523_preprocessing/1/dali.py", line 21, in pipe
images = dali.fn.normalize(
File "/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/site-packages/nvidia/dali/fn.py", line 88, in fn_wrapper
return op_wrapper(*inputs, **kwargs)
File "/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/site-packages/nvidia/dali/fn.py", line 80, in op_wrapper
return op_class(**init_args)(*inputs, **call_args)
File "/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/site-packages/nvidia/dali/ops.py", line 621, in __init__
_add_spec_args(self._schema, self._spec, kwargs)
File "/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/site-packages/nvidia/dali/ops.py", line 398, in _add_spec_args
converted_value = _type_convert_value(dtype, value)
File "/opt/tritonserver/backends/dali/conda/envs/dalienv/lib/python3.8/site-packages/nvidia/dali/types.py", line 110, in _type_convert_value
return _known_types[dtype][1](val)
TypeError: float() argument must be a string or a number, not 'list'
To give a bit more context, the input image is of shape [-1, -1, 3]. After the resize and the permutation, it should be [3, 224, 224] based on the above pipeline I provided.
Many thanks @szalpal for you help !!!
@Vincouux
Generally, for image normalization it'd be better if you'd go for fn.crop_mirror_normalize
instead of fn.normalize
. For two reasons, one is that the former fuses 3 possible operations in one and the second is that the API would be more user friendly.
Therefore I'd advice you to use this instead of fn.normalize
:
images = dali.fn.crop_mirror_normalize(images,
dtype=types.FLOAT,
output_layout="CHW",
crop=(224, 224),
mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
Should you have any more problems, don't hesitate to reach us out. Good luck in using DALI ;)
Furthermore, if you'd like to squeeze even more perf from this snippet, I'd suggest, that you plug the Rescaling 0-255 to 0-1.
into the normalization by multiplying your normalization parameters by 255
:
mean=[0.485 * 255, 0.456 * 255, 0.406 * 255],
std=[0.229 * 255, 0.224 * 255, 0.225 * 255])
Hi all, I am super new to Dali Backend, but the benchmark I recently ran are incredible. I am trying to preprocess the data with Dali backend, but struggle with the "normalize" operation. I would like to specify a "per channel" mean & stddev ([c1, c2, c3] for example), but keep getting errors.
Here is what I've tried so far:
Here is the full code of my pipeline:
Many thanks !!