search

deephealthproject / pyecvl

Python wrapper for the ECVL.
MIT License
4 stars 2 forks source link

Error with DLDataset: Unable to convert function return value to a Python type #72

Closed thistlillo closed 2 years ago

thistlillo commented 2 years ago

Hello, I have a couple of questions on the ECVL dataset using PyECVL. The dataset has this form:

name: ECVL dataset for UC5

description: eddl_ext

classes: [4, 8, 14, 16, 17, 20]

images:
  - location: ../data/image/CXR3919_IM-1992-1001.png
    label: [4]
    values: [1, 351, 84, 16, 6, 32, 3, 89, 309, 6, 93, 2]
  - location: ../data/image/CXR3967_IM-2028-2001.png
    label: [14]
    values: [1, 4, 33, 31, 5, 21, 13, 53, 42, 18, 8, 2]
  - location: ../data/image/CXR527_IM-2137-1001.png
    label: [16]
    values: [1, 50, 8, 28, 12, 52, 13, 3, 95, 25, 17, 2]
  - location: ../data/image/CXR2189_IM-0798-2001.png
    label: [16]
    values: [1, 15, 18, 8, 28, 56, 52, 21, 13, 53, 20, 2]
  - location: ../data/image/CXR345_IM-1672-1001.png
    label: [16]
    values: [1, 33, 31, 8, 20, 69, 6, 21, 13, 53, 16, 2]
  - location: ../data/image/CXR3912_IM-1988-2001-0001.png
    label: [4]
    values: [1, 161, 5, 396, 200, 4, 121, 12, 8, 143, 121, 2]
  - location: ../data/image/CXR1109_IM-0076-2001.png
    label: [16]
    values: [1, 15, 18, 8, 28, 56, 52, 21, 13, 53, 20, 2]
  - location: ../data/image/CXR551_IM-2154-2001.png
    label: [8, 20]
    values: [1, 16, 6, 225, 27, 304, 7, 4, 476, 8, 125, 2]
  - location: ../data/image/CXR587_IM-2182-3001.png
    label: [17]
    values: [1, 4, 33, 31, 5, 21, 13, 53, 42, 132, 3, 2]
split:
  training: [0, 1, 2]
  validation: [3, 4, 5]
  test: [6, 7, 8]

The numbers in values are word indexes of the text associated to the images.

When I try and access the data with these lines:

[...]
augs = ecvl.DatasetAugmentations(augs=[train_augs, test_augs, test_augs])
dataset = ecvl.DLDataset(out_fn, batch_size=1, augs=None, ctype=ecvl.ColorType.RGB, ctype_gt=ecvl.ColorType.GRAY, num_workers=2, queue_ratio_size=2, drop_last=drop_last)
dataset.SetSplit(ecvl.SplitType.training)
dataset.ResetAllBatches(shuffle=True)
dataset.Start()
a, b, c = dataset.GetBatch()
dataset.Stop()
print("done.")

I get this error: 

Unable to convert function return value to a Python type! The signature was
        (self: pyecvl._core.ecvl.DLDataset) -> Tuple[List[pyecvl._core.ecvl.Sample], Tensor, Tensor]

with the following being the full stack trace:

Traceback (most recent call last):
  File "E_build_ecvl_dataset_str.py", line 120, in <module>
    fire.Fire(main)
  File "/root/miniconda3/envs/eddl/lib/python3.8/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/root/miniconda3/envs/eddl/lib/python3.8/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/root/miniconda3/envs/eddl/lib/python3.8/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "E_build_ecvl_dataset_str.py", line 114, in main
    a, b, c = dataset.GetBatch()
  File "/root/miniconda3/envs/eddl/lib/python3.8/site-packages/pyecvl/support_eddl.py", line 192, in GetBatch
    return _ecvl.DLDataset.GetBatch(self)
TypeError: Unable to convert function return value to a Python type! The signature was
        (self: pyecvl._core.ecvl.DLDataset) -> Tuple[List[pyecvl._core.ecvl.Sample], Tensor, Tensor]
Makefile_eddl_ext.mk:342: recipe for target 'ecvl_dataset.yml' failed
make: *** [ecvl_dataset.yml] Segmentation fault (core dumped)
make: *** Deleting file 'ecvl_dataset.yml'

I cannot understand what I am doing wrong. I looked at other use cases using the ECVL DLDataset but could not spot any differences. The error occurs even if I remove the values from each image.

Using PyEDDL/cudnn version 1.2,0 and PyECVL/cudnn version 1.1.0, installed via Conda on linux.

simleo commented 2 years ago

This happens when a function returns an EDDL type but PyEDDL has not been imported. Add import pyeddl.eddl or import pyeddl.tensor to the program. I've updated the docs with this info.

thistlillo commented 2 years ago

Thank you, indeed I used the loader without problems with previous versions for accessing images. I was evaluating how easy is to manage the word indexes in Sample and did not import Tensor up to that point.