1109 refactor augmentations/transformations

Ezward commented 1 year ago

Issue https://github.com/autorope/donkeycar/issues/1109

Transformations

Use standard parts for image transformations.

Refactor transformations to be simple parts that take an image and return an image.
- add a number of parts to cv.py to support a lot of transformations.
- image_transformations.py has a factory to create parts from a transformation name and the Config object and an class ImageTransformations that can take an array of transformation names and turn that into a transformation pipeline part.
Update train so that it can handle a set of transformation before augmentations are appliend and another after augmentations are applied. For instance, CROP or CANNY are better applied after augmentation.
Update complete.py to merge transform pipelines for use at inference time.
Can create one or more custom transformations and add them to the pipeline. See comments in https://github.com/autorope/donkeycar/blob/1ecc26e95ed1afe4d1a6840dfb0f26902ad97504/donkeycar/parts/image_transformations.py#L105 and https://github.com/autorope/donkeycar/blob/7ed5849fb1bd5e286a8b1122ee43c0e7c246c4fa/donkeycar/templates/cfg_complete.py#L427

Augmentations

augmentations.py

Refactor augmentations to use the albumentations library; this is a port of Dirk's branch.
This requires updating to latest opencv, so install yaml and setup.py are updated.

Deep Learning Template

complete.py

added the ability to show the transformed image that is used for inference when the autopilot is active. This is similar to what we do in the computer vision template in autopilot mode. By default this is turned off, but can be enabled by setting SHOW_PILOT_IMAGE to True.

Computer Vision Parts

cv.py

fixed a bug in CvCam where some cameras will produce an image that is not the size requested. Now the camera detects this and resizes if necessary.
fixed CvCam to wait for a image during warm up; addresses Issue https://github.com/autorope/donkeycar/issues/968
added parts to do color conversions.
added a part that will output a given image; used in self-test but could also be used as a mock camera
added a main() that can run most of the parts for testing

This should fix the imgaug bug https://github.com/autorope/donkeycar/issues/970 but that still needs to be tested in a Jetson.

Ezward commented 1 year ago

This works on my Mac with Python 3.8, but the package test fail with;

  Looking for: ['python=3.7', 'numpy==1.23.4', 'h5py', 'pillow', 'opencv', 'matplotlib', 'tornado', 'docopt', 'pandas', 'pylint', 'pytest', 'pytest-cov', 'codecov', 'pip', 'imgaug', 'progress', 'paho-mqtt', 'prettytable', 'pyfiglet', 'mypy', 'pytorch', 'torchvision=0.12', 'torchaudio', 'pytorch-lightning', 'psutil', 'kivy=2.0.0', 'plotly', 'pyyaml', 'fastai', 'pynmea2', 'pyserial', 'utm', 'albumentations', 'python=3.7']

  Could not solve for environment specs
  Encountered problems while solving:
    - package numpy-1.23.4-py310h9638375_0 requires python >=3.10,<3.11.0a0, but none of the providers can be installed

  The environment can't be solved, aborting the operation

Error: The process '/usr/local/miniconda/condabin/mamba' failed with exit code 1

DocGarbanzo commented 1 year ago

I see the CI failing in the pipeline test where a tf tensor to numpy conversion has some problems. Looking in the logs, I see numpy version 1.21 here, whereas our last commit on main was using 1.19, and that ran successfully. We actually might need to pin that. I’m wondering if albumentations are the reason for the numpy upgrade, I will check that. I’m sure its all fine once we are on tf 2.9.

Ezward commented 1 year ago

I replaced kernal -> kernel and KERNAL -> KERNEL. Thank you Colonel Dirk, my spelling is really bad.

autorope / donkeycar