albumentations-team / albumentations

Fast and flexible image augmentation library. Paper about the library: https://www.mdpi.com/2078-2489/11/2/125
https://albumentations.ai
MIT License
14.06k stars 1.64k forks source link

pickled Transforms from pre 1.2.0 breaks after 1.2.0 #1337

Open Erotemic opened 1 year ago

Erotemic commented 1 year ago

🐛 Bug

I have an older torch.package model where the author baked in a few albumentations objects. Using the latest version of albumentations breaks because albumentations.core.composition.Transforms no longer exists.

Here is a MWE:

import albumentations as A
import pathlib
import pickle
import sys

class MyObject:
    def __init__(self):
        self.transforms = A.Compose([
            A.GaussianBlur(),
        ])

def save():
    obj = MyObject()
    fpath = pathlib.Path('./test.pickle')
    fpath.write_bytes(pickle.dumps(obj))

def load():
    fpath = pathlib.Path('./test.pickle')
    obj = pickle.loads(fpath.read_bytes())
    print(f'obj={obj}')

if __name__ == '__main__':
    """
    CommandLine:

        cd ~/code/albumentations
        pip install -e .

        git checkout 1.1.0
        python mwe.py save

        python mwe.py load

        git checkout 1.3.0
        python mwe.py load

    """
    if 'save' in sys.argv:
        save()
    if 'load' in sys.argv:
        load()

In the above code, I have a copy of this repo, and I checkout the 1.1.0 tag, and run mwe.py save to save the object. Running mwe.py load works fine in this branch. But when I update to a later branch e.g. 1.3.0, running load results in:

(pyenv3.10.5) joncrall@toothbrush:~/code/albumentations$         python mwe.py load
Traceback (most recent call last):
  File "/home/joncrall/code/albumentations/mwe.py", line 45, in <module>
    load()
  File "/home/joncrall/code/albumentations/mwe.py", line 22, in load
    obj = pickle.loads(fpath.read_bytes())
AttributeError: Can't get attribute 'Transforms' on <module 'albumentations.core.composition' from '/home/joncrall/code/albumentations/albumentations/core/composition.py'>

Is there a monkey patch I can make that allows me to load these older pickles without having to pin my version of albumentations to something old? Alternatively (although less desirable), is there a way to update the pickle file so it works with newer versions?

Dipet commented 1 year ago

The simplest way - load transforms using old library version and save them using ours serialization to json/yaml. You can then load these transforms from newer versions of the library. https://albumentations.ai/docs/examples/serialization/#Serializing-an-augmentation-pipeline-to-a-JSON-or-YAML-file