IBM / heart-library

Hardened Extension of the Adversarial Robustness Toolbox (HEART) supports assessment of adversarial AI vulnerabilities in Test & Evaluation workflows
MIT License
7 stars 0 forks source link

Known Issue: python==2.4.1 pickling error (Can't pickle <built-in function reset_code>) when executing AutoAttack with parallel=true #1

Open lockwoodar opened 1 month ago

lockwoodar commented 1 month ago

Description of Issue:

Private GitLab internal identifier for this issue is #319

Upstream adversarial-robustness-toolbox AutoAttack fails with _pickle.PicklingError: Can't pickle <built-in function reset_code>: it's not found as torch._C._dynamo.eval_frame.reset_code when executing with parallel=true

Testing Environment

dependency version
python 3.10
adversarial-robustness-toolbox 1.18.1
heart-library 0.4.0
pytorch 2.4.0
OS Ubuntu 22.04.4

Unit Test Stack Trace

# test_parallel_auto_attack ---------------------------------------------------------

heart_warning = <function heart_warning.<locals>._heart_warning at 0x7fdd7ecc83a0>

    def test_parallel_auto_attack(heart_warning):
        try:
            from art.attacks.evasion.auto_attack import AutoAttack
            from art.attacks.evasion.projected_gradient_descent.projected_gradient_descent_pytorch import (
                ProjectedGradientDescentPyTorch,
            )
            import numpy as np

            labels = ["airplane", "automobile", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck"]
            ptc = get_cifar10_image_classifier_pt(from_logits=True, is_jatic=False)

            (x_train, y_train), (_, _), _, _ = load_dataset("cifar10")
            x_train = x_train[:10].transpose(0, 3, 1, 2).astype("float32")
            y_train = y_train[:10]

            attacks = []
            attacks.append(
                ProjectedGradientDescentPyTorch(
                    estimator=ptc, norm=np.inf, eps=0.1, max_iter=10, targeted=False, batch_size=32, verbose=False
                )
            )
            jatic_attack_noparallel = AutoAttack(estimator=ptc, attacks=attacks, targeted=True, parallel=False)
            jatic_attack_parallel = AutoAttack(estimator=ptc, attacks=attacks, targeted=True, parallel=True)
            core_attack = AutoAttack(estimator=ptc, attacks=attacks, targeted=True)

            no_parallel_adv = jatic_attack_noparallel.generate(x=x_train, y=y_train)
>           parallel_adv = jatic_attack_parallel.generate(x=x_train, y=y_train)

# multiprocess----------------------------------------------

self = <multiprocess.reduction.ForkingPickler object at 0x7fdd7f125060>, obj = <built-in function reset_code>, name = 'reset_code'

    def save_global(self, obj, name=None):
        write = self.write
        memo = self.memo

        if name is None:
            name = getattr(obj, '__qualname__', None)
        if name is None:
            name = obj.__name__

        module_name = whichmodule(obj, name)
        try:
            __import__(module_name, level=0)
            module = sys.modules[module_name]
            obj2, parent = _getattribute(module, name)
        except (ImportError, KeyError, AttributeError):
>           raise PicklingError(
                "Can't pickle %r: it's not found as %s.%s" %
                (obj, module_name, name)) from None
E           _pickle.PicklingError: Can't pickle <built-in function reset_code>: it's not found as torch._C._dynamo.eval_frame.reset_code

../../miniconda3/envs/heart-env/lib/python3.10/pickle.py:1071: PicklingError

FAILED tests/attacks/test_auto_attack.py::test_parallel_auto_attack - _pickle.PicklingError: Can't pickle <built-in function reset_code>: it's not found as torch._C._dynamo.eval_frame.reset_code

Current Workaround

Roll back platform dependency to pytorch<=2.3.1