When adaptive cropping, my l2 norm is very high

[helin0815]

What is your question?

def _get_update_norm(update: NDArrays) -> float: flattened_update = update[0] for i in range(1, len(update)): flattened_update = np.append(flattened_update, update[i]) # type: ignore return float(np.sqrt(np.sum(np.square(flattened_update))))

def add_gaussian_noise(update: NDArrays, std_dev: float) -> NDArrays: """Add iid Gaussian noise to each floating point value in the update.""" update_noised = [ layer + np.random.normal(0, std_dev, layer.shape) for layer in update ] return update_noised

def clip_by_l2(update: NDArrays, threshold: float) -> Tuple[NDArrays, bool]: """Scales the update so thats its L2 norm is upper-bound to threshold.""" update_norm = _get_update_norm(update) print("update_norm:",update_norm) scaling_factor = min(1, threshold / update_norm) update_clipped: NDArrays = [layer * scaling_factor for layer in update] return update_clipped, (scaling_factor < 1) In this section of the code, The update_norm（by function clip_by_l2） value is very large, 4720.949363466235, but my threshold value is 0.1, which leads to significant cropping. Is this reasonable.

[helin0815]

The dataset I am using is CIFAR10, and the model is NVIDIA Efficientnet B0, the model is loaded using the following code：def load_efficientnet(entrypoint: str = "nvidia_efficientnet_b0", classes: int = None): """Loads pretrained efficientnet model from torch hub. Replaces final classifying layer if classes is specified.

Args:
    entrypoint: EfficientNet model to download.
                For supported entrypoints, please refer
                https://pytorch.org/hub/nvidia_deeplearningexamples_efficientnet/
    classes: Number of classes in final classifying layer. Leave as None to get the downloaded
             model untouched.
Returns:
    EfficientNet Model

Note: One alternative implementation can be found at https://github.com/lukemelas/EfficientNet-PyTorch
"""
efficientnet = torch.hub.load(
    "NVIDIA/DeepLearningExamples:torchhub", entrypoint, pretrained=True
)

if classes is not None:
    replace_classifying_layer(efficientnet, classes)
return efficientnet

[helin0815]

how to resolve: do not use NVIDIA Efficientnet B0,this model will generate too large l2 norm,use alexnet or other model to replace,do not use NVIDIA Efficientnet B0!!!!