IBM / aihwkit

IBM Analog Hardware Acceleration Kit
https://aihwkit.readthedocs.io
MIT License
365 stars 148 forks source link

How to Configure ReRAM as an RPU #684

Closed 24367452 closed 3 months ago

24367452 commented 3 months ago

I want to use ReRAM as an RPU unit for simulating CNNs, but I don't know what interface to use to configure it. At first, I thought that the noise model that configures the inference RPU also configures the RPU unit, as shown below:

    my_rpu_config = InferenceRPUConfig(noise_model=ReRamWan2022NoiseModel())
    my_rpu_config.mapping.digital_bias = True
    my_rpu_config.mapping.out_scaling_columnwise = True
    my_rpu_config.mapping.learn_out_scaling = True
    my_rpu_config.mapping.weight_scaling_omega = 1.0
    my_rpu_config.mapping.weight_scaling_columnwise = False
    my_rpu_config.mapping.max_input_size = 512
    my_rpu_config.mapping.max_output_size = 512

    my_rpu_config.noise_model = ReRamWan2022NoiseModel()
    my_rpu_config.remap.type = WeightRemapType.CHANNELWISE_SYMMETRIC
    my_rpu_config.clip.type = WeightClipType.LAYER_GAUSSIAN
    my_rpu_config.clip.sigma = 2.5

    # train input clipping
    my_rpu_config.forward.noise_management = NoiseManagementType.NONE
    my_rpu_config.forward.bound_management = BoundManagementType.NONE
    my_rpu_config.forward.out_bound = 10.0  # quite restrictive
    my_rpu_config.pre_post.input_range.enable = True
    my_rpu_config.pre_post.input_range.manage_output_clipping = True
    my_rpu_config.pre_post.input_range.decay = 0.001
    my_rpu_config.pre_post.input_range.input_min_percentage = 0.95
    my_rpu_config.pre_post.input_range.output_min_percentage = 0.95

    my_rpu_config.modifier.type = WeightModifierType.ADD_NORMAL
    my_rpu_config.modifier.std_dev = 0.1

    my_rpu_config.runtime.data_type = DATA_TYPE

But then I realised that it seems that using

rpu_config = SingleRPUConfig(device=ReRamESPresetDevice())

this is the way to configure ReRAM as an RPU unit.

In 06 Example (Hardware-aware training for simulated CNN - https://github.com/IBM/aihwkit/blob/master/examples/06_lenet5_hardware_aware.py#L79), SingleRPUConfig is not used, so how is the RPU unit configured in this exampleit?Does it use ReRAM or PCM? If you know the right way to configure ReRAM as an RPU unit in CNN, please let me know, thank you very much!

24367452 commented 3 months ago

How do I configure the RPU to be ReRAM when reasoning with my_rpu_config = InferenceRPUConfig(noise_model=ReRamWan2022NoiseModel())?

maljoras commented 3 months ago

Closing, since it is a duplicate see #686