Slicing Process not defined

[avemio-digital]

layer_distances.csv Hi,

We were trying to prune the Gemma 2b model. We already calculated the layer_similarity_distances of the model. But now the problem is we don't figure out how to define the layer ranges? We have researched all of your repository but couldn't find any lead on this. Kindly update your response on this regard. I have also uploaded the CSV of layer similarilty distances as a reference .

slices:

• sources:
  • model: google/gemma-2b-it layer_range: [0, 21] < How to select the layer range from our mentioned and uploaded file?
• sources:
  • model: google/gemma-2b-it layer_range: [29,32] < How to select the layer range from our mentioned and uploaded file?

merge_method: passthrough dtype: bfloat16

[shamanez]

Here’s how you can define layer ranges:

1. Identify Minimum Average Distance: This value usually represents layers that are less similar to each other, thus good candidates for pruning or reconfiguration. From your data, the minimum average_distance is around 0.265 for the layer range from block_start 3 to block_end 11.

2. Configuring the Layer Range:

   • You select layer ranges based on the block_start and block_end values. These define the start and end of a block or a group of layers in your model where the average distance between layers was calculated.
   • For the smallest value from your data (0.2651205078125), the corresponding layer range would be from 3 to 11.

Based on this, if you're looking to modify your configuration for pruning, you could adjust the layer_range field as follows:

sources:
  - model: google/gemma-2b-it
    layer_range: [0, 3]  # Adjusted to select the initial layers
  - model: google/gemma-2b-it
    layer_range: [12, 32]  # Adjusted to select layers from 12 to the end at 32
merge_method: passthrough
dtype: bfloat16

Note: Adjust the indexing based on whether your model's layers start counting from 0 or 1. Here, I adjusted them to be zero-indexed (typical in many programming environments).

If you need further assistance with this configuration or have other questions about model pruning, feel free to ask!

[avemio-digital]

Thanks a lot for your detailed answer.