search

intel / ideep

Intel® Optimization for Chainer*, a Chainer module providing numpy like API and DNN acceleration using MKL-DNN.
MIT License
165 stars 90 forks source link

Add ideep wrapper for the new serialize API in oneDNN v3.4.1 #305

Closed chunyuan-w closed 7 months ago

chunyuan-w commented 7 months ago

Pitch

Add a wrapper layer for the memory desc serialization and deserialization APIs in oneDNN v3.4.1 into ideep. Update the IDEEP_VERSION_REVISION to 2.

Description

Starting from oneDNN v3.4.1, the oneDNN memory descriptor serialization API has been introduced (9b848c859).

We added two new APIs in ideep accordingly:

  1. For the serialization: std::vector<uint8_t> get_blob() is added for get_blob() of oneDNN md.

  2. For the deserialization, desc(const std::vector<uint8_t> &blob) is added for desc(const std::vector &blob) of oneDNN md.

For groups, we need to add it into the serialized ideep desc. For example, a 4D FWK weight with groups > 1 will become a 5D oneDNN md, while FWK still needs it to be a 4D tensor. ideep desc will wrap the 5D oneDNN md back to a 4D ideep desc when interacting with the FWK, thus we insert this groups information into the end of the blob during the serialization. During the deserialization, we exclude these groups bytes when constructing the oneDNN md and save the groups into the constructed ideep desc.

These two new APIs will be used in AOTInductor to save and load the prepacked oneDNN weights that're in opaque format.

Limitation

We have the below two assumptions

  1. The sizeof(int) (groups is of int type) is the same on the machine during the serialization and the deserialization.
  2. The machine of the serialization and the deserialization are little endian.
chunyuan-w commented 7 months ago

@jgong5 could you please help review this PR?