[Feature request] Add initializers to session options for Generator model C++

[martinkorelic]

Feature request To add more customizable models with different initializers, it would be very useful to access initializers and modify them when creating the object. This would only require to expose the session options property to the OgaModel or the generation model.

Additional context Is such an option already available in C++? I haven't observed that we can access this, but it would be very useful to access and edit inference session options after the generation model has been created.

[yufenglee]

We don't expose SessionOption through API now, but you can config the session option through config file (https://onnxruntime.ai/docs/genai/reference/config.html#example-file-for-phi-2). And why do you want to modify the initializer? for multi-lora?

[martinkorelic]

I have some layers which need to be loaded externally separately to the session options before the inference session is created, that's why this would be useful to have, or at least to pass SessionOptions as a parameter to the model constructor before the inference session is created.

[yufenglee]

we don't have support for this now. One workaround is that you can make those initializers as input of the models, and then you can allocate/create tensors in advance and pass them to InferenceSession through this API: https://github.com/microsoft/onnxruntime-genai/blob/c7eba3c63a454edd6662eb007ff397d1146cc081/src/ort_genai_c.h#L205