[GenAI] Add LLaMA support

[LittleLittleCloud]

We are excited to review your PR.

So we can do the best job, please check:

• [x] There's a descriptive title that will make sense to other developers some time from now.
• [x] There's associated issues. All PR's should have issue(s) associated - unless a trivial self-evident change such as fixing a typo. You can use the format Fixes #nnnn in your description to cause GitHub to automatically close the issue(s) when your PR is merged.
• [x] Your change description explains what the change does, why you chose your approach, and anything else that reviewers should know.
• [x] You have included any necessary tests in the same PR.

7169

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 40.34166% with 454 lines in your changes missing coverage. Please review.

Project coverage is 68.83%. Comparing base (fa8c822) to head (19a2d7e). Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/Microsoft.ML.GenAI.LLaMA/Module/LlamaModel.cs	18.75%	76 Missing and 2 partials :warning:
...crosoft.ML.GenAI.LLaMA/Module/LlamaDecoderLayer.cs	28.57%	65 Missing :warning:
src/Microsoft.ML.GenAI.LLaMA/LlamaCausalLMAgent.cs	0.00%	50 Missing :warning:
src/Microsoft.ML.GenAI.LLaMA/LlamaForCausalLM.cs	14.03%	49 Missing :warning:
...osoft.ML.GenAI.LLaMA/LlamaTextCompletionService.cs	0.00%	41 Missing :warning:
.../Microsoft.ML.GenAI.Core/Module/RotaryEmbedding.cs	42.62%	34 Missing and 1 partial :warning:
...crosoft.ML.GenAI.Core/Extension/ModuleExtension.cs	0.00%	29 Missing :warning:
src/Microsoft.ML.GenAI.Core/Module/Attention.cs	40.00%	24 Missing :warning:
...osoft.ML.GenAI.LLaMA/LlamaChatCompletionService.cs	0.00%	19 Missing :warning:
...soft.ML.GenAI.LLaMA/Llama3_1ChatTemplateBuilder.cs	68.62%	8 Missing and 8 partials :warning:
... and 11 more

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #7220 +/- ## ========================================== - Coverage 68.89% 68.83% -0.07% ========================================== Files 1441 1453 +12 Lines 270827 271482 +655 Branches 28019 28090 +71 ========================================== + Hits 186584 186870 +286 - Misses 77038 77393 +355 - Partials 7205 7219 +14 ``` | [Flag](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | Coverage Δ | | |---|---|---| | [Debug](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | `68.83% <40.34%> (-0.07%)` | :arrow_down: | | [production](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | `63.34% <35.25%> (-0.08%)` | :arrow_down: | | [test](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | `89.03% <93.93%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files with missing lines](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | Coverage Δ | | |---|---|---| | [src/Microsoft.ML.GenAI.Core/Module/GenAILinear.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Core%2FModule%2FGenAILinear.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5Db3JlL01vZHVsZS9HZW5BSUxpbmVhci5jcw==) | `100.00% <ø> (ø)` | | | [...icrosoft.ML.GenAI.Core/Module/NewGELUActivation.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Core%2FModule%2FNewGELUActivation.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5Db3JlL01vZHVsZS9OZXdHRUxVQWN0aXZhdGlvbi5jcw==) | `27.27% <ø> (ø)` | | | [.../Microsoft.ML.GenAI.Core/Module/QuantizedLinear.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Core%2FModule%2FQuantizedLinear.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5Db3JlL01vZHVsZS9RdWFudGl6ZWRMaW5lYXIuY3M=) | `92.24% <ø> (ø)` | | | [src/Microsoft.ML.GenAI.Core/Module/RMSNorm.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Core%2FModule%2FRMSNorm.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5Db3JlL01vZHVsZS9STVNOb3JtLmNz) | `40.00% <100.00%> (ø)` | | | [...c/Microsoft.ML.GenAI.LLaMA/LlamaTokenizerHelper.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.LLaMA%2FLlamaTokenizerHelper.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5MTGFNQS9MbGFtYVRva2VuaXplckhlbHBlci5jcw==) | `100.00% <100.00%> (ø)` | | | [src/Microsoft.ML.GenAI.Phi/Module/Phi2Attention.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Phi%2FModule%2FPhi2Attention.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5QaGkvTW9kdWxlL1BoaTJBdHRlbnRpb24uY3M=) | `32.09% <ø> (ø)` | | | [src/Microsoft.ML.GenAI.Phi/Module/Phi2MLP.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Phi%2FModule%2FPhi2MLP.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5QaGkvTW9kdWxlL1BoaTJNTFAuY3M=) | `54.54% <ø> (ø)` | | | [src/Microsoft.ML.GenAI.Phi/Module/Phi3MLP.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Phi%2FModule%2FPhi3MLP.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5QaGkvTW9kdWxlL1BoaTNNTFAuY3M=) | `55.55% <100.00%> (ø)` | | | [...ML.GenAI.Phi/Module/Phi3SuScaledRotaryEmbedding.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Phi%2FModule%2FPhi3SuScaledRotaryEmbedding.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5QaGkvTW9kdWxlL1BoaTNTdVNjYWxlZFJvdGFyeUVtYmVkZGluZy5jcw==) | `24.39% <ø> (ø)` | | | [...enAI.Phi/Phi3/Phi3CausalLMChatCompletionService.cs](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree&filepath=src%2FMicrosoft.ML.GenAI.Phi%2FPhi3%2FPhi3CausalLMChatCompletionService.cs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet#diff-c3JjL01pY3Jvc29mdC5NTC5HZW5BSS5QaGkvUGhpMy9QaGkzQ2F1c2FsTE1DaGF0Q29tcGxldGlvblNlcnZpY2UuY3M=) | `65.00% <100.00%> (ø)` | | | ... and [23 more](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet) | | ... and [8 files with indirect coverage changes](https://app.codecov.io/gh/dotnet/machinelearning/pull/7220/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dotnet)