FlagOpen / FlagGems

FlagGems is an operator library for large language models implemented in Triton Language.
Apache License 2.0
296 stars 27 forks source link

[bugfix] convert inp of index_select to enable cpu mode #206

Closed StrongSpoon closed 1 month ago

StrongSpoon commented 1 month ago

PR Category

[OP Test]

Type of Change

[Buf Fix]

Description

convert input tensor and index to specific device for reference. otherwise the ref_out will not be on CPU when option --device is cpu.

Issue

Progress

Performance