[tracking] Model and Op Support

[stellaraccident]

Tracking model burndown for Gen-AI models and variants that we seek to be serving via Turbine.

• [ ] Dynamic shaped llama2

  • [x] #153
  • [ ] Int4 Quantization with parameters from llama.cpp
  • [ ] Meeting latency requirements

• [ ] SHARK Model Porting

  • [ ] #152 (@aviator19941 )
  • [x] Unet
  • [x] VAE
  • [x] CLIP
  • [ ] Controlnet support (@gpetters94 )
  • [ ] LoRA support
  • [ ] https://github.com/nod-ai/SHARK/issues/1931 (@monorimet )
  • [ ] SHARK TANK -- in progress: https://github.com/nod-ai/SHARK/pull/1904 (@monorimet )

• [ ] Priority op requests

  • [ ] #210

  • [x] #110 @AmosLewis

  • [ ] OPS to linalg in llama_test when https://github.com/nod-ai/SHARK-Turbine/pull/212:

  • [ ] torch.aten.empty_strided

  • [ ] torch.aten.mean.dim

  • [ ] torch.aten.expand

  • [ ] torch.aten._softmax

  • [ ] torch.aten.silu

• [ ] General torch-mlir op support

  • [ ] torch.aten.kthvalue
  • [ ] torch.aten._fft_c2c
  • [ ] torch.aten.fmod.Tensor
  • [ ] torch.aten.affine_grid_generator
  • [ ] torch.aten._fft_c2r
  • [ ] torch.aten.histc
  • [ ] torch.aten.poisson
  • [ ] torch.aten.logit
  • [ ] torch.aten._linalg_solve_ex
  • [ ] torch.aten._pdist_forward
  • [ ] torch.aten.linalg_cholesky_ex
  • [ ] torch.aten.triu_indices
  • [ ] torch.aten.lgamma
  • [x] #323
  • [ ] torch.aten.all.dim
  • [ ] torch.aten.resize_as
  • [ ] torch.aten._cdist_forward
  • [ ] torch.aten._sample_dirichlet
  • [ ] torch.aten.remainder.Tensor
  • [ ] torch.aten._sparse_addmm
  • [ ] torch.aten.cosh #320
  • [ ] torch.aten.renorm
  • [ ] torch.aten.diagonal
  • [ ] torch.aten.tril_indices
  • [ ] torch.aten.upsample_trilinear3d
  • [ ] torch.aten.linalg_cross
  • [ ] torch.aten.pixel_unshuffle
  • [ ] torch.aten.rrelu_with_noise
  • [ ] #160
  • [ ] torch.aten._sparse_coo_tensor_with_dims_and_tensors
  • [ ] torch.aten.lerp.Scalar
  • [ ] torch.aten.trace
  • [ ] torch.aten.normal.float_float
  • [ ] torch.aten.hann_window.periodic
  • [ ] torch.aten._adaptive_avg_pool3d
  • [ ] torch.aten.resize
  • [ ] torch.aten.normal.Tensor_Tensor
  • [ ] torch.aten.unfold
  • [x] #231
  • [ ] #345
  • [ ] torch.aten.norm.Scalar
  • [ ] torch.aten._fft_r2c
  • [ ] torch.aten._linalg_slogdet
  • [ ] torch.aten.prod.dim_int
  • [ ] torch.aten._trilinear
  • [ ] torch.aten.as_strided
  • [x] #293
  • [ ] torch.aten.exponential
  • [ ] torch.aten.randperm
  • [ ] torch.aten.min.dim
  • [ ] torch.aten.argmin
  • [ ] torch.aten.replication_pad3d
  • [ ] torch.aten.acos
  • [x] #288
  • [ ] torch.aten.max_unpool2d
  • [ ] #163
  • [x] #161
  • [ ] #286
  • [x] #159
  • [ ] torch.aten.normal_functional
  • [ ] torch.aten.max_pool3d_with_indices
  • [x] #148
  • [ ] torch.aten._weight_norm_interface
  • [x] #337

• [ ] ONNX op support https://github.com/nod-ai/SHARK-Turbine/issues/215

[newling]

@stellaraccident in meeting chat:

If you are actively working on one, please click the "target" hover to the right of any task to create an issue. Then assign yourself. Note the issue in PRs. Discuss prioritization on the tracking issue and details of the op on the op issue.

[newling]

The list of ops in the issue description is the same as the one @saienduri gathered (in increasing order of appearance, i.e. later ops more important) from parity-bench (I think using https://github.com/nod-ai/SHARK-Turbine/blob/main/tests/generated/running_tests.md) and shared with me and Avinash yesterday.

Someone on the call today also mentioned pixel_shuffle as an important op to support.

[newling]

@stellaraccident I'm not sure I've done what you had in mind - I created a new issue by clicking "New issue" (top right of this page, for me at least). I don't seem to have permission to assign myself to it though

[stellaraccident]

You should now have an invite to the organization and I think I added you to a team such that you have write access to the repo.

[lisaliu1]

@stellaraccident I plan to implemente torch.aten.replication_pad2d op, and created the following issue for that. But I cannot link the op to the created issue. Maybe I missed something. https://github.com/nod-ai/SHARK-Turbine/issues/286

[schnkmwt]

Opened a issue tracker for the op torch.aten.diag_embed. @stellaraccident Can you help link it from the list above? https://github.com/nod-ai/SHARK-Turbine/issues/288

[kumardeepakamd]

@stellaraccident , I had taken up torch.aten.acos (https://github.com/nod-ai/SHARK-Turbine/issues/293). But, @schnkmwt pointed out that https://github.com/frederik-h has taken that up as https://github.com/llvm/torch-mlir/issues/2604 . So, maybe link the op to that issue. I can take up a different one: reflection_pad1d. Please link https://github.com/nod-ai/SHARK-Turbine/issues/293 to that.

[schnkmwt]

@kumardeepakamd I think this op is already being implemented: https://github.com/llvm/torch-mlir/issues/2604

[AmosLewis]

For ALL new contributors, let's use this to track your newly implemented ops. [tracking] TorchToLinalg and ONNX Op Support #215

[AmosLewis]

Close this issue temporarily so we focus on #215 .