search

SJTU-IPADS / reef

REEF is a GPU-accelerated DNN inference serving system that enables instant kernel preemption and biased concurrent execution in GPU scheduling.
Apache License 2.0
85 stars 7 forks source link

How do I get the source code from the TVM model? #1

Closed battleonthebridge closed 2 years ago

battleonthebridge commented 2 years ago

I've got the REEF's sample code and reef-artifacts/evaluation all working, and I'd like to use my own model. But when I ran script/tvm_generate_model.py, the json and param files output what looks correct, but the cu file is 0 bytes.

I assume that I need to customize TVM, but do you have any plans to publish the details of the customization to TVM and the procedure to obtain the cu file?

francis0407 commented 2 years ago

Hi @battleonthebridge , I just upload a patch for generating REEF models using TVM, see https://github.com/francis0407/tvm/tree/reef.

To generate a model like models in resource, you can:

  1. use a script like script/tvm_generate_model.py, which will generate 4 files, including the device code (.cu), the tvm schedule, the host call trace, and the parameters.
  2. then, use the script script/generate_final_schedule.py to merge the tvm schedule and host call trace files into the final reef schedule file.
battleonthebridge commented 2 years ago

I installed CUDA and built the TVM you uploaded and was able to generate the cu file successfully.
It looks like I can now convert my own model.

Thank you very much, @francis0407. I sincerely appreciate it.

juinshell commented 2 years ago

I installed CUDA and built the TVM you uploaded and was able to generate the cu file successfully. It looks like I can now convert my own model.

Thank you very much, @francis0407. I sincerely appreciate it.

Hello. I've now got a final_schedule.json file via script/tvm_generate_model.py and script/generate_final_schedule.py, how do I get the final .cu file next?Also, I am using nvidia-gpu.

Looking forward to your reply! 🌹