Open sangkyuleeKOR opened 2 years ago
Can you try time.perf_counter()
instead of time.time()
?
Moving to CPU is going to potentially require synchronization as most GPU operations are asynchronous. So it depends on whether the GPU result is actually there and the time will often be due to other operations that aren’t yet completed.
@braindotai Thanks for replying!! I tried time.perf_counter() instead of time.time(). then I found GPU to CPU time takes 0.01sec. your second opinion seems to correct. Is there any way to solve this synchronization problem??? should I wait until another gpu operation finished?
I am also having same issue, postprocess takes a lot of time, the only thing it does is just uploading :(
Please have a look at FAQ's and Troubleshooting guide, your query may be already addressed.
Your issue may already be reported! Please search on the issue tracker before creating one.
Context
Your Environment
Expected Behavior
I'm trying to serve pytorch model using torchserve. I created custom model handler and hear is the code below.
Current Behavior
But, When I send a request to torchserve, Postprocess has huge amount of time. so i tried to figure out which part is problem, I tried "torch.cuda.synchronize()" it took a lot of time to do, so I delete it. after that, torch.tensor to "cpu" time increase.
![sync](https://user-images.githubusercontent.com/44499503/147329490-94bd06ee-75c7-4d10-8091-e23febb746c4.PNG)
I don't know why postprocess took a lot of time. In my opinion something has wrong in my setting. Please let me now If i'm wrong.
Possible Solution
Steps to Reproduce
1. 2. ...
Failure Logs [if any]