Why am I inferring this model through rtx4090? Its inference speed is very slow. The model used is DAT. Inference speed takes about five seconds to infer a picture. If you check the gpu, you can confirm that it is the gpu that is being called.
Did you try to check the time by 100 times in average? The initial loading to the weight is time-consuming. For the second run, it should be efficient. Also, the input resolution is an issue.
Why am I inferring this model through rtx4090? Its inference speed is very slow. The model used is DAT. Inference speed takes about five seconds to infer a picture. If you check the gpu, you can confirm that it is the gpu that is being called.