microsoft / T-MAC

Low-bit LLM inference on CPU with lookup table
MIT License
577 stars 44 forks source link

Andorid tune kernel question #53

Open qw1319 opened 1 month ago

qw1319 commented 1 month ago

when i tune kernel on andorid,accoding to docs/android.md : python -m tvm.exec.rpc_tracker; But i meet this question:root@a23bc1a1da28:/home/merge_tmac/T-MAC# python -m tvm.exec.rpc_tracker 2024-09-26 13:39:49.926 INFO bind to 0.0.0.0:9190

This interface nerver change for three or more hours

qw1319 commented 1 month ago

when i run python -m tvm.exec.query_rpc_tracker i meet this question

`Tracker address 127.0.0.1:9190

Server List

server-address key


Queue Status

key total free pending

---------------------------`

kaleid-liner commented 1 month ago

You need to make sure your phone and host pc are in the same network, and if you have already checked it, the most possible cause is your firewall blocking incoming connections. A simple solution is to close your firewall.

qw1319 commented 1 month ago

You need to make sure your phone and host pc are in the same network, and if you have already checked it, the most possible cause is your firewall blocking incoming connections. A simple solution is to close your firewall.

my firewall is closed & my phone and pc in the same network. do u have any method?

qw1319 commented 1 month ago

You need to make sure your phone and host pc are in the same network, and if you have already checked it, the most possible cause is your firewall blocking incoming connections. A simple solution is to close your firewall.

my firewall is closed & my phone and pc in the same network. do u have any method?

when i run python -m tvm.exec.query_rpc_tracker i meet this question

Tracker address 127.0.0.1:9190

this is not same as docs/android.md

image

qw1319 commented 1 month ago

i run cmd python3 ../../T-MAC/3rdparty/tvm/apps/android_rpc/tests/android_rpc_test.py but get error 2024-09-27 10:14:14.996 WARNING Cannot connect to tracker ('0.0.0.0', 9190), retry in 5 secs... 2024-09-27 10:14:20.002 WARNING Cannot connect to tracker ('0.0.0.0', 9190), retry in 5 secs... RuntimeError: Failed to connect to server ('0.0.0.0', 9190)

qw1319 commented 1 month ago

after run this command adb reverse tcp:9190 tcp:9190 python3 -m tvm.exec.query_rpc_tracker --host=0.0.0.0 --port=9190 i see this `Tracker address 0.0.0.0:9190

Server List

server-address key

   127.0.0.1:5001    server:android

Queue Status

key total free pending

android 1 1 0 -------------------------------`

But when i run tune cmd i meet this for many hours: `[Task qgemm_lut_t4_int8_m12288_k4096_n1_b3] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/3) | 0.00 s'

qw1319 commented 1 month ago

the question like this https://discuss.tvm.apache.org/t/android-rpc-requesting-remote-session-from-rpc-tracker-failed/8413

qw1319 commented 1 month ago

after run this command adb reverse tcp:9190 tcp:9190 python3 -m tvm.exec.query_rpc_tracker --host=0.0.0.0 --port=9190 i see this `Tracker address 0.0.0.0:9190

Server List

server-address key

   127.0.0.1:5001    server:android

Queue Status

key total free pending

android 1 1 0 -------------------------------`

But when i run tune cmd i meet this for many hours: `[Task qgemm_lut_t4_int8_m12288_k4096_n1_b3] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/3) | 0.00 s'

and report error InternalError: Check failed: (sock.Connect(addr)) is false: Connect to 127.0.0.1:5001 failed