use hello-world as a performance tracker

[hanxiao]

Describe the feature

Tracking jina overall perf (index & search) across all releases. Use the time elapsed of hello-world as a metric.

Your proposal

• on every devel or new release, use docker run jinaai/jina:devel hello-world write down the elapsed time;
• keep all time in CSV/JSON and maintain them a separate repo
• plot it, x-axis version, y-axis time

---

Environment

Screenshots

[hanxiao]

Single shot:

docker run -v /tmp:/tmp jinaai/jina:devel hello-world > /tmp/hw.log && perl -lne 'print "$1\t$2" if(/done in ⏱ (.*)s 🐎 (.*)\/s/)' /tmp/hw.log

K-fold:

for j in `seq 3`; do docker run --rm -v /tmp:/tmp jinaai/jina:0.3.5 hello-world > /tmp/hw.log && perl -lne 'print "$1\t$2" if(/done in ⏱ (.*)s 🐎 (.*)\/s/)' /tmp/hw.log; done

gives three lines:

• download time (s) and speed (don't care)
• indexing time (s) and speed (docs/s)
• searching time (s) and speed (docs/s)

[hanxiao]

locally, 0.3.5		time(s)	speed (docs/s)
download	4.3	866.2
index	31.4	1924.2
query	5.6	28.4

0.3.6 (afte #618 where Zmqlet replaced by ZmqStreamlet)		time(s)	speed (docs/s)
download	4.3	881.2
index	28.0	2160.1
query	3.9	40.9

so #618 worth it! 🥳

[jina-bot]

This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 4 days

[jina-bot]

This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 4 days

[jina-bot]

This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 4 days