search

BCDA-APS / gemviz

Data visualization for tiled
https://bcda-aps.github.io/gemviz/
Other
4 stars 0 forks source link

BUG: speed up GUI response times #184

Closed prjemian closed 11 months ago

prjemian commented 1 year ago
prjemian commented 1 year ago

Time (measured by hand) to connect with http://otz.xray.aps.anl.gov from gemviz client on otz and show first five runs of training catalog:

branch time, s
183-bug-slow 5.3
main 5.5

Keep looking for the source of delays in the GUI response...

prjemian commented 1 year ago

Observation: Used this branch yesterday to plot & show recent 1-D scans for 8-ID XPCS. Response time was reasonable.

prjemian commented 12 months ago

During testing today, time to load a catalog was very slow (>30s), time to change page size, time to switch context (bring window to front, bring other application to front) and to change the splitter were all very slow.

prjemian commented 12 months ago

With the debugger, found this code makes many requests that show up on the tiled server: https://github.com/BCDA-APS/gemviz/blob/88fc4b114a2aedf9a18d0983ab8d53b6f96f425c/gemviz/bluesky_runs_catalog_table_model.py#L40-L47

(part of the) tiled server output

INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK
INFO:     192.168.144.123:63856 - "GET /api/v1/search/training?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=d94d2126-fd52-461c-9284-23e5f34b326b&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=ge&filter%5Bcomparison%5D%5Bcondition%5D%5Boperator%5D=le&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bkey%5D=time&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1618203600.0&filter%5Bcomparison%5D%5Bcondition%5D%5Bvalue%5D=1700147812.491687&sort=time HTTP/1.1" 200 OK

These lines are identical.

prjemian commented 12 months ago

Time to refactor: https://github.com/BCDA-APS/gemviz/blob/88fc4b114a2aedf9a18d0983ab8d53b6f96f425c/gemviz/bluesky_runs_catalog_table_model.py#L75-L81 and https://github.com/BCDA-APS/gemviz/blob/88fc4b114a2aedf9a18d0983ab8d53b6f96f425c/gemviz/bluesky_runs_catalog_table_model.py#L39-L50

prjemian commented 12 months ago

This line seems to be the root cause of the many requests to the tiled server: https://github.com/BCDA-APS/gemviz/blob/88fc4b114a2aedf9a18d0983ab8d53b6f96f425c/gemviz/tapi.py#L46

It's parent.metadata that requests.

prjemian commented 11 months ago

@rodolakis This is ready for you to test again.

prjemian commented 11 months ago

Catalog size is not a problem now. Can browse large catalogs.

prjemian commented 11 months ago

You'll see slower paging times for pageSize > 25 (25 runs per page). The first time these runs are displayed in a table, it takes time to get their metadata from the server. For page size of 250, took longer (10-30 seconds, did not time it).

Default page size is now 10. Considering limiting it to 100. Larger than that, use the filtering panel.

rodolakis commented 11 months ago

Looks good on Mac