irods-contrib / metalnx-web

Metalnx Web Application
https://metalnx.github.io/
BSD 3-Clause "New" or "Revised" License
36 stars 36 forks source link

GenQuerySearchUtil search takes a long time #340

Closed KKTRKKT closed 1 year ago

KKTRKKT commented 1 year ago

i'm trying to search with hundred thousand dataobjects and collections in a Zone, and result takes 1-2 minutes.

https://github.com/irods-contrib/metalnx-web/blob/1156a3a73f190a18ee6bb0d6aff95893939745c9/src/metalnx-web/src/main/java/com/emc/metalnx/controller/utils/GenQuerySearchUtil.java#L207

StopWatch sw = new StopWatch(); sw.start(); IRODSQueryResultSet resultSet = executor.executeIRODSQueryAndCloseResult(gql, _input.offset); sw.stop(); System.out.printf("take secends %s\n", sw.getTotalTimeSeconds());

take secends 83.175796

and this is my conditions

but, it is fast when results is small.

korydraughn commented 1 year ago

I'm a little confused by your inputs. You appear to be doing a wildcard search with no wildcards. Is that correct?

Are you looking for a specific data object / collection?

KKTRKKT commented 1 year ago

@korydraughn yes i search without wildcards. is it wrong? i currently search without metadata when no metadata and search speed is very fast. I'm not sure, but is there a problem with this method?

korydraughn commented 1 year ago

Well, I'm a bit confused by the LIKE operator being used without any wildcards (e.g. % or _).

Can you post pictures of the Metalnx search page with your search criteria. Seeing what options you chose in the web app would help a lot.

KKTRKKT commented 1 year ago

@korydraughn image

actually, i didn't think use wildcards... just show everything dataobject in collection. i was being idiot

when i saw there is no way to search without metadata in search criteria so i changed metalnx-web/src/metalnx-web/src/main/java/com/emc/metalnx/controller/utils/GenQuerySearchUtil.java this row to use custom query

anyway, I should have forced use wildcards

korydraughn commented 1 year ago

The interface supports querying metadata and a few other things.

when i saw there is no way to search without metadata in search criteria

What do you mean by this? Can you explain what you're trying to find? I may be able to assist.

Can you find what you're looking for using the icommands/iquest?

KKTRKKT commented 1 year ago

@korydraughn

when i saw there is no way to search without metadata in search criteria

this mean, when i saw IRODSGenQueryExecutor always execute containing r_meta_main db. it's not sure..

Can you find what you're looking for using the icommands/iquest?

umm... i don't know well jargon and irods i can't understand sorry.

It's a happening because I didn't use wildcards. thank you for asking

korydraughn commented 1 year ago

No problem. Good luck :-)