search

techact / solrmeter

Automatically exported from code.google.com/p/solrmeter
0 stars 0 forks source link

Query Extractor from Log File #48

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
It should be possible to add a Solr Log File instead of a straight queries 
file.

Original issue reported on code.google.com by tflo...@gmail.com on 2 Oct 2010 at 10:26

GoogleCodeExporter commented 8 years ago 
It should accept more than one log file at a time. For example, instead of 
specifying one file path in the settings panel, specify:
pathToLog1;pathToLog2;...

Original comment by tflo...@gmail.com on 8 Nov 2010 at 6:41

GoogleCodeExporter commented 8 years ago 
Few days ago i played with Solrmeter code and I started adding support for 
external files containing queries. In the current search project i am working, 
we are extracting queries from jetty log files and we would like to execute 
them to test our system.
I have done a small change to the UI as well.
If you are interested I can send you the patch.
Edo

Original comment by edoardo.tosca on 15 Nov 2010 at 4:19

GoogleCodeExporter commented 8 years ago 
Sure, please send the patch. What kind of changes have you make to the UI?

Original comment by tflo...@gmail.com on 15 Nov 2010 at 7:54

GoogleCodeExporter commented 8 years ago 
Sure, I am still working because i want to refactor and clean some code.
From the point of view of the UI i have changed the QUerySerttingsPanel, Now 
you have another tabbed panel where you can select Random Queries (the normal 
one), or External File Query. I want to change it a bit because now it's a 
little bit confusing.
I attach two screenshots.
As you can see i started adding a dropdown panel for true/false options.

The next step will be adding a parser for Solr log files.

Let me know, I am really happy if I can help!
Edo

Original comment by edoardo.tosca on 16 Nov 2010 at 11:19

GoogleCodeExporter commented 8 years ago 
About the random Queries and external file queries: Both executors use an 
external file to generate the queries, the difference between them is the 
interval of execution. The "random" will execute queries on a random instant of 
time inside a minute, the "constant" executor will calculate the interval 
between queries and execute one query every N seconds. Why did you duplicate it 
in both panels?

About the true/false dropdown: that looks nice!

Original comment by tflo...@gmail.com on 16 Nov 2010 at 2:13

GoogleCodeExporter commented 8 years ago 
I did it because the first code i developed used a custom Executor (extending 
the random implementation), with a custom Operation class. 
Now i have changed it, deleting the custom one. So executors are "constant" and 
"random". I also changed the code to use the same Query File parameter.
I have just added an option inside the QueryOperation class.

I'm trying to sort out a solution for the UI using a single panel.
The idea could be: ask for Solr Url. Then a radio button so you can choose 
between the two query mode. If you choose what i have called "Random queries" 
it shows the default list of params. If you choose "External Query File" 
everything will be hidden except "Query File" parameter. 
In this way you have to fill only fields that you really need.
What do you think about this solution?

Regarding dropdown, I am happy to hear that. Anyway, I'm wondering to move this 
change to a UI "improvement" issue rather than to keep it in this issue.
Edo

Original comment by edoardo.tosca on 16 Nov 2010 at 2:30

GoogleCodeExporter commented 8 years ago 
I still don't fully understand the change. You said: "So executors are 
"constant" and "random"". Isn't that the same as before?
"I also changed the code to use the same Query File parameter", and why do you 
need to different inputs?
Maybe when I see the code I'll understand better your contribution.

About the dropdown, I've created an issue for this: 
http://code.google.com/p/solrmeter/issues/detail?id=16 (I know, the issue title 
sucks :S). I want to add different types of selectors for different types of 
data.

Original comment by tflo...@gmail.com on 16 Nov 2010 at 6:45

GoogleCodeExporter commented 8 years ago 
Hi I attach a first patch that let you execute a set of queries using an 
external file.
The file should contain a list of queries, one per row. There is an example 
query file called externalQueries.txt. You will find it in the example folder.
I have also changed a bit of UI adding a ComboBoxPropertyPanel.

I hope this helps. Next step will be the implementation of an extractor of 
queries from a solr log file.

Original comment by edoardo.tosca on 26 Nov 2010 at 5:51

Attachments:

GoogleCodeExporter commented 8 years ago 
Nice! this is a good idea. I'll commit this when i have some time.

Original comment by tflo...@gmail.com on 1 Dec 2010 at 2:14

GoogleCodeExporter commented 8 years ago 
Excellent, i will carry on and write some code to extract queries from Solr 
logs!

Original comment by edoardo.tosca on 1 Dec 2010 at 2:19

GoogleCodeExporter commented 8 years ago 
I've created issue http://code.google.com/p/solrmeter/issues/detail?id=55 and 
solve it. The code is already committed to trunk

Original comment by tflo...@gmail.com on 3 Jan 2011 at 1:21