search

fullstorydev / solr-bench

Solr benchmarking and load testing harness
Apache License 2.0
17 stars 10 forks source link

Initial import of the benchmarking suite #1

Closed chatman closed 4 years ago

chatman commented 4 years ago

Just running through the prerequisites (in the README.md) and then ./start.sh config.json should suffice.

@donjaime , please add Steve Mitchell as a reviewer as well.

donjaime commented 4 years ago

Adding @sgmitchell and @bschaeffer. For some reason the PR won't allow multiple assignees.

donjaime commented 4 years ago

@ZackFS to review licensing prior to us open source this.

chatman commented 4 years ago

@bschaeffer Can you please review? Steve is on vacation, and need your help here, please. :-)

jbdutton commented 4 years ago

I ran this again today and the logs (both in GCS and on the local filesystem) seem to be empty. The script creates a tar file for each Solr node but there's nothing inside.

chatman commented 4 years ago

I ran this again today and the logs (both in GCS and on the local filesystem) seem to be empty. The script creates a tar file for each Solr node but there's nothing inside.

Okay, I'll take a look today. Local is currently not collecting logs. GCP should.

jbdutton commented 4 years ago

@chatman the real issue was earlier in the logs:

Exception in thread "main" java.lang.RuntimeException: File not found: solr-custom.tgz, was expecting the gs://solr-wikipedia-benchmarks/solr-custom.tgz would've been downloaded.
    at org.apache.solr.benchmarks.BenchmarksMain.getSolrPackagePath(BenchmarksMain.java:73)
    at org.apache.solr.benchmarks.BenchmarksMain.main(BenchmarksMain.java:96)
Pulling logs
tar: solr-custom.tgz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Removing leading `/' from member names
tar: /server/logs: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors

Let me fix this and try again...

chatman commented 4 years ago

File not found: solr-custom.tgz, was expecting the gs://solr-wikipedia-benchmarks/solr-custom.tgz would've been downloaded.

Ah, that is a sample config.json that points to my personal GCP bucket. I'll give the bucket some permissions to make sure anyone can make it work.

chatman commented 4 years ago

Let me fix this and try again...

Sure, please go ahead! Thanks.

jbdutton commented 4 years ago

Ah, that is a sample config.json that points to my personal GCP bucket. I'll give the bucket some permissions to make sure anyone can make it work.

No need, I was able to get things working out of our own fs-playpen bucket. I'll probably tweak the README in the near future to reflect some of the config changes I needed to make.

chatman commented 4 years ago

Thanks @jbdutton, @sgmitchell, @theMPatel for your timely reviews and valuable feedback. Hope to work together to improve this project and make it more useful.

chatman commented 4 years ago

I ran this again today and the logs (both in GCS and on the local filesystem) seem to be empty. The script creates a tar file for each Solr node but there's nothing inside.

Just tested it on a fresh instance, and it seems to work. I see the logs and results. https://gist.github.com/chatman/c25a5ca373b56a9dc8552de3fe8a0617

jbdutton commented 4 years ago

I ran this again today and the logs (both in GCS and on the local filesystem) seem to be empty. The script creates a tar file for each Solr node but there's nothing inside.

Just tested it on a fresh instance, and it seems to work. I see the logs and results. https://gist.github.com/chatman/c25a5ca373b56a9dc8552de3fe8a0617

Same, there were issues with my config file that needed adjustment.

thempatel commented 4 years ago

🎉 🎉 🎉