search

USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
411 stars 143 forks source link

adds response_time to solr #150

Closed Rohithyeravothula closed 6 years ago

Rohithyeravothula commented 6 years ago

What changes were proposed in this pull request?

This PR adds a new field to solr, "response_time" which stores the time taken by HTTP server to provide the web page (That is the time taken from the first byte of the request to the last byte of response) (Please fill in changes proposed in this fix)

Is this related to an already existing issue on sparkler?
If so, mention that issue by referencing its number here. related to #144

Will it close an existing issue?
Say 'Closes #IssueNum' here. Closes #144

How was this patch tested?

Manual tests were done, no new test cases added the existing tests are kept intact all tests passed for mvn clean test

Please review https://github.com/USCDataScience/sparkler/blob/master/.github/CONTRIBUTING.md before opening a pull request.

thammegowda commented 6 years ago

Merged. Thanks @Rohithyeravothula