search

USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
410 stars 143 forks source link

Broken run script #219

Closed buggtb closed 3 years ago

buggtb commented 3 years ago

Hey @nhandyal / @lewismc

https://github.com/USCDataScience/sparkler/commit/801f1ae05b0fabd7931f2cd2b396305889bf784b#diff-ba9a32629fea83dfb633800ea410676e10fc6d41d8cbef7401f0e437dc5fa6ddR12

That change is breaking to the run script for anyone (like myself) who moves the build out of a build directory

lewismc commented 3 years ago

Hi @buggtb we discussed this today. The team are working on a fix.

lewismc commented 3 years ago

@nhandyal @felixloesing @kyan2601 @KilometersFan who is working on fixing this?

nhandyal commented 3 years ago

Hey @buggtb thanks for reporting and your patience while we resolved this. Here's a PR that addresses this: https://github.com/USCDataScience/sparkler/pull/223 . I'm not sure how your setup is configured, so can you let me know if this resolves the issue for you. I think it should as the fix in the PR first tries to resolve the JAR path using relative paths (how things worked to begin with), then tries the absolute path (how we changed it to be).

buggtb commented 3 years ago

Sorry folks, missed this. I've got a build to do tomorrow, so I'll test it then for you.

Thanks!

On Mon, Mar 22, 2021 at 4:56 PM Nikhil Handyal @.***> wrote:

Hey @buggtb https://github.com/buggtb thanks for reporting and your patience while we resolved this. Here's a PR that addresses this: #223 https://github.com/USCDataScience/sparkler/pull/223 . I'm not sure how your setup is configured, so can you let me know if this resolves the issue for you. I think it should as the fix in the PR first tries to resolve the JAR path using relative paths (how things worked to begin with), then tries the absolute path (how we changed it to be).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/USCDataScience/sparkler/issues/219#issuecomment-804227702, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAZI6G7HW7XIC7DCPBEGZLTE5ZDDANCNFSM4Y4ZMNVQ .

--

Spicule Limited is registered in England & Wales. Company Number:

  1. Registered office: First Floor, Telecom House, 125-135 Preston Road, Brighton, England, BN1 6AF. VAT No. 251478891.

All engagements are subject to Spicule Terms and Conditions of Business. This email and its contents are intended solely for the individual to whom it is addressed and may contain information that is confidential, privileged or otherwise protected from disclosure, distributing or copying. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Spicule Limited. The company accepts no liability for any damage caused by any virus transmitted by this email. If you have received this message in error, please notify us immediately by reply email before deleting it from your system. Service of legal notice cannot be effected on Spicule Limited by email.