Open pranay29 opened 6 years ago
Hi @pranay29 did you get around this ?
Hi @chitralverma ,
I still have not got around this. I don't know if its fixed yet.
It's the classic guava version conflict. You can probably workaround it by pinging guava and/or other dependency versions.
seems to be coming from spark 2.2.1 d3pendency, can you try it with 2.2.0
On Tue, Apr 17, 2018, 12:25 Neville Li notifications@github.com wrote:
It's the classic guava version conflict. You can probably workaround it by pinging guava and/or other dependency versions.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/spotify/spark-bigquery/issues/62#issuecomment-382112302, or mute the thread https://github.com/notifications/unsubscribe-auth/AKnoOM_LW_Nxc7I_JEax3CGPgpJWbLxmks5tpkG9gaJpZM4ScjEF .
I am not sure if I change the existing Spark version since its in production environment.
My spark verion is 2.2.1 run in to tis error kindly help
spark-shell --packages com.github.samelamin:spark-bigquery_2.11:0.2.4
import com.samelamin.spark.bigquery._
df.saveAsBigQueryTable("myproj:dataset.target1") java.lang.NoSuchMethodError: com.google.cloud.hadoop.io.bigquery.BigQueryStrings.parseTableReference(Ljava/lang/String;)Lcom/google/api/services/bigquery/model/TableReference; at com.samelamin.spark.bigquery.BigQueryDataFrame.saveAsBigQueryTable(BigQueryDataFrame.scala:40) ... 50 elided
has anyone found a work around for this?
I have found it. Year ago
On Tue, Sep 17, 2019, 10:06 PM mikerlt notifications@github.com wrote:
has anyone found a work around for this?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AIMRWXAJQMJ2GFGBLVWPJSTQKEBSBA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65EFLY#issuecomment-532300463, or mute the thread https://github.com/notifications/unsubscribe-auth/AIMRWXF2RFLJ72MYWEKI3LLQKEBSBANCNFSM4ETSGECQ .
Ok, and what did you do to get it to work?
I have found it. Year ago … On Tue, Sep 17, 2019, 10:06 PM mikerlt @.***> wrote: has anyone found a work around for this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#62?email_source=notifications&email_token=AIMRWXAJQMJ2GFGBLVWPJSTQKEBSBA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65EFLY#issuecomment-532300463>, or mute the thread https://github.com/notifications/unsubscribe-auth/AIMRWXF2RFLJ72MYWEKI3LLQKEBSBANCNFSM4ETSGECQ .
You have to add dependent jar file to the spark library
On Tue, Sep 17, 2019, 10:17 PM mikerlt notifications@github.com wrote:
Ok, and what did you do to get it to work?
I have found it. Year ago … <#m-9049897132146471791> On Tue, Sep 17, 2019, 10:06 PM mikerlt @.***> wrote: has anyone found a work around for this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#62 https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AIMRWXAJQMJ2GFGBLVWPJSTQKEBSBA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65EFLY#issuecomment-532300463>, or mute the thread https://github.com/notifications/unsubscribe-auth/AIMRWXF2RFLJ72MYWEKI3LLQKEBSBANCNFSM4ETSGECQ .
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AIMRWXAMODZNKBJ7RRMMKDLQKEC3RA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65FFJY#issuecomment-532304551, or mute the thread https://github.com/notifications/unsubscribe-auth/AIMRWXH4EWE5MP2OL5X2NODQKEC3RANCNFSM4ETSGECQ .
Do you know which jar needs to be added?
On Wed, Sep 18, 2019 at 4:13 AM sunilpashikanti notifications@github.com wrote:
You have to add dependent jar file to the spark library
On Tue, Sep 17, 2019, 10:17 PM mikerlt notifications@github.com wrote:
Ok, and what did you do to get it to work?
I have found it. Year ago … <#m-9049897132146471791> On Tue, Sep 17, 2019, 10:06 PM mikerlt @.***> wrote: has anyone found a work around for this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#62 https://github.com/spotify/spark-bigquery/issues/62 ?email_source=notifications&email_token=AIMRWXAJQMJ2GFGBLVWPJSTQKEBSBA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65EFLY#issuecomment-532300463, or mute the thread
https://github.com/notifications/unsubscribe-auth/AIMRWXF2RFLJ72MYWEKI3LLQKEBSBANCNFSM4ETSGECQ .
— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AIMRWXAMODZNKBJ7RRMMKDLQKEC3RA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65FFJY#issuecomment-532304551 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AIMRWXH4EWE5MP2OL5X2NODQKEC3RANCNFSM4ETSGECQ
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AINNSFWVHDJK35X6EQZJYQ3QKHPMHA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67G54I#issuecomment-532573937, or mute the thread https://github.com/notifications/unsubscribe-auth/AINNSFWVJ6JMQOL7BR6VAPDQKHPMHANCNFSM4ETSGECQ .
-- Mike Jones CIO mike.jones@rootleveltech.com | 281.825.3801 Root Level Technology | http://www.rootleveltech.com 20008 Champion Forest Dr. STE 103 Spring, TX 77379
https://app.cloudphysics.com/RootLevel+GCP/getStarted
The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.
I have done it months ago. Let me check and let you know.
On Wed, Sep 18, 2019, 5:25 PM mikerlt notifications@github.com wrote:
Do you know which jar needs to be added?
On Wed, Sep 18, 2019 at 4:13 AM sunilpashikanti notifications@github.com wrote:
You have to add dependent jar file to the spark library
On Tue, Sep 17, 2019, 10:17 PM mikerlt notifications@github.com wrote:
Ok, and what did you do to get it to work?
I have found it. Year ago … <#m-9049897132146471791> On Tue, Sep 17, 2019, 10:06 PM mikerlt @.***> wrote: has anyone found a work around for this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#62 <https://github.com/spotify/spark-bigquery/issues/62
?email_source=notifications&email_token=AIMRWXAJQMJ2GFGBLVWPJSTQKEBSBA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65EFLY#issuecomment-532300463>,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIMRWXF2RFLJ72MYWEKI3LLQKEBSBANCNFSM4ETSGECQ
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub <
, or mute the thread <
https://github.com/notifications/unsubscribe-auth/AIMRWXH4EWE5MP2OL5X2NODQKEC3RANCNFSM4ETSGECQ
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AINNSFWVHDJK35X6EQZJYQ3QKHPMHA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67G54I#issuecomment-532573937 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AINNSFWVJ6JMQOL7BR6VAPDQKHPMHANCNFSM4ETSGECQ
.
-- Mike Jones CIO mike.jones@rootleveltech.com | 281.825.3801 Root Level Technology | http://www.rootleveltech.com 20008 Champion Forest Dr. STE 103 Spring, TX 77379
https://app.cloudphysics.com/RootLevel+GCP/getStarted
The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AIMRWXFAD2RH75FQZTJVRILQKIJK3A5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67ZW6A#issuecomment-532650872, or mute the thread https://github.com/notifications/unsubscribe-auth/AIMRWXFTEQO4E62MFGMLB33QKIJK3ANCNFSM4ETSGECQ .
Bigquery-connector-latest-hadoop2.jar
On Wed, Sep 18, 2019, 11:01 PM Pashikanti Sunil < sunilpashikanti2000@gmail.com> wrote:
I have done it months ago. Let me check and let you know.
On Wed, Sep 18, 2019, 5:25 PM mikerlt notifications@github.com wrote:
Do you know which jar needs to be added?
On Wed, Sep 18, 2019 at 4:13 AM sunilpashikanti <notifications@github.com
wrote:
You have to add dependent jar file to the spark library
On Tue, Sep 17, 2019, 10:17 PM mikerlt notifications@github.com wrote:
Ok, and what did you do to get it to work?
I have found it. Year ago … <#m-9049897132146471791> On Tue, Sep 17, 2019, 10:06 PM mikerlt @.***> wrote: has anyone found a work around for this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#62 <https://github.com/spotify/spark-bigquery/issues/62
?email_source=notifications&email_token=AIMRWXAJQMJ2GFGBLVWPJSTQKEBSBA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65EFLY#issuecomment-532300463>,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIMRWXF2RFLJ72MYWEKI3LLQKEBSBANCNFSM4ETSGECQ
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub <
, or mute the thread <
https://github.com/notifications/unsubscribe-auth/AIMRWXH4EWE5MP2OL5X2NODQKEC3RANCNFSM4ETSGECQ
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AINNSFWVHDJK35X6EQZJYQ3QKHPMHA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67G54I#issuecomment-532573937 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AINNSFWVJ6JMQOL7BR6VAPDQKHPMHANCNFSM4ETSGECQ
.
-- Mike Jones CIO mike.jones@rootleveltech.com | 281.825.3801 Root Level Technology | http://www.rootleveltech.com 20008 Champion Forest Dr. STE 103 Spring, TX 77379
https://app.cloudphysics.com/RootLevel+GCP/getStarted
The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AIMRWXFAD2RH75FQZTJVRILQKIJK3A5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67ZW6A#issuecomment-532650872, or mute the thread https://github.com/notifications/unsubscribe-auth/AIMRWXFTEQO4E62MFGMLB33QKIJK3ANCNFSM4ETSGECQ .
Thank you!!
On Sep 18, 2019, at 1:36 PM, sunilpashikanti notifications@github.com wrote:
Bigquery-connector-latest-hadoop2.jar
On Wed, Sep 18, 2019, 11:01 PM Pashikanti Sunil < sunilpashikanti2000@gmail.com> wrote:
I have done it months ago. Let me check and let you know.
On Wed, Sep 18, 2019, 5:25 PM mikerlt notifications@github.com wrote:
Do you know which jar needs to be added?
On Wed, Sep 18, 2019 at 4:13 AM sunilpashikanti <notifications@github.com
wrote:
You have to add dependent jar file to the spark library
On Tue, Sep 17, 2019, 10:17 PM mikerlt notifications@github.com wrote:
Ok, and what did you do to get it to work?
I have found it. Year ago … <#m-9049897132146471791> On Tue, Sep 17, 2019, 10:06 PM mikerlt @.***> wrote: has anyone found a work around for this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#62 <https://github.com/spotify/spark-bigquery/issues/62
?email_source=notifications&email_token=AIMRWXAJQMJ2GFGBLVWPJSTQKEBSBA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65EFLY#issuecomment-532300463>,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIMRWXF2RFLJ72MYWEKI3LLQKEBSBANCNFSM4ETSGECQ
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub <
, or mute the thread <
https://github.com/notifications/unsubscribe-auth/AIMRWXH4EWE5MP2OL5X2NODQKEC3RANCNFSM4ETSGECQ
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AINNSFWVHDJK35X6EQZJYQ3QKHPMHA5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67G54I#issuecomment-532573937 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AINNSFWVJ6JMQOL7BR6VAPDQKHPMHANCNFSM4ETSGECQ
.
-- Mike Jones CIO mike.jones@rootleveltech.com | 281.825.3801 Root Level Technology | http://www.rootleveltech.com 20008 Champion Forest Dr. STE 103 Spring, TX 77379
https://app.cloudphysics.com/RootLevel+GCP/getStarted
The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/spotify/spark-bigquery/issues/62?email_source=notifications&email_token=AIMRWXFAD2RH75FQZTJVRILQKIJK3A5CNFSM4ETSGEC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67ZW6A#issuecomment-532650872, or mute the thread https://github.com/notifications/unsubscribe-auth/AIMRWXFTEQO4E62MFGMLB33QKIJK3ANCNFSM4ETSGECQ .
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
I am running my spark-shell with Scala on version 2.2.1
I am trying to load some data in BigQuery which returns an error as shown below--
Kindly help !!