search

medjed / embulk-input-bigquery

BigQuery input plugin for Embulk loads records from BigQuery
MIT License
15 stars 25 forks source link

Unable to install plugin on embulk 0.9.23 #37

Open docwho76 opened 2 years ago

docwho76 commented 2 years ago

If I tell embulk to install this plugin today the install hangs and never completes.

vagrant@upwork-embulk-ubuntu-1604-chef-14:~$ /home/embulk/.embulk/bin/embulk gem install embulk-input-bigquery --verbose --backtrace
2021-11-03 00:55:36.484 +0000: Embulk v0.9.23

Gem plugin path is: /home/vagrant/.embulk/lib/gems

HEAD https://api.rubygems.org/api/v1/dependencies
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=embulk-input-bigquery
200 OK
Getting SRV record failed: DNS result has no information for _rubygems._tcp.api.rubygems.org
GET https://api.rubygems.org/api/v1/dependencies?gems=google-api-client,google-cloud-bigquery,google-cloud-core,google-cloud-env,signet
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=concurrent-ruby,googleauth,mime-types
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=google-apis-core,google-apis-generator
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=addressable,httpclient,mini_mime,representable,retriable
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=declarative,trailblazer-option,uber
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=public_suffix
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=faraday,jwt,multi_json
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=faraday-em_http,faraday-em_synchrony,faraday-excon,faraday-httpclient,faraday-net_http,faraday-net_http_persistent,faraday-patron,faraday-rack,multipart-post,ruby2_keywords
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=mime-types-data
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=memoist,os
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=google-cloud-errors
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=activesupport,gems,google-apis-discovery_v1,thor
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=i18n,minitest,tzinfo,zeitwerk
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=rexml,webrick
200 OK

curl the last URL it seems hung at and I get

[{  :   nameI"
rexml:ET:numberI"
3.1.8;T:
platformI"  ruby;T:dependencies[{   ;I"
rexml;T;I"
3.2.3;T;I"  ruby;T; [{  ;I"
rexml;T;I"3.1.7.3;T;I" ruby;T; [{  ;I"
rexml;T;I"
3.2.5;T;I"  ruby;T; [{  ;I"
rexml;T;I"
3.1.9;T;I"  ruby;T; [{  ;I"
rexml;T;I"
3.2.1;T;I"  ruby;T; [{  ;I"
rexml;T;I"
3.2.0;T;I"  ruby;T; [{  ;I"
rexml;T;I"
3.2.2;T;I"  ruby;T; [{  ;I"
rexml;T;I"
3.2.4;T;I"  ruby;T; [{  ;I"
rexml;T;I"3.1.9.1;T;I" ruby;T; [{  ;I"webrick;T;I"
1.6.0;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.7.0;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.3.1;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.5.0;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.4.0;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.4.3;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.5.1;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.4.1;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.4.2;T;I"  ruby;T; [{  ;I"webrick;T;I"
1.4.4;T;I"  ruby;T; [{  ;I"webrick;T;I"1.4.0.beta1;T;I"    ruby;T; [{  ;I"webrick;T;I"
1.6.1;T;I"  ruby;T; [

it looks like it just hangs ?

docwho76 commented 2 years ago

Additonally, if you do

DEBUG_RESOLVER=true gem install embulk-input-bigquery --verbose

you can see the resolver just keeps going in a loop

gomesuit commented 2 years ago

@docwho76

I have same issue. Try this command. It worked for me.

embulk gem install google-cloud-env -v 1.2.1
embulk gem install google-cloud-core -v 1.3.2
embulk gem install googleauth -v 0.9.0
embulk gem install signet -v 0.11.0
embulk gem install representable -v 3.0.4
embulk gem install google-api-client -v 0.32.1
embulk gem install embulk-input-bigquery -v 0.0.9
MinSeok-Chae commented 2 years ago

@docwho76 @gomesuit I have same issue. I tried to install it with embulk-output-bigquery for processing installing and It looked like installed embulk-input-bigquery properly. But it didn't install at all because I met not found plug-in error when I ran embulk.

embulk gem install embulk-input-postgresql -v 0.12.2 
embulk gem install embulk-output-postgresql -v 0.10.1 
embulk gem install embulk-filter-add_time -v 0.2.0 
embulk gem install google-cloud-core -v 1.3.0 
embulk gem install embulk-output-bigquery -v 0.6.7 
embulk gem install embulk-input-bigquery  -v 0.0.9
MinSeok-Chae commented 2 years ago

@docwho76 @gomesuit Hey, guys. I finally found the solution below link and also I already checked it work. FYI.

https://kkkw.hatenablog.jp/entry/2020/07/30/embulk-input-bigquery%E3%81%AE%E3%82%A4%E3%83%B3%E3%82%B9%E3%83%88%E3%83%BC%E3%83%AB%E3%81%A7%E3%82%A8%E3%83%A9%E3%83%BC

JoeUnsung commented 2 years ago

It's bit old issue, but I success to install this plug-in by downgrading embulk into 0.8.39 which is released around 3 years ago. (last commit of this repo was similar) Think there've been lot changes on embulk versions, so it seems like dependency issues still exist. also used JRuby 9.1.15.0 It's officially suggested by embulk project team so far.