search

treasure-data / omnibus-td-agent

td-agent (Fluentd) Packaging Scripts
https://docs.treasuredata.com/articles/td-agent-changelog
Apache License 2.0
82 stars 131 forks source link

Building omnibus-td-agent #192

Open akshatsharma4 opened 5 years ago

akshatsharma4 commented 5 years ago

Hi,

I am trying to build omnibus-td-agent. I have done the installation stage and have ruby and bundler installed on my machine. but when I run "bin/omnibus build td-agent3" command, i am getting the following error:

"open_http: Non-HTTP proxy URI:[my-proxy] (RuntimeError)"

Please tell me how to proceed.

Thanks

repeatedly commented 5 years ago

Could you show which point causes this error? And your environment has proxy?

I have never hit this problm on my build. So I assume this is environment issue.

akshatsharma4 commented 5 years ago
capture

Please find the attachment. The error is coming when I execute the command "bin/omnibus build td-agent3" and it tries to download from the https url.

I have exported http and https proxies and I think, it is because of the https proxy this error is coming up(reference : http://railspro.blogspot.com/2011/05/non-http-proxy-uri-in-open-uri_6702.html).

So I unset the https proxy as Ruby's open-uri library only support http protocol, but then again it is failing because it is trying to download jemalloc and other tars from an https url.

I am completely stuck here, please let me know how I can proceed.

akshatsharma4 commented 5 years ago

Adding up to previous comment...

I am using RHEL 7.5 image, and my ruby version is ruby 2.4.3p205( I have tried with ruby 2.2 and 2.3.1 as well). I have installed bundler as well.

As you told, you have not faced this issue during your build and this might be an environment issue, could you please tell me which environment are you building it in ? or if any specific environment is required for building omnibus-td-agent? Do we need to set any proxy for ruby or any other gem?

Please let me know about the environment and the proxy settings, if required.

Thanks

repeatedly commented 5 years ago

So I unset the https proxy as Ruby's open-uri library only support http protocol

This is too old information. Recent open-uri supports http, https and ftp.

Do we need to set any proxy for ruby or any other gem?

No need from my experience.

could you please tell me which environment are you building it in ?

I build td-agent RPM with Vagrant and VirtualBox.

https://github.com/treasure-data/omnibus-td-agent#vagrant-based-virtualized-build-lab

After setup vagrant, run command like vagrant up centos-7.2. It starts td-agent build.

akshatsharma4 commented 5 years ago

Okay, Thanks a lot! I'll try this and get back to you.

akshatsharma4 commented 5 years ago

@repeatedly I used vitualbox and installed centos in it and it worked. I just followed the steps provided in github.

But when I tried building it by changing the "install_dir", say, I change from /opt/td-agent to /home/td-agent, I does not work.

My query here is, How can I change the install_dir in the source?

I am trying to build relocatable td-agent rpm by changing the source code. For now, I am just hard-coding the source, if it works, we can put some variable and try making it relocatable.

Any suggestions or input in this regards will be highly appreciated.

Thank you.

akshatsharma4 commented 5 years ago

Adding up to previous comment:

I did a grep and changed all the /opt to /home/akshat inside "omnibus-td-agent" folder. But now when I run the command "bin/omnibus build td-agent3", it still searches for /opt/td-agent.

grep command: grep -rl '\/opt' . | xargs sed -i 's/\/opt/\/home\/akshat/g'

find the error logs below:

[NetFetcher: postgresql] I | 2018-12-21T00:36:01+05:30 | Verifying checksum [Software: td-agent] I | 2018-12-21T00:36:01+05:30 | Resolving manifest entry for td-agent [NullFetcher: td-agent] I | 2018-12-21T00:36:01+05:30 | Fetching td-agent' (nothing to fetch) [Software: td-agent-files] I | 2018-12-21T00:36:01+05:30 | Resolving manifest entry for td-agent-files [NullFetcher: td-agent-files] I | 2018-12-21T00:36:01+05:30 | Fetchingtd-agent-files' (nothing to fetch) [Software: td] I | 2018-12-21T00:36:01+05:30 | Resolving manifest entry for td [NullFetcher: td] I | 2018-12-21T00:36:01+05:30 | Fetching td' (nothing to fetch) [Software: td-agent-cleanup] I | 2018-12-21T00:36:01+05:30 | Resolving manifest entry for td-agent-cleanup [NullFetcher: td-agent-cleanup] I | 2018-12-21T00:36:01+05:30 | Fetchingtd-agent-cleanup' (nothing to fetch) [Software: version-manifest] I | 2018-12-21T00:36:01+05:30 | Resolving manifest entry for version-manifest [NullFetcher: version-manifest] I | 2018-12-21T00:36:01+05:30 | Fetching version-manifest' (nothing to fetch) /home/akshat/.rvm/rubies/ruby-2.4.5/lib/ruby/2.4.0/fileutils.rb:230:inmkdir': Permission denied @ dir_s_mkdir - /opt/td-agent (Errno::EACCES) from /home/akshat/.rvm/rubies/ruby-2.4.5/lib/ruby/2.4.0/fileutils.rb:230:in fu_mkdir' from /home/akshat/.rvm/rubies/ruby-2.4.5/lib/ruby/2.4.0/fileutils.rb:208:inblock (2 levels) in mkdir_p' from /home/akshat/.rvm/rubies/ruby-2.4.5/lib/ruby/2.4.0/fileutils.rb:206:in reverse_each' from /home/akshat/.rvm/rubies/ruby-2.4.5/lib/ruby/2.4.0/fileutils.rb:206:inblock in mkdir_p' from /home/akshat/.rvm/rubies/ruby-2.4.5/lib/ruby/2.4.0/fileutils.rb:191:in each' from /home/akshat/.rvm/rubies/ruby-2.4.5/lib/ruby/2.4.0/fileutils.rb:191:inmkdir_p'

I am trying to build the rpm which builds in a custom path. Please let me know how I can proceed.

akshatsharma4 commented 5 years ago

Hi @repeatedly ,

Could you please look into above comments and guide me how to proceed?

Thanks, Akshat Sharma

vwbusguy commented 4 years ago

@akshatsharma4 - You don't have access to /opt/td-agent and/or it doesn't exist yet from the looks of it.