Open FooBarWidget opened 9 years ago
This gist seems interesting to list all your gems with native extensions :
require 'rubygems'
specs_with_extensions = Gem::Specification.each.select { |spec| spec.extensions.any? }
specs_with_extensions.each do |spec|
puts "#{spec.name} (extensions: #{spec.extensions.inspect})"
end
(source https://gist.github.com/aelesbao/1414b169a79162b1d795)
P.S: I don't know if this is wanted, but your Google Doc is restricted.
The doc is supposed to be public. I've fixed that and updated the link.
I know I can lookup all gems with native extensions, but I cannot package all gems because it's too much work. The best thing I can do is package the most popular ones, hence the poll.
@FooBarWidget Can't packaging be automated for any gem? Do you need to choose?
No it cannot be automated. See https://github.com/phusion/traveling-ruby/issues/3#issuecomment-66348237 where I said:
I think it is impossible to generalize and to automate this. The current Traveling Ruby binaries are hand-tweaked to ensure that they are portable and have no dependencies on non-system libraries. I think it is impossible to guarantee this for gems: to make the resulting binary portable would probably require a lot of manual work.
Packaging nokogiri and sqlite already took me a lot of manual work. I had to install libxml, libxslt and libsqlite3 from source, compile them with very specific settings, edit Makefiles everywhere to ensure correct linking flags, and do this twice (once for Linux, once for OS X).
So yes, I have to choose.
The set above includes my requirements for BOSH CLI https://github.com/cloudfoundry/bosh/blob/master/Gemfile.lock thanks!
@drnic Thanks, but can you help me by filtering out the gem names that contain native extensions, and modifying the spreadsheet? :)
I'd vote for porting https://github.com/ffi/ffi
@sheerun Ruby 2.1 has Fiddle: http://ruby-doc.org/stdlib-2.1.0/libdoc/fiddle/rdoc/Fiddle.html Doesn't that make ffi obsolete?
There are many many existing gems that use ffi/ffi
I was linking for completeness. I think nokogiri, mysql2, sqlite, pg are the native gems
On Wed, Dec 10, 2014 at 8:48 AM, Hongli Lai notifications@github.com wrote:
@drnic Thanks, but can you help me by filtering out the gem names that contain native extensions, and modifying the spreadsheet? :)
Reply to this email directly or view it on GitHub: https://github.com/phusion/traveling-ruby/issues/6#issuecomment-66482464
Also, gems that want to be compatible with Ruby versions before 2.1 still have to use FFI for awhile. I also have no idea where JRuby is on implementing Fiddle, which may matter for compatibility as well.
ffi, mysql2, pg and bcrypt have now been packaged.
Nokogiri should definitely be on the list.
Nokogiri is already packaged. See spreadsheet.
And sqlite3
please?
Already packaged. See spreadsheet.
Great work guys. Traveling Ruby saves us working on VirtKick a lot of work. :-) Thanks for this!
@Nowaker Glad you like it. :) I'm compiling a list of Traveling Ruby users. Could you write a short ~50 words introduction about Virtkick, and could you write another ~50 words paragraph in which you describe how Traveling Ruby has helped you?
@FooBarWidget Sure, no problem!
VirtKick in 50 words: VirtKick is a self-hosted cloud panel, similar to DigitalOcean. With privacy in mind we simplify creating, managing, hosting and providing virtual servers. Forget about installing packages or editing configuration files - all is set up automatically and ready to use. 100% open source.
VirtKick + Traveling Ruby: @Rush will provide a nice text, he did the hacking. :)
@FooBarWidget https://github.com/cloudfoundry-community/traveling-bosh is using it too
BOSH is a Cloud Foundry project for release engineering, deployment, and lifecycle management of large-scale cloud software. BOSH can provision and deploy software over hundreds of VMs and performs monitoring, failure recovery and software updates with zero-to-minimal downtime.
@FooBarWidget: Here is how traveling-ruby helps VirtKick:
VirtKick's goal is a one click installation via standalone installer and/or a distro package. We use rails and other dependencies, many of which we have at bleeding-edge versions from git. traveling-ruby bundles it all. Without it, the installation process was long, required a network connection and lots of hard dependencies.
@Rush @drnic Thanks. :) Could you also provide photos, icons or pictures that I can use? Should be at least 256x256 pixels.
@FooBarWidget Some logos are here: https://press.virtkick.io/presskit. Let me know if it's enough, or you need something more.
@Nowaker Looks good. Should I quote you or @Rush? What are your positions within your organization?
I've added your testimonials to the website, but I've edited them a little bit: http://phusion.github.io/traveling-ruby/ Is this ok?
@FooBarWidget Looks good, thanks. :-)
Is it possible to include https://github.com/brianmario/yajl-ruby?
Sorry I didn't spot it in https://github.com/cloudfoundry/bosh/blob/master/Gemfile.lock#L31 before as a native extension.
Specifically yajl-ruby (~> 1.1.0)
(perhaps in addition to 1.2.1)
See other issues - I'm struggling to build this myself on both osx & linux - sorry.
I can only package a single version. Maintaining two versions is too much of a burden.
Sent from my Android phone.
@FooBarWidget ok, the latest yajl-ruby is ok if that's all that's possible. I'll try to work it in upstream. TIA.
Could you tell me why you need yajl-ruby? Is the regular json gem not good enough?
I'm the middleman - trying to package the BOSH CLI which is developed by a team at Pivotal; so I'm not sure if there's history around the choice of JSON gem /cc @cppforlife (BOSH product manager).
On Wed, Dec 17, 2014 at 3:46 PM, Hongli Lai notifications@github.com wrote:
Could you tell me why you need yajl-ruby? Is the regular json gem not good enough?
Reply to this email directly or view it on GitHub: https://github.com/phusion/traveling-ruby/issues/6#issuecomment-67418527
Unsure still about switching to pure JSON gem, but BOSH CLI has upgraded to latest yajl-ruby version https://github.com/cloudfoundry/bosh/pull/715#issuecomment-67543637
@FooBarWidget is it ok to include yajl-ruby please? Its my last dependency (afaict) to ship the the bosh-bootstrap experience into traveling-bosh https://github.com/cloudfoundry-community/bosh-bootstrap/blob/master/ChangeLog.md#v015
@drnic Done. Will be part of the next version.
Thanks!!
On Fri, Dec 19, 2014 at 11:18 AM, Hongli Lai notifications@github.com wrote:
@drnic Done. Will be part of the next version.
Reply to this email directly or view it on GitHub: https://github.com/phusion/traveling-ruby/issues/6#issuecomment-67683657
And 20141219 is released.
Ahh, damn (for me). The whole bunch of the BOSH dependency structure requires nokogiri ~> 1.5. Started PRing the dependencies and created parent issue. https://github.com/cloudfoundry/bosh/issues/719
(Just an FYI for the challenges of us only having one native C version of a gem)
Ooh I found a hack for the moment:
def create_package(target)
...
# HACK: Backporting the pre-built nokogiri.bundle to the bundled 1.5.11
extn = Dir["#{package_dir}/lib/vendor/ruby/2.1.0/extensions/*"].first
sh "mv #{extn}/2.1.0-static/nokogiri-{1.6.5,1.5.11}"
if !ENV['DIR_ONLY']
...
end
Could we add eventmachine
1.0.4 (latest) please?
And thin
please (its use is embedded in a bosh CLI process; I can't just switch out to webrick).
I'm working on support for nokogiri 1.5, but it's rather old doesn't have a bundled libxml, so I need to compile that.
It turns out to be harder than I thought. It's best if you upgrade to Nokogiri 1.6.
I have a functioning workaround for nokogiri for the moment whilst I'm trying to get everyone to upgrade. Thin & eventmachine are the newly discovered deps.
On Sun, Dec 21, 2014 at 2:41 AM, Hongli Lai notifications@github.com wrote:
It turns out to be harder than I thought. It's best if you upgrade to Nokogiri 1.6.
Reply to this email directly or view it on GitHub: https://github.com/phusion/traveling-ruby/issues/6#issuecomment-67766284
BTW, new workaround for using latest nokogiri and latest thin/eventmachine (when they are released) is Gemfile + github branches; whilst waiting for PRs & gems to be cut.
Eventmachine and Thin have been added.
Thanks mate!
On Sun, Dec 21, 2014 at 12:27 PM, Hongli Lai notifications@github.com wrote:
Eventmachine and Thin have been added.
Reply to this email directly or view it on GitHub: https://github.com/phusion/traveling-ruby/issues/6#issuecomment-67783376
@FooBarWidget can you bump this ticket again when you have a chance to cut new releases? thanks! http://traveling-ruby.s3-us-west-2.amazonaws.com/list.html
Thin/Eventmachine released as an xmas present? :D
@drnic Done. 20141224 is released.
In issue #3 I have described how I envision support for native extensions to work. Since requiring developers to build native extensions themselves is too much hassle, I proposed providing precompiled versions of the most popular native extensions, that developers can just easily drop into their package.
But this raises the question: which native extensions? We can't automatically package them all. For reasons explained in #3, packaging native extensions requires a lot of manual work. So we have to choose.
I plan on packaging these gems:
Because they're popular:
Because Rails apps likely need these:
I've opened a poll for more native extensions. Please add your input here: https://docs.google.com/spreadsheets/d/1oNgrwHyV9UaMGFTttsfA9QPvL461NWLfHeeAap3Az_A/edit?usp=sharing