search

ruby-rdf / rdf-vocab

Common RDF Vocabularies
The Unlicense
50 stars 29 forks source link

autoload vocabularies slows 2.7 #64

Closed gkellogg closed 4 years ago

gkellogg commented 4 years ago

As noted in https://github.com/ruby-rdf/linkeddata/issues/13, there seems to be an issue in Ruby 2.7 when using many gems, especially rdf-vocab, which use autoload. It doesn't happen when gems are included in a Gemfile and invoked using Bundler, but otherwise, there is some mysterious slow-down.

At one time, Matz was looking to deprecate autoload, but later recanted.

This issue may be resolved in Ruby core, but we may want to consider relying more on const_missing in the future.

doriantaylor commented 4 years ago

I may have to do something about this ASAP. require 'rdf-vocab' takes literally half an hour.

gkellogg commented 4 years ago

Research shows const_missing is not an appropriate solution. I’ve been away, but when I’ll back I’ll see if an issue has been raised in Ruby Core. Workaround is to use 2.6 or Bundler. Most apps use Bundler, I think.

doriantaylor commented 4 years ago

Can you paste links?

I mean, if there are any besides the original two.

doriantaylor commented 4 years ago

This is another possibility: https://github.com/fxn/zeitwerk

gkellogg commented 4 years ago

@doriantaylor It turns out that the issue wasn't autoload, but differences in loading files in Ruby 2.7 using require. The RDF::Vocab.each method override was loading classes on each call. Now, it forces and autoload on the first call only, which seems to get performance to where it is when using bundler.

If you can, please check it out on the features/slow_load branch. If it looks good, I'll merge and release an update.

doriantaylor commented 4 years ago

thanks! checking…

doriantaylor commented 4 years ago

So here's a weird thing: I thought I had 2.7 installed but the Homebrew latest is 2.6.5. However, it was slow anyway. Notwithstanding, the unit tests on that new branch appear to run at a sensible pace.

I did notice that running the entailments takes a while but I assume this is due to additional vocabularies.

gkellogg commented 4 years ago

Yes, entailment slow with the number of added relationships. They’re cached, so it improves if the process stays live.

It might be possible to create a gem that could marshal these relationships to reduce load time, but it could be brittle and would increase the process initialization time.

doriantaylor commented 4 years ago

I think what may make sense in that regard is perhaps some mechanism to preemptively prune large vocabs from being autoloaded, or perhaps otherwise scan a graph and load only the vocabs for which there are entities belonging therein.

gkellogg commented 4 years ago

Perhaps raise an issue in the rdf-reasoner repo.

gkellogg commented 4 years ago

Fixed in 3.1.2.