Open eapache opened 10 years ago
I gave this a try over the past few days, utilizing Parallel.each
for parsing the AST, such as:
def add_files(files)
Parallel.each do |file|
@output.extra("Adding `#{file}`")
parse_file(file)
end
end
Unfortunately, since parsing the AST depends upon db
's @table
and @meta
instance variables, putting the parsing into threads via parallel
meant that those instance variables were not shared across threading contexts. Unwinding this is probably possible, but wasn't dead-simple. I think using https://github.com/ruby-concurrency/concurrent-ruby may require less surgery within starscope
.
Yeah, that's possible.
In practice I've found, for very large ruby projects, https://github.com/eapache/starscope/issues/95 and https://github.com/eapache/starscope/issues/154 dominate as performance issues anyway. If I had the time to do performance surgery, that's where I'd start.
Perhaps via https://github.com/grosser/parallel or some other gem.
Parsing files (in particular large Ruby files) is slow, so parallelizing it would be a big speed-up on multi-core systems.