search

toptal / chewy

High-level Elasticsearch Ruby framework based on the official elasticsearch-ruby client
MIT License
1.88k stars 366 forks source link

Witchcraft memory allocation #644

Open Gargron opened 6 years ago

Gargron commented 6 years ago

I've run my Rails app through memory_profiler. There is one Chewy index definition that does not use the "witchcraft" technology. Nevertheless, Chewy comes up at the top of memory allocating gems with 13.6797 MiB, with the witchcraft dependency graph being the most of it:

    $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy.rb: 13.6797 MiB
      $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/type.rb: 7.3242 MiB
        $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/type/witchcraft.rb: 7.3242 MiB
          $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parser-2.5.1.0/lib/parser/current.rb: 7.3242 MiB
            $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parser-2.5.1.0/lib/parser/ruby25.rb: 7.3242 MiB
              $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parser-2.5.1.0/lib/parser.rb: 5.2656 MiB
                $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/parser-2.5.1.0/lib/parser/lexer.rb: 5.2656 MiB
      $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/index.rb: 3.3047 MiB
        $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/search.rb: 3.3047 MiB
          $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/search/parameters.rb: 2.7891 MiB
            $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/search/parameters/concerns/query_storage.rb: 1.2891 MiB
              $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/elasticsearch-dsl-0.1.5/lib/elasticsearch/dsl.rb: 1.2891 MiB
            $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/search/parameters/track_scores.rb: 1.2422 MiB
          $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/query.rb: 0.5156 MiB
      $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/elasticsearch-6.0.2/lib/elasticsearch.rb: 1.6055 MiB
        $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/elasticsearch-api-6.0.2/lib/elasticsearch/api.rb: 1.3477 MiB
          $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/elasticsearch-api-6.0.2/lib/elasticsearch/api/actions/cat/plugins.rb: 0.3164 MiB
      $HOME_DIR/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/chewy-5.0.0/lib/chewy/strategy.rb: 0.9297 MiB

I believe that should be preventable, since I am not using that part of the code, there should be a way to make it optional, right?

pyromaniac commented 6 years ago

Yeah, I'm pretty sure it is possible, it just requires some patching which I don't really have time for now. If you will be so kind to send a PR - would be nice.