burrito-brothers / shiba

Catch bad SQL queries before they cause problems in production
https://shiba-sql.com
Other
292 stars 14 forks source link

No problems found caused by the diff - although problems exist #27

Open golddiga opened 5 years ago

golddiga commented 5 years ago

I have created a method that clearly is of high severity when I test it with shiba locally. But when pushing this mehtod + specs to CircleCI it returns me that no errors where found in the diff.

The diff includes the problematic lines in spec/../model_spec.rb and models/model.rb

def find_something
   Model.where('title like "%something%")
end
RAILS_ENV=test SHIBA_DEBUG=true bundle exec shiba review --verbose --submit
Finding default options from CI environment.
diff: 
branch: 098b26375a707373f590910840ce900143b88c3a
pull_request: 919
 origin/HEAD...098b26375a707373f590910840ce900143b88c3a
grep /tmp/shiba/ci.json -e .circleci/config.yml -e Gemfile -e Gemfile.lock -e app/models/model.rb -e db/migrate/20190517103657_add_indexes.rb -e db/schema.rb -e spec/models/model_spec.rb -e spec/rails_helper.rb
Updated lines: [[".circleci/config.yml", 68..68], [".circleci/config.yml", 119..122], ["Gemfile", 121..123], ["Gemfile.lock", 94..94], ["Gemfile.lock", 100..100], ["Gemfile.lock", 449..449], ["Gemfile.lock", 616..618], ["Gemfile.lock", 710..710], ["Gemfile.lock", 772..772], ["app/models/model.rb", 87..91], ["db/migrate/20190517103657_add_indexes.rb", 1..7], ["db/schema.rb", 13..13], ["db/schema.rb", 52..52], ["db/schema.rb", 103..103], ["db/schema.rb", 105..105], ["db/schema.rb", 126..128], ["db/schema.rb", 149..156], ["db/schema.rb", 179..179], ["db/schema.rb", 186..186], ["db/schema.rb", 300..300], ["db/schema.rb", 561..561], ["spec/models/model_spec.rb", 25..31], ["spec/rails_helper.rb", 11..13]]
No problems found caused by the diff

circleci/config.yml

  run_tests:
    <<: *build_defaults
    parallelism: 2
    steps:
      - *attach_workspace
      - *restore_bundle_cache
      - run: bundle install --deployment --with development test
      - run:
          name: Set up test database
          command: RAILS_ENV=test bundle exec rake db:create db:migrate
      - run:
          name: Run tests
          command: |
            if [[ `git log -1 --pretty=%B` == *"[hotfix]"* ]]; then
              echo "Hotfix fast-laned. Skipping tests.";
              exit 0;
            else
              RAILS_ENV=test CIRCLE_BUILD_NUM=$CIRCLE_WORKFLOW_ID bundle exec rspec $(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=filesize);
            fi
      - run:
          name: Review SQL queries
          command: RAILS_ENV=test SHIBA_DEBUG=true bundle exec shiba review --verbose --submit
eac commented 5 years ago

Thanks for the detailed report. Is this running mysql or postgres? Can you provide the output of analyzing the query in the rails console? This requires running the same database locally as in production. If you're using sqlite you'll receive an error. Alternatively the output from your local run is fine too.

rails console

require 'shiba/console'
shiba Model.find_something

Console usage is documented at https://github.com/burrito-brothers/shiba#analyze-queries-from-the-developer-console

eac commented 5 years ago

Also, I see you're running parallel jobs. Is the output for both identical?

golddiga commented 5 years ago

Thanks, We are running on mysql 5.6 and there is only one output

image

The output when running in local is:

2.6.3 :004 > require 'shiba/console'
 => true
2.6.3 :005 > shiba Model.find_something

Severity: high
----------------------------
 * Fuzzed Data: Table sizes estimated as follows -- 100000: models
 * Table Scan: mysql reads 100% (100000) of the of the rows in **models**, skipping any indexes.
 * Results: mysql returns 40.1mb (100000 rows) to the client.
 * Estimated query time: 15.69s

 => #<Shiba::Console::ExplainRecord:0x00007f95748c8eb0>: 'SELECT `models`.* FROM `models` WHERE (title like "%something%")'. Call the 'help' method on this object for more info.