search

aboutcode-org / scancode-toolkit

:mag: ScanCode detects licenses, copyrights, dependencies by "scanning code" ... to discover and inventory open source and third-party packages used in your code. Sponsored by NLnet project https://nlnet.nl/project/vulnerabilitydatabase, the Google Summer of Code, Azure credits, nexB and others generous sponsors!
https://github.com/aboutcode-org/scancode-toolkit/releases/
2.08k stars 537 forks source link

Timeouts scanning espree JSON test fixtures #701

Open sschuberth opened 7 years ago

sschuberth commented 7 years ago

Yet another report of mine about scan timeouts. Scanning this with ScanCode version 2.0.1.post106.0d1bfba9 results in timeouts for various files:

Some files failed to scan properly:
Path: tests/fixtures/libraries/jquery-1.9.1.js.result.json
  ERROR: Processing interrupted: timeout after 600 seconds.

Path: tests/fixtures/libraries/mootools-1.4.5.js.result.json
  ERROR: Processing interrupted: timeout after 600 seconds.

Path: tests/fixtures/libraries/angular-1.2.5.js.result.json
  ERROR: Processing interrupted: timeout after 600 seconds.

Path: tests/fixtures/libraries/esmangle.browser.js.result.json
  ERROR: Processing interrupted: timeout after 600 seconds.

Path: tests/fixtures/libraries/yui-3.12.0.js.result.json
  ERROR: Processing interrupted: timeout after 600 seconds.

Maybe there's something less involving than the general purpose solution proposed here to address these (JSON-specific?) timeouts?

pombredanne commented 7 years ago

@sschuberth Thanks! For this as a more general solution I am inching towards something like --maximum-size arg where when a file is bigger than a certain size, only that much of it will be scanned.

sschuberth commented 7 years ago

Ah, I was hoping that a fix of the kind in #641 or #694 would be possible.

pombredanne commented 7 years ago

@sschuberth it may too. Still working a solution in a not-yet-pushed branch atm

pombredanne commented 5 years ago

With #1552 this now work as explained in https://github.com/nexB/scancode-toolkit/issues/695#issuecomment-498178213 Everything is detected correctly AND does not timeout