search

CodeIntelligenceTesting / jazzer.js

Coverage-guided, in-process fuzzing for Node.js
https://code-intelligence.com
Apache License 2.0
279 stars 25 forks source link

Help contents added in error message #698

Open manunio opened 10 months ago

manunio commented 10 months ago

Description

Hi, while attempting to make jazzer run in https://github.com/meriyah/meriyah jazzer v2.1.0 adds help content along with error message.

❯ npx jazzer fuzz
jazzer <fuzz_target> [corpus..]

Coverage-guided, in-process fuzzer for the Node.js platform.

The "target" module has to export a function "fuzz" which accepts a byte array
as first parameter and uses that to invoke the actual function to fuzz.

The "corpus" directory is optional and can be used to provide initial seed
input. It is also used to store interesting inputs between fuzzing runs.

To pass options to the internal fuzzing engine (libFuzzer) use a double-dash,
"--", to mark the end of the normal fuzzer arguments. An example is shown in the
examples section of this help message.

Positionals:
  fuzz_target  Name of the module that exports the fuzz target function.[string]
  corpus       Paths to the corpus directories. If not given, no initial seeds
               are used nor interesting inputs saved.                   [string]

Fuzzer:
  -f, --fuzz_entry_point, --fuzz_function   Name of the fuzz test entry point.
                                            It must be an exported function with
                                            a single Buffer parameter
                                                        [string] [default: fuzz]
  -i, --includes,                           Part of filepath names to include in
  --instrumentation_includes                the instrumentation. A tailing "/"
                                            should be used to include
                                            directories and prevent confusion
                                            with filenames. "*" can be used to
                                            include all files.
                                            Can be specified multiple times.
                                                        [array] [default: ["*"]]
  -e, --excludes,                           Part of filepath names to exclude in
  --instrumentation_excludes                the instrumentation. A tailing "/"
                                            should be used to exclude
                                            directories and prevent confusion
                                            with filenames. "*" can be used to
                                            exclude all files.
                                            Can be specified multiple times.
                                             [array] [default: ["node_modules"]]
  -h, --custom_hooks                        Allow users to hook functions. This
                                            can be used for writing bug
                                            detectors, for stubbing, and for
                                            writing feedback functions for the
                                            fuzzer.        [array] [default: []]
      --disable_bug_detectors               A list of patterns to disable
                                            internal bug detectors. By default
                                            all internal bug detectors are
                                            enabled. To disable all, use the
                                            '.*' pattern.Following bug
                                            detectors are available:
                                            command-injection
                                            path-traversal
                                            prototype-pollution
                                                           [array] [default: []]
  -m, --mode                                Configure if fuzzing should be
                                            performed, 'fuzzing' mode, or if the
                                            fuzz target should only be invoked
                                            using existing corpus entries,
                                            'regression' mode.Regression mode is
                                            helpful if only coverage reports
                                            should be generated.
                                                   [string] [default: "fuzzing"]
  -d, --dry_run                             Perform a run with the fuzzing
                                            instrumentation disabled.
                                                      [boolean] [default: false]
      --timeout                             Timeout in milliseconds for each
                                            fuzz test execution.
                                                        [number] [default: 5000]
      --sync                                Run the fuzz target synchronously.
                                                      [boolean] [default: false]
  -v, --verbose                             Enable verbose debugging logs.
                                                      [boolean] [default: false]

Coverage:
      --coverage, --cov                     Enable code coverage.
                                                      [boolean] [default: false]
      --coverage_directory, --cov_dir       Directory for storing coverage
                                            reports.
                                                  [string] [default: "coverage"]
      --coverage_reporters,                 A list of reporter names for writing
      --cov_reporters                       coverage reports.
                              [array] [default: ["json","text","lcov","clover"]]

Options:
      --version  Show version number                                   [boolean]
      --help     Show help                                             [boolean]

Examples:
  jazzer package/target -i packages/foo -i  Start a fuzzing run using the "fuzz"
  packages/bar                              function exported by "target" and
                                            only instrument code in the
                                            "packages/a" and "packages/b"
                                            modules.
  jazzer package/target corpus --           Start a fuzzing run using the "fuzz"
  -max_total_time=60                        function exported by "target" and
                                            use the directory "corpus" to store
                                            newly generated inputs. Also pass
                                            the "-max_total_time" flag to the
                                            internal fuzzing engine (libFuzzer)
                                            to stop the fuzzing run after 60
                                            seconds.

Happy fuzzing!

/home/maxx/dev/security/oss-fuzz-projects/meriyah/dist/src/meriyah.js:1
import { parseSource } from './parser';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at internalCompileFunction (node:internal/vm:73:18)
    at wrapSafe (node:internal/modules/cjs/loader:1178:20)
    at Module._compile (node:internal/modules/cjs/loader:1220:27)
    at Module._compile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/source-map-support/source-map-support.js:568:25)
    at Module.replacementCompile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Function.Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/fuzz.js:1:25)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._compile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/source-map-support/source-map-support.js:568:25)
    at Module.replacementCompile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Function.Module._load (node:internal/modules/cjs/loader:960:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

While downgrading jazzer to v2.0.0 prints following error message

❯ npx jazzer fuzz
/home/maxx/dev/security/oss-fuzz-projects/meriyah/dist/src/meriyah.js:1
import { parseSource } from './parser';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at internalCompileFunction (node:internal/vm:73:18)
    at wrapSafe (node:internal/modules/cjs/loader:1178:20)
    at Module._compile (node:internal/modules/cjs/loader:1220:27)
    at Module._compile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/source-map-support/source-map-support.js:568:25)
    at Module.replacementCompile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Function.Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/fuzz.js:1:25)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._compile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/source-map-support/source-map-support.js:568:25)
    at Module.replacementCompile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Function.Module._load (node:internal/modules/cjs/loader:960:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

Steps to reproduce

oetr commented 10 months ago

Try exporting typescript code to es5 (set "target": "es5", in the tsconfig.json).

ES6 is not supported in CLI mode. You could also use the @jazzer.js/jest-runner for fuzzing--it supports everything that Jest supports, including ES6: https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/docs/jest-integration.md

manunio commented 10 months ago

Try exporting typescript code to es5 (set "target": "es5", in the tsconfig.json).

ES6 is not supported in CLI mode. You could also use the @jazzer.js/jest-runner for fuzzing--it supports everything that Jest supports, including ES6: https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/docs/jest-integration.md

Hi, Thanks for reply and tip for es5, The problem i'm facing is error message adds lots of text (with jazzer -help content) , is this expected behavior ? for example it includes this part

jazzer <fuzz_target> [corpus..]

Coverage-guided, in-process fuzzer for the Node.js platform.

The "target" module has to export a function "fuzz" which accepts a byte array
as first parameter and uses that to invoke the actual function to fuzz.

while it should only include this

/home/maxx/dev/security/oss-fuzz-projects/meriyah/dist/src/meriyah.js:1
import { parseSource } from './parser';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at internalCompileFunction (node:internal/vm:73:18)
    at wrapSafe (node:internal/modules/cjs/loader:1178:20)
    at Module._compile (node:internal/modules/cjs/loader:1220:27)
    at Module._compile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/source-map-support/source-map-support.js:568:25)
    at Module.replacementCompile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Function.Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/fuzz.js:1:25)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._compile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/source-map-support/source-map-support.js:568:25)
    at Module.replacementCompile (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:60:13)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Object.<anonymous> (/home/maxx/dev/security/oss-fuzz-projects/meriyah/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Function.Module._load (node:internal/modules/cjs/loader:960:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)
bertschneider commented 10 months ago

Yes, indeed, that should not happen! We will take a look. Thanks for reporting :+1:

oetr commented 10 months ago

Ah, that should be fixed with https://github.com/CodeIntelligenceTesting/jazzer.js/pull/695 that's currently in review.