search

owasp-amass / amass

In-depth attack surface mapping and asset discovery
https://owasp.org/www-project-amass/
Other
11.97k stars 1.88k forks source link

JSON output is not a valid json #398

Closed omerlh closed 4 years ago

omerlh commented 4 years ago

When running amass enum -d <> -json out.json, the output is not a valid JSON - it contains a list of object, but not in an array format

caffix commented 4 years ago

Back when we supported the array format, users did not like that either. Do you have a reference for which option meets a standard that will make it valid JSON? I am comfortable with either option

crdx commented 4 years ago

I believe this format is known as "JSON Lines" as it allows for streaming the output: http://jsonlines.org/

JSON Lines is a convenient format for storing structured data that may be processed one record at a time. It works well with unix-style text processing tools and shell pipelines. It's a great format for log files. It's also a flexible format for passing messages between cooperating processes.

Perhaps the option -json should output regular JSON and -jsonl should output JSON lines? This is not a backwards-compatible change though.

C-Sto commented 4 years ago

As results are streamed to the file rather than buffered and written all at once, having an array-like output would result in broken output if the program is terminated early or interrupted, and other programs using the output file will need to wait until the run has fully completed before consuming the output file.

I think leaving it as-is is a sane way of doing it, as otherwise you'd need to buffer all of the results in-memory, and for extremely large datasets that doesn't make sense.

r0oth3x49 commented 4 years ago

@caffix for me with latest release amass not saving the output to json file.

just downloaded from releases.

platfrom: ubuntu

***@***:~/path/to/executable$ ./amass -version
v3.6.3

***@***:~/path/to/executable$ ./amass enum -d test.com -passive -timeout 1 -json out.json
Querying ViewDNS for test.com subdomains
... strip ...
test.com
telecoms.test.com
cdn.test.com
otsint.test.com
... strip ..

it prints out on terminal but no json file is generated

caffix commented 4 years ago

@r0oth3x49 There's no JSON output when Amass is run in passive mode

r0oth3x49 commented 4 years ago

@r0oth3x49 There's no JSON output when Amass is run in passive mode

Thanks for the fast reply. and really appreciate your work on this tool. It was producing json file on Amass v3.5.4 with passive flag and now it is not :(. take a look at below screenshot.

Is it removed in latest version?

command used. amass enum -passive -d test.com -json test.json.

caffix commented 4 years ago

@r0oth3x49 It has been explicitly removed given that the same information is easily obtained via the text file output. The JSON output is meant for the normal and active mode, where the output is more complicated. That said, I suppose the JSON output could be activated when the user provides the flag

m-raheem commented 2 months ago

@r0oth3x49 It has been explicitly removed given that the same information is easily obtained via the text file output. The JSON output is meant for the normal and active mode, where the output is more complicated. That said, I suppose the JSON output could be activated when the user provides the flag

`amass enum -d example.com -brute -min-for-recursive 3 -active -r 8.8.8.8,1.1.1.1 -w /usr/share/seclists/Discovery/DNS/dns-Jhaddix.txt -max-dns-queries 100 -p 80,443 -nocolor -json ~/Desktop/targets/expediacruises/amass-subdomain.json -if ~/.config/amass/datastore.yaml flag provided but not defined: -json

Using -active but its saying -json is not defined `