help

[Kalimon12]

what does it mean --include-other-source

[StasonJatham]

I know what you mean, the descriptions are a little weird.... for --include-other-source it says Also include other-source's urls (still crawl and request) I actually don't understand half of them I had to read the entire code and still don't really get those length check ones haha

The flag you named runs this code if set

                            if includeOtherSourceResult {
                                if crawler.JsonOutput {
                                    sout := core.SpiderOutput{
                                        Input:      crawler.Input,
                                        Source:     "other-sources",
                                        OutputType: "url",
                                        Output:     url,
                                    }
                                    if data, err := jsoniter.MarshalToString(sout); err == nil {
                                        outputFormat = data
                                    }
                                } else if crawler.Quiet {
                                    outputFormat = url
                                }
                                fmt.Println(outputFormat)

                                if crawler.Output != nil {
                                    crawler.Output.WriteToFile(outputFormat)
                                }
                            }

From what I can tell is that all this does is log the URLs that were found from "other sources" like Virustotal, otx, wayback and whatever else they use. I think this is actually not necessary, because whenever you specify that you want to crawl other source you probably also want to see the output :D