search

devanshbatham / ParamSpider

Mining URLs from dark corners of Web Archives for bug hunting/fuzzing/further probing
MIT License
2.48k stars 422 forks source link

Fixing fetching retry , union session parameters into a single txt file and merging output urls patch #127

Open ISLAM-XGAMER opened 3 months ago

ISLAM-XGAMER commented 3 months ago

There were mainly 3 issue with the code

  1. Retry mechanism logic is to stop after 3 tries , @nitish800 have changed this issue to continue to the next URL and I formatted it to be more user friendly

  2. Any URL with slash sign " / " etc...(example.com/my) will be passed normally as an input the process of fetching will work fine until creating the txt name with that URL containing slash sign "/" , the script will raise error that result_file = os.path.join(results_dir, f"{domain}.txt")cant create such txt file and that's normal cause @@Linux file system cant accept / as a name as it is represents directories ,I have replaced it with fraction slash character "\u2044" , this will be effective when a user inputs a redirected subdomain comes from another tool

  3. The parameters for every domain is saved in a txt file with the name of that domain so most hunters will have to merge them by the CLI like cat * > params.txt so I have made additional feature that saves all generated parameters in a txt file for every session aka execute of the script named by the date and time of the execution of the sctipt example (Session : 2024_06_13_20:41:27.txt)

knrredhelmet commented 2 weeks ago

i solved most of the issue here https://github.com/knrredhelmet/paraminer