search

rverton / webanalyze

Port of Wappalyzer (uncovers technologies used on websites) to automate mass scanning.
MIT License
908 stars 134 forks source link

Update does not work due to source format change #55

Closed codypierce closed 2 years ago

codypierce commented 2 years ago

./webanalyze -update no longer works due to a format change at https://github.com/AliasIO/wappalyzer/tree/master/src/technologies

$ ./webanalyze -update 2021/09/09 19:14:06 app definition file updated from https://raw.githubusercontent.com/AliasIO/Wappalyzer/master/src/technologies.json $ cat technologies.json 404: Not Found%

rverton commented 2 years ago

Oh wow, what have they done.. The technologies.json file was split into [a-z].js files now. I suggest we download all and merge them again to a single file. If someone wants to step in with a PR I'm happy to merge. Otherwise I will need a few days until I can allocate some time.

rverton commented 2 years ago

I just pushed a fix for this, however I did notice some other problems with the schema they are using now. Some fields are renamed, some are missing and some are new. I will open a new issue for this.

Let me know if this fix works for you.

CaledoniaProject commented 2 years ago

Hi rverton, looks like you've merged all the JSON files locally and uploaded to this repo. Why not download all the A-Z files and merge it in the handler function instead?

rverton commented 2 years ago

I did not just update the technologies.json, I also added the -update handler/flag to download all files and merge them into a single technologies.json file. This allows to update to the current wappalyzer apps definitions.