search

projectdiscovery / wappalyzergo

A high performance go implementation of Wappalyzer Technology Detection Library
MIT License
723 stars 110 forks source link

Update signatures to pull from chrome extension instead of github #70

Closed TheTechromancer closed 7 months ago

TheTechromancer commented 8 months ago

Since the wappalyzer github repo is now closed-source, this PR pulls the signatures from the chrome extension instead. 😉

olearycrew commented 8 months ago

Thanks for this contribution @TheTechromancer !

Closes #69

UnaPibaGeek commented 8 months ago

Perhaps it would be better to upload the JSON files to a new ProjectDiscovery's GitHub repo and allow the community to maintain them (and pull the files from that repository instead of from the Chrome extension). Wdyt?

TheTechromancer commented 8 months ago

Perhaps it would be better to upload the JSON files to a new ProjectDiscovery's GitHub repo and allow the community to maintain them (and pull the files from that repository instead of from the Chrome extension). Wdyt?

Not a bad idea. But you'd need a way to reconcile the changes between the community fork and the "upstream" closed-source version. According to the author, 90% of the signature contributions were made by himself (I'm unsure how true this is), but at any rate we wouldn't want to fork the signatures and miss out on new updates from the original author.

UnaPibaGeek commented 8 months ago

Yes, but I'm now thinking that there might be legal issues regarding extracting files from a Chrome extension, given that the solution has become closed-source, or no? 🤔. That's why it might be better to start from the last available version of the JSON files, create a repository, and give the community the chance to maintain it (as it was before). Project Discovery has a lot of support from the community (at least for Nuclei); I'd give it a try if they agree 🙏

TheTechromancer commented 8 months ago

Agreed that ProjectDiscovery is in a good position to take this over. The chrome extension is public information so personally I don't see an issue, but I'm not a lawyer. At any rate I doubt legal issues will prevent future wappalyzer clones from periodically stealing the official signatures ;)

Either way, this code will tide us over until a good alternative emerges. It's been three months since the signatures were updated, so it's much needed.

@olearycrew what are your thoughts

olearycrew commented 8 months ago

@ehsandeep is working on a longer-term strategy for how we update our technology detection - and we've also recently added hybrid detection using templates: https://github.com/projectdiscovery/nuclei/pull/4656

I don't think we should make derivative works of any ongoing development from wappalyzer itself as it is now closed source

ehsandeep commented 7 months ago

@TheTechromancer thank you for the PR, with change pushed in https://github.com/projectdiscovery/wappalyzergo/pull/74, wappalyzergo now using Lissy93/wapalyzer as source of information and not making any manual updates.