Change in original technologies.json file structure, Will break wappalyzergo

[bugbaba]

Hello Team,

As the technologies.json file was growing the wappalyzer team had decide to break it into multiple files based on starting character In this commit https://github.com/AliasIO/wappalyzer/commit/2401457a994f40f23140b62e406590bdaa0476a2

We can see the new json files here https://github.com/AliasIO/wappalyzer/tree/master/src/technologies and as this project depends on technologies.json at https://github.com/projectdiscovery/wappalyzergo/blob/master/cmd/update-fingerprints/main.go#L20 it will return 404 error and break the process.

So now we need to download all thee files and then merge them together, before using them.

-- Regards, @bugbaba

[bugbaba]

On looking into this I found that we can merge json files using the jq utility, So another idea/method is we handle this process outside the program that is we download the files using wget and merge those using jq and then pass the output file to the update-fingerprints binary in the github action https://github.com/projectdiscovery/wappalyzergo/blob/master/.github/workflows/update-fingerprints.yaml

jq command to merge multiple files

jq -s 'flatten'  a.json b.json

[CaledoniaProject]

The original repo implemented this recently, consider update the codebase? https://github.com/rverton/webanalyze/issues/55#issuecomment-937749128

[ehsandeep]

thanks for pointing it out @CaledoniaProject, We're looking into it right now.

[ehsandeep]

@CaledoniaProject @bugbaba this is now fixed, thanks for reporting.

[bugbaba]

Thanks @ehsandeep bhau :)