tidev / module-search-www

Search native Titanium modules with ease!
https://modules.titaniumsdk.com
Apache License 2.0
3 stars 2 forks source link

Generate local data file and serve via API #4

Closed ewanharris closed 1 year ago

ewanharris commented 1 year ago

Most likely not production ready but more of a discussion topic.

This moves from searching the GitHub API directly, and instead generating a data file and then using that to then serve an API off the next server that the frontend will use. It works locally, but no idea how the build setup works on dokku (i.e will the data file not be included, there's almost certainly a better place than in the root of the repo).

The data has been reduced from the raw GitHub API but we can add more and then each run of the script will pull that in

m1ga commented 1 year ago

very nice :+1: that "fixes" my comment in https://github.com/tidev/organization-docs/issues/33#issuecomment-1415256876 Then we can do some checks (see if it has manifest file or timodule.xml) on the results before adding it to the file and we can run multiple queries like https://github.com/search?q=topic%3Atitanium-module&type=repositories (that will only return valid modules unless someone else used the tag) and merge them.

ewanharris commented 1 year ago

Added a new commit that makes a best effort at determining supported platform for a module based on whether it contains an android, ios, or iphone module.

This will filter out bad repos (total is now 103 before it was ~200-300) and also allow showing supported platforms on the site, however it can take anywhere fro 3-6 minutes to run the script now due to the amount of API calls that are made.

I think what might be best is to take that data and then create an ignore list from it (manually filtering out deprecated modules etc.), and a known list from it, that can then be used to reduce the amount of platform lookups we have to do by running the search and then only looking up new modules.