search github for packages

[umlaeute]

make this configurable, so multiple projects can be used as package providers

[umlaeute]

the biggest problem I expect is: how to enable multiple projects?

• I don't think it makes sense to search all of github for released fiels matching a certain pattern (apart being impossible, this would just take forever and generate a large number of false positives).
• I don't think we should hardcode the searched projects in the plugin, as this makes maintainance a nightmare (i expect projects to come and go more frequently than deken releases (once it has stabilized enough).
• so we should probably maintain a page that lists all the projects and that can be queried by the plugin.

[chr15m]

I was thinking a bit about this and had a look at the Github API. It's possible to search issues, but not releases. I think we can search all of Github as follows:

• Developer must make an issue which includes the text "Externals release" and add a tag to it with the label "pd-externals-release".
• Inside the issue they have links to the release on Github for each platform.

It's a hack and maybe too onerous, I am not sure.

[umlaeute]

accessing https://api.github.com/repos/${USER}/${PROJECT}/releases gives me a nice json with all the releases and the attached files. i thought about using that, and doing a dumb pattern matching.

it's not exactly search, but we have to pattern match anyhow, to get the architecture. (I was really assuming that the files must follow the same naming-scheme as on puredata.info).

[umlaeute]

but using your idea, how about this:

• run https://api.github.com/search/issues?q=label:pd-externals-release to obtain a list of all repositories that claim to have releases.
• discard any other information in the issue
• for each found repository (remove duplicates):
  • search for files in the releases that match our naming scheme.

thus each repo-owner only needs to create a single issue, marking their repository as "to-be-included"; and from then everything works as if it were puredata.info...

[chr15m]

Great!

[umlaeute]

for performance (crawling through multiple webpages) and implementation reasons (parsing json in tcl), I wonder whether the collecting should actually be done on some server (e.g. a cron-job on puredata.info).

[chr15m]

I feel like that gives us a single point of failure when having multiple backends seems like a way to avoid points of failure. If we require the user to create an issue for each release with the name of the package, not just one, then the performance hit would not be large as we can also add a search string. So I don't know.

[pierreguillot]

Hi, I manage to deploy my PD libraries on Github with Travis CI and Appveyor. Here is a dummy repository for testing. It would be great if the Github binaries could also be available via Deken, do you have any news about this? One problem would be their naming because Github doesn't support parentheses but this could be solved by using another character. Anyway thanks for your work !

[umlaeute]

Rather than putting that logic into the plugin, i think a better place to implement this would be the webinterface (https://deken.puredata.info/). This way, older plugins would automatically benefit from whatever additional source (also, it is much easier to write code in Python than in in Tcl).

Apart from that, no progress has been made on this topic.

[umlaeute]

sidenote: i guess most Pd installations out there do not have https enabled, so these wouldn't be able to download files from github (and i don't really feel like setting up a proxy, that could be abused for all kinds of things)