search

platformio / platformio-libmirror

PlatformIO libraries mirror
Apache License 2.0
16 stars 25 forks source link

Add TOP Arduino based libraries #8

Closed ivankravets closed 8 years ago

ivankravets commented 9 years ago

Using search sorted by STARS results by GitHub request arduino library

Our contributor @gandy92 :blush:

P.S: I processed approximately first 4-5 pages, but I would ask you to re-check it again from the begining.

gandy92 commented 9 years ago

Wow, quite a list.. :100:

Do you have any criteria which libraries qualify and which maybe don't? Can I exclude repositories that already contain library.json files?

ivankravets commented 9 years ago

Do you have any criteria which libraries qualify and which maybe don't? They are already sorted by STARS (most popular on TOP). I mean, that we will add each by each, beginning from the first library.

Can I exclude repositories that already contain library.json files?

Sure, I've mentioned above that I already added the libs from first 4-5 pages. However, please re-check that all libs from 4-5 pages are added to PlatformIO Library Registry.

Not all libs contain library.json, because owners of the libs have not merged by pull-requests yet. If you are going to create new library.json for the library, then please verify:

  1. Lib doesn't have library.json
  2. Lib is not registered in PlatformIO Library Registry. Use on-line search http://platformio.org/#!/lib
  3. Lib doesn't contain any "pull-requests" with library.json or by "PlatformIO...." title.

P.S: Sorry for these tasks, but I really don't have time for it. I hope, you see how much issues are opened for PlatformIO https://github.com/ivankravets/platformio/issues. Need to fix them :smile:

gandy92 commented 9 years ago

I'm currently looking into a slightly different approach, that would allow for a little more scripting, a.k. automization.

  1. I wrote a script that downloads the first 100 pages of the github search for "android library", sorted by "Most stars". From this, I extract all found github projects.
  2. I installed all available platformio libraries, so I can access their respective library.json files. From those I extract all github hosted repository-urls
  3. I check all projects found in (1) to see if the project is already listed in (2) - if not, it's a candidate for closer inspection.

This scheme allows me to get an up-to-date picture just by running the script, rather than manually inspecting all found repositories. Here is the result for the first 25 findings ("O" denotes a match, "-" denotes a possible candidate for closer inspection):

However, I am not quite happy with (2) since I have to install all available libraries, first.

Is there a way to download the complete list of libraries, and their respective library.json files? Surely the PlatformIO Library Manager must have a way to do so. Can I access this information via http? It would help making the script more independent from the installed libraries.

ivankravets commented 9 years ago

Email me@ikravets.com , I will send you these information like export from DB. PlatformIO Library Registry has API, but this information isn't available in DB. See what API can propose you,

http://api.platformio.org/lib/info/1

Here are all models from API : https://github.com/ivankravets/platformio-api/blob/develop/platformio_api/models.py

I can export you Libs table, it contains conf_url, then download JSON and extract repository data.

gandy92 commented 9 years ago

A pull-request for a library I registered with an URL pointing to my fork was just accepted, so the crawler could now use the official repository. I placed a platformio lib register with the new URL but maybe there is a better/preferrable way to do this?

ivankravets commented 9 years ago

I have more then 100+ pull requests for libs, and time by time I change "forked" lib URLs by "merged" original. Moreover, it's hard to monitor who have accepted pull-requests or not.

How about additional script that analyses libs which have "forked" URL and "merged" library.json.

For example, we have library with this library.json https://raw.githubusercontent.com/gandy92/Arduino-SerialCommand/master/library.json.

Then we ...

[https://raw.githubusercontent.com/kroimon/Arduino-SerialCommand/] + [branch/] + [field "include" if exists] + [library.json]

The default branch name can be fetched using GitHub API.

If GitHub returns "200 OK", then library has merged library.json and we can create the list with these libs.

What do you think?

gandy92 commented 9 years ago

Just beautiful, let me give it a try.

What would be the best output format of this script? Two columns comprising library ID and new URL? Possibly an additional column with the old URL (so we know who should contact the library author for a follow-up)?

ivankravets commented 9 years ago

The "Two columns comprising library ID and new URL" would be good for me. Then I can occasional check this file and update DB.

ivankravets commented 9 years ago

https://github.com/platformio/platformio-libmirror/blob/master/scripts/github-top/list.accepted.md looks good for me. I've just corrected all libs which have own(merged) library.json.

Please don't re-register libs twice: first via fork, then via master. I will check your list.accepted.md time by time and update DB :+1:

hschwane commented 8 years ago

could you not somehow interface with the Arduino library manager, and add all libraries that are in there to PlatformIO? I just switched to Platform IO and have to install a lot of libraries manually.

ivankravets commented 8 years ago

@hschwane all libraries are on our registry. Please check it http://platformio.org/lib

Now, PlatformIO support multiple manifests: Arduino library.properties, ARM mbed yotta module.json. You can submit these manifests without changes.

See http://docs.platformio.org/en/latest/userguide/lib/cmd_register.html