search

drupal-composer / drupal-packagist

:package: Drupal Packagist server to automatically build package information from projects on drupal.org
http://packagist.drupal-composer.org
28 stars 4 forks source link

Researching existing approaches and join forces #1

Closed derhasi closed 9 years ago

derhasi commented 9 years ago

We should compare existing approaches and try to join forces on building "the drupal packagist", performant, complete, just awesome:

Existing projects and related issues

davidbarratt commented 9 years ago

I was wondering if it would be possible to create a Drupal site that serves as a drupal packagist?

Composer gives a pretty good overview of what it's looking for: https://getcomposer.org/doc/05-repositories.md

and Drupal.org has a REST API: https://www.drupal.org/api

I think we could write a module to ingest the D.o. API, store what we need in custom entities, and then simply return the JSON that is required by Composer. (ok, it sounds simple when I write it out like that, haha)

dawehner commented 9 years ago

@davidbarratt So you mean we could do exactly what @webflo did on http://drupal-packagist.webflo.io/

davidbarratt commented 9 years ago

yeah, except I think the site should be on Drupal rather than Symfony, but that might be pie in the sky hope. :)

dawehner commented 9 years ago

I would argue that there is no point at all ... drupal.org could host that instance or yeah reinvent the wheel.

kasperg commented 9 years ago

Nice job @webflo!

I think a fork of Packagist is great at this early point. On the other hand Packagist contains a lot of duplicated functionality - search for packages, showing packages information, user registration, publishing and maintaining packages etc. At best this is unneeded - at worst confusing.

As I see it the core of what we need is Composer repository - not a complete Packagist site.

Before this makes it into the drupal.org infrastructure I think we should consider our options. One approach to be to work with the Composer community to separate Packagist frontend from backend.

webflo commented 9 years ago

Thx, but the @winmillwill deserves the credit. I took his code, studied it and deployed it :)

I think Packagist is a good starting point. It´s very straight forward and does a good job to manage all package information and keep them up to date. The generated json files are static, there is no need to expose the web frontend if we move this into the d.o infrastructure.

Currently its just very convenient to view the actual package information and dependencies between modules.

winmillwill commented 9 years ago

The other thing to consider is that the composer user agent can (and does) use the search end point. Granted, this does not happen in the critical install and update use cases.

Still, the easiest thing is to just configure the http server to keep anyone from accessing confusing paths.

derhasi commented 9 years ago

Building a drupal packagist from scratch with Drupal would be a lot of work. I think using packagist as base is great, as it most of the functionality we need is already in there. The original project is the way composer is used to work with. With the web interface we've got some good starting point for "proofreading" and can double-check our composer.json's information.

@winmillwill are you hosting http://drupal-packagist.org/? If so, what code is it running on?

winmillwill commented 9 years ago

static.drupal-packagist.org is just an S3 bucket to which I've uploaded metadata generated with my packagist fork from periodically spinning up several instances with the code and crawling all the Drupal.org repos that have 7.x releases.

My last commits to that fork were for experimental automatic update functionality, but the updates rss feed is all downcase, but the repo URLs are case sensitive, so there will need to be an additional provision for keeping releases.tsv fresh, or just not supporting projects with stupid names. On 24 Dec 2014 09:57, "Johannes Haseitl" notifications@github.com wrote:

Building a drupal packagist from scratch with Drupal would be a lot of work. I think using packagist as base is great, as it most of the functionality we need is already in there. The original project is the way composer is used to work with. With the web interface we've got some good starting point for "proofreading" and can double-check our composer.json's information.

@winmillwill https://github.com/winmillwill are you hosting http://drupal-packagist.org/? If so, what code is it running on?

— Reply to this email directly or view it on GitHub https://github.com/drupal-composer/drupal-packagist/issues/1#issuecomment-68041384 .

derhasi commented 9 years ago

Would you be fine with webflo's fork running on your drupal-packagist.org? I could see to move his code here for "community development". We should not rush on that, but for getting some drive in people actually using composer for Drupal projects, this could be an important step.

winmillwill commented 9 years ago

I am happy with the changes that @webflo has made to the packagist fork, and I'm happy to be available to help and glad for the interest. There is ongoing discussion about the details of representing package dependencies in drupal/parse-composer#.

The thing is that the drupal-packagist.org domain is just an AWS Route 53 DNS entry that doesn't point anywhere. Only static.drupal-packagist.org actually points at a service, and that's just an S3 bucket. If someone wants to pay for a server to run the packagist fork in this organization, I'm happy to hand over the domain.

I think we should just have a few http server rewrites for the front page and other potentially confusing paths so that it's clear that we don't support the normal packagist.org workflow (register, login, add a repo). This way we can leave it on and use the experimental auto update functionality.

Longer term, I think we can do as @kasperg suggests and write some pull-requests for upstream packagist so that our code can be a separate repo instead of a fork, and like @dawehner said, I think we should see about getting someone from drupal.org and/or the Drupal Association interested so that this can become the official way to manage drupal packages.

In any case, I think this issue is resolved unless there are more projects/approaches that we need to consider?

derhasi commented 9 years ago

@webflo just pushed that code to this project. So we have no github fork anymore and can use the repo search :) We hope to get some more visibility from that too.

@winmillwill, If you want we could move drupal-parse-composer too to the org, as it is a crucial part for the packagist. I'd add you to the organisation as well, if you are interested in that too :) I agree with you on the other points and therefore should create separate issues for those.

derhasi commented 9 years ago

@winmillwill ++ :)