[RFC] Moving extensions into separate repos for Marketplace

[GwendolenLynch]

As @rossriley is making great progress with Marketplace, I thought it was a good time to start a conversation about moving extensions out of core and into repos so they can be accessed by the new system in 2.0

What's going where and who needs help?

Ross has a draft wiki page outlining what needs to be changed and I have a very basic working example.

Current extensions and listed authors

Extension	Author(s)	Status
Authenticate	TwoKings (Lodewijk Evers, Tobias Dammers, Bob den Otter)	Complete
Disqus	Bob den Otter	Complete
Editable	Rix Beck	Complete
FacebookComments	Bob den Otter	Complete
FacebookLike	Bob den Otter	Complete
GoogleAnalytics	Bob den Otter	Complete
HelloWorld	Bob den Otter	Complete
ImportWXR	Bob den Otter	Complete
JSONAccess	Tobias Dammers	Complete
MenuEditor	Steven Wüthrich / bacbos lab	Complete
MetaTags	Xiao-Hu Tai	Complete
NiceUrls	WeDesignIt, Patrick van Kouteren, Miguel Angel Gabriel	Complete
Omnisearch	Xiao-Hu Tai	Integrated
PasswordProtect	Bob den Otter	Complete
PinIt	Gawain Lynch	Complete
PlusOne	Till Klocke	Complete
RateIt	Gawain Lynch	Complete
Redirector	'Mike Anthony / Foundry Code'	Complete
RelatedContentByTags	Xiao-Hu Tai	Complete
RSSAggregator	Sebastian Klier (passed to Gawain Lynch)	Complete
RSSFeed	WeDesignIt, Patrick van Kouteren, Gawain Lynch	Complete
SimpleForms	Bob den Otter	Complete
Sitemap	Bob den Otter / Patrick van Kouteren	Complete
Socialite	Gawain Lynch	Complete
TestSnippets	Bob den Otter	Complete
TitleCase	Bob den Otter	Complete
TwitterButton	Bob den Otter	Complete
TwitterFeed	Gawain Lynch	Complete
WaffleOrders	Tobias Dammers	Complete

[jadwigo]

pinging myself for https://github.com/jadwigo/TaxonomyList

[xiaohutai]

Hm, nice. Are we also going to have an official Bolt Github for extensions. Or do we put them all under our user accounts?

I'll probably convert these:

• https://github.com/xiaohutai/bolt-MetaTags
• https://github.com/xiaohutai/bolt-RelatedContentByTags

As far as I know, Omnisearch can be removed. Since it's in the new Bolt.

[GwendolenLynch]

Are we also going to have an official Bolt Github for extensions

There was mention of that in IRC, but I was hoping to get a more solid confirmation via this thread myself..

[bobdenotter]

Off the top of my head, there are three options:

• Keep "ours" inside the Bolt org. ie. github.com/bolt/Authenticate
• Move as many as possible to their respective owners. ie. github.com/tobias2k/JSONAccess
• Create a new org with the same members, and place most of them there. ie. github.com/bolt-extensions/Disqus

The advantage of option 1 is that we're keeping the current ones together. The downside might be that the bolt org will get a bit crowded. But, is that really a problem? An additional benefit is that they will be seen as "sort of official" The second option might be nicer for people to have them in their own repo's. The major downside is ongoing maintenance. For example, what happens if Tobias is too busy to maintain his extensions? nobody will be able to fix these, then. The third one might solve the "too crowded" argument for option 1, but it feels contrived.

I'm leaning towards option 1, with the exception that for some extensions their respective original authors might want to "reclaim" ownership. (for example, I can imagine @bacbos wanting to have and maintain his menu-editor in his own account)

[jadwigo]

as many to their owners + a new org for the official / recommended / maintained by core team extensions?

[bobdenotter]

New org seems like a hassle to me, personally. :-)

[GwendolenLynch]

Options 1, 2 & 3 can all work together. @rossriley's Marketplace relies on a VCS repo for the extension.

Personally I see great advantage to a 'blessed' org with repos for Authticate -> RateIt, etc. The advantage as I see it is:

• Single issue tracker for most extensions
• Authors can be given commit access to the extension that they are considered 'In-charge-of'
• Others can still spot and fix bugs

[mikerockett]

Right, so all I need to do with Redirector is implement the Composer autoloader thingy? Sorry, haven't been following all the threads about the new Marketplace.

[GwendolenLynch]

@mike-anthony the brief summary for the extension code is in Ross' wiki page.

Then you can submit the VCS URL and details to the test extension site (BIG SCARY ALPHA/BETA WARNING).

Also a number of us are now hanging out on FreeNode IRC in #boltcms if you get stuck.

[bobdenotter]

Hey Mike,

There's some preliminary documentation on it here: https://github.com/bolt/bolt/wiki/Bolt-Extensions:-Converting-1.x-extensions-to-work-in-version-2

edit: Derp! Gawain beat me to it! shakes fist

[mikerockett]

Thanks guys - I did see that :) Will convert Redirector over the weekend.

[xiaohutai]

A few questions.

What does this part mean and what's supposed to be in there?

    "extra": {
        "branch-alias": {
            "dev-master": "1.7.*"
        }
    }

And, is this necessary for all new extensions now?

        "files": [
            "init.php"
        ],

Thanks!

[rossriley]

What does this part mean and what's supposed to be in there?

"extra": {
    "branch-alias": {
        "dev-master": "1.7.*"
    }
}

This is optional, it allows you to specify a version that the current master branch is equivalent to. Most extension publishers can ignore this, in fact we could take the decision to allow Bolt to only download stable extensions and not the dev-master branch.

And, is this necessary for all new extensions now?

    "files": [
        "init.php"
    ],

Yes. Responsibility for loading is now given to the extension, this is the equivalent in the current version of adding an extension to the enabled_extensions config. The init file just calls $app['extensions']->register($extensionObject);

[xiaohutai]

Many thanks for the answers, @rossriley. I guess I'll keep the "extra" stuff in, so you can see it when editing.

[xiaohutai]

So, I've been converting some extensions. I just saw that you can't submit too many extensions within a certain timespan on beta.extensions.bolt.cm. If you try to update one of your extensions, you'll see a

Failed to execute git clone --mirror ...

Not sure what this is for, but perhaps we can just fetch the raw composer.json directly from github instead?

---

Converted these:

• Disqus
• FacebookComments
• FacebookLike
• GoogleAnalytics
• HelloWorld
• ImportWXR
• PasswordProtect
• Sitemap
• TestSnippets
• TitleCase
• TwitterButton
• WaffleOrders

I haven't converted Authenticate and SimpleForms yet, because I sense that people are still working on those at the moment.

[rossriley]

I've been converting some extensions. I just saw that you can't submit too many extensions within a certain timespan

I do know about that, we're just hitting the Github API limit, I've got an auth token to add to it and then this should go away.

We're using the API to do it because it's more complicated that just fetching the single composer.json, we need to also scan branches and tags to get the different versions of the extension.

[xiaohutai]

Not sure what I did wrong with the extensions for Bolt:

I keep getting the following output when trying to install them:

Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - The requested package bolt/disqus could not be found in any version, there may be a typo in the package name.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details.

Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Installation failed, reverting ./composer.json to its original content.

But xiaohutai/bolt-MetaTags and xiaohutai/bolt-RelatedContentByTags work fine.

[GwendolenLynch]

In your composer.json you have "name": "xiaohutai/bolt-Disqus" it should be just "name": "bolt/Disqus" in this case to match what is set on beta.extensions.bolt.cm

[xiaohutai]

I have made multiple tests, I first tried with the repo bolt/Disqus, later forked a test under my user account. See https://github.com/bolt/Disqus , not sure what is wrong with this.

[rossriley]

These should all be sorted now. The problem was the build script was throwing an exception on one of the extensions which had an error in the composer.json. I've added a skip-errors to the script which continues the build but marks the affected extensions as unavailable, so this should fix any problems with adding extensions.

Note also that since we've bumped the master version to 2.0 now any extensions require setting need to fall within this range. something like:

"require": {
    "bolt/bolt": ">1.9,<3.0.0"
},

[mikerockett]

Right, I hope I've done the right thing here... Please check foundry-code/bolt-redirector to see if I've done the right thing... :smiley:

Edit: Do I need to move the contents of Redirector to the root, and get rid of the rest?

[GwendolenLynch]

@mike-anthony Yes, the contents in root.

Also extensions.php should be renamed to Extensions.php, but other than than it looks good at first glance.

[mikerockett]

@GawainLynch Cool, thanks. Shall commit shortly. Then I assume its safe to submit to the Marketplace?

[GwendolenLynch]

Yes mate. Just be aware that @rossriley has reserved the right to empty the database on the beta site before we go live. So you might need to resubmit closer to release but it is about 15 seconds worth of work each time anyway :-)

[mikerockett]

No worries there - just for testing :+1:

[mikerockett]

Done!

[GwendolenLynch]

Oh @mike-anthony one more thing, the info() block is not needed — doesn't do anything any more — so if you're OCD like me, feel free to remove.

[mikerockett]

Right, shall remove.

[mikerockett]

Extensions site is just showing Array ( [] => Array ( [dev-master] => ) ) :open_mouth:

[GwendolenLynch]

Yeah, I'd gather @rossriley is working on it at the moment or something, looks to be all extensions are showing the same.

[mikerockett]

Ah, right! Didn't even check. Thanks @GawainLynch :+1:

[mikerockett]

Site is now throwing this fault:

No valid composer.json was found in any branch or tag of https://github.com/foundry-code/bolt-redirector, could not load a package from it.

Here's my composer.json file.

I'm not very familiar with the inner working of these files, because I've never really used them before, so there's a good chance that something's wrong with it.

[rossriley]

Hi Mike.

The only problem is this line:

"bolt/bolt" : ">2.0,<3.0.0"

the current version is 2.0.0 so what you will need is:

"bolt/bolt" : ">=2.0.0,<3.0.0"

[mikerockett]

Thanks @rossriley. :+1:

[bobdenotter]

I believe all (or at least most) extension have been moved to their own repos. I'll check and remove them from the main repo.

[GwendolenLynch]

@bobdenotter some of mine are still yet to be done, but I can grab them from release/1.6 so nuke away

[mikerockett]

Redirector seems done to me. Works on my local install, and the marketplace is also happy (albeit a silly little hack).

[GwendolenLynch]

Cool @mike-anthony, updated the table.

Also feel free to submit a PR removing the old extension from master, alternatively just let use know and we'll handle it.

[mikerockett]

Shall do so shortly :smiley:

[mikerockett]

And done :+1:

[bacbos]

FYI: refactorings of MenuEditor are done, it's now fully compatible with 2.0.0 for those interested

[xiaohutai]

Whoop! JSONAccess has its own repo now, at https://github.com/bolt/jsonaccess (see also #1595 and #1711)

[GwendolenLynch]

@xiaohutai You just beat me to commenting the same! :beers:

Job done, thanks to all :-)