Move the web entry point

[chrisdavenport]

Currently the web entry point is /www/index.php but I think the default web entry point should be /api/api.php. Ideally, this should be configurable so you could move api.php to the web root or some other directory if you prefer. Installation in the web root would mean that people would need to edit their htaccess file before it could be used, whereas using a sub-directory would mean we can ship with a default htaccess file which simply needs to be copied/renamed.

[wilsonge]

Slightly the wrong way around imho. The web entry point can go where-ever you want. It's the paths of everything relative to the web root that you'd need to configure :)

[chrisdavenport]

Yes, I thought that's what I said. But the point is that I think the default location should be /api/api.php and not /www/index.php. You can't have two different web roots. By default I want the URLs to look like this: http://www.example.com/api/contacts/1234 rather than http://www.example.com/www/contacts/1234

Renaming to api.php means that I could, if I wanted to, move it to the web root and have URLs like this http://www.example.com/api.php/contacts/1234 or with a bit of htaccess magic http://www.example.com/contacts/1234 despite having a Joomla CMS in the same location.

[mbabker]

Again this assumes that you're dropping the repo as is in the web root of a CMS install. There is nothing binding saying you can't clone there repo in the current state, put the src repo outside the root and put the www folder anywhere that's web accessible. As long as you edited www/index.php to reference the right system paths, you could put it at media/system/css/crazy.php for all anyone cares.

[chrisdavenport]

That's right. But what I'm aiming for is a drop-dead simple installation process that, by default, will put the web entry point into /api/api.php. The code in /src can go somewhere, not sure where.

It could be that we leave the repo structure as it is now, but then use a packaging script to generate the package that will be installed. That's fine too. In fact, that's probably a better idea.

[mbabker]

Repo structure is next to nothing in the long run. You're still making code decisions that impact everything. From the configuration file format to now requiring the defined or die checks because you're gonna drop the entire codebase in the web root, things like that. And at this point I really don't see the benefit to making this distributed in such a way that it can be magically installed and configured as a CMS extension other than you target Joomla's marketplace of people who couldn't tell you what php cli/deletefiles.php does.

[ronnikc]

Id go with the "complex" solution which requires more knowledge, configuration etc. - and then consider a simpler /api/ entry point that does the brain dead logics with conventions over anything as a layer of standard specifications for the lowest level of threshold for entry to using webservices.

Then you can still accomodate the "masses" while allowing for something alot more flexible and powerful (which was also the original intent for us).

[chrisdavenport]

I want to make it as easy as possible to install while still retaining the advanced installation options for those with the requisite skills. If we make it hard to install we limit its adoption.

Using the CMS installer is one way to make it really easy to install, but if that's going to cause problems, then I'm open to other methods. One way or another we need to get the installation process working smoothly. The current installation procedure requires too much technical knowledge.

I'm open to suggestions.

[mbabker]

Look at the feature demand and the possible target market. Is webservices really something that is going to be used by a majority of CMS installations? Or does it have a niche marketplace and can be more tuned for that instead of going for a default "make it dead simple for everybody everywhere" stance? I don't think it's gonna have the mass distro appeal as in general webservices is more technical in nature than "conventional" CMS installs, so for now I'd suggest going with it as is. If it's found later that there is a mass appeal to having it be easily downloadable and installed through the CMS, then make the changes needed.

[chrisdavenport]

I disagree that the target market is purely technical. This is a foundational technology. By itself it doesn't provide any real value; it's what you build on top of it that defines the market. And we have no way of knowing what people will want to build on top of it. But if we start by making something that is inherently difficult to install then we also restrict the target markets of the things people can build on it. No-one would create a mass market product if the foundation on which it's built requires deep technical knowledge to deploy. I think we should make it as easy as possible for anyone to install. If we can't achieve a "one click and it's done" experience, then so be it, but we should endeavour to get as near to that as we can. Why does it need to be hard?

[mbabker]

It all boils down to what your target market is and how you expect people to build on it. If your aim is to replicate a component's experience, then a fully standalone application is overkill. If this is supposed to be the building blocks for people writing code to build on top of, then honestly if they can't figure out how to install this then their skill set needs to be called into question being perfectly blunt about it.

Using your assumptions about install difficulty, installing the Symfony Standard Edition is beyond the capabilities of a lot of Joomla's userbase (http://symfony.com/doc/current/book/installation.html) because it is not a simple ZIP package that you extract onto a server or load into an interface.

[chrisdavenport]

It's not the developers I'm worried about. As you say, they'll figure it out. It's the users of the products that those developers will create that I'm worried about.

It's like saying that we don't need to provide an easy-to-use CMS extension installer because only developers will be writing extensions. Sure, you need to be a developer to write an extension, but you shouldn't need to be a developer to install one.

Joomla is a platform on which others (developers) build products. If we don't make it easy for people to deploy those products then we may not even attract the developers to create the products in the first place.

Let's take an example. Suppose I'm a third-party extension developer and I want to create a mobile app that does something cool in combination with my component. My component is easy to install because I just distribute the package file and my customers use the CMS installer. I'd want to make the experience of installing the webservices API required to support my mobile app just as simple. If I know it's going to be painful for my customers to deploy, then I'm probably not going to bother creating that product in the first place.

The easier we make it, the more doors we open.

[mbabker]

So I go back to my argument then that this should be more closely structured to the CMS application stack itself versus something segregated from it like this structure is. I don't see this standalone application being an extension of Joomla in any way beyond the component and plugin that are in the repo now (which from the looks of things provides more admin tasks to the CMS user than actual functionality to a webservices layer, correct me if I'm wrong). Rather, I see this application as a bridging layer between someone consuming this data by whatever interfaces this layer provides and the actual Joomla application. So by its very nature right now it's already not something that's an easy-to-use solution to end users or even a drop-in solution.

You're also opening a whole new can of worms if you get into users trying to install this. From the CMS installer not supporting the notion of an application at all (and files extensions are just a weak catch all for any case that isn't supported) to the lack of dependency management in the CMS itself. What do you do when the day comes that you have a WSv1 and WSv2 and components have different requirements?

[chrisdavenport]

Some of the changes I'm suggesting are indeed intended to make it "more closely structured to the CMS application stack". That's because the repo structure we have right now makes it hard to install in the CMS context. Historically, this started as a pure CMS extension, then George refactored it as a pure standalone application. With the benefit of experience I now think that a standalone application which is structured to play nicely alongside the CMS is the best path forwards.

You're right about the admin component being purely for the benefit of the CMS user. That's the intention. Everything that can be done through the admin component can be done by directly editing the config files in the webservices application. (Actually right now that's not quite true, but I want to remove the last few dependencies at some point). The admin component makes it easy to manage, but you can throw it away and the API will continue working. It's like the GUI on top of a Linux application. The application works without it, but the GUI puts a user-friendly face on it.

Yes, it is kind of a bridging layer, but I disagree that we can't make it something that is (relatively) easy to drop-in when it's needed.

I wasn't originally thinking of using the CMS installer, but it does solve, at least partially, a couple of problems. Firstly, it makes it easy to install. Even though you might not like files extensions they offer a useful workaround until something better comes along. Secondly, it provides an update mechanism that we would otherwise have to (re-)invent for ourselves. As a standalone application there is no easy way to even know that an update is available, let alone an easy way to install that update. With the CMS installer that all comes for free. True, if you choose to install it in a non-default configuration, such as moving the files outside the web root, then you're on your own, but that would have been the case anyway.

Versioning of the application and versioning of the services themselves are separate. The application is able to support multiple versions of the same service. I would hope that we won't need to do a new major version of the application itself before Joomla 4 (which should make it obsolete anyway), but if we do then the (last resort) approach would be to create a new entry point (eg. /api2 instead of /api). Better would be to include the version in the content negotiation and route accordingly, but until we know just what is being changed that triggered the new major version it's difficult to know if that is feasible.

[HermanPeeren]

Practical solution: make a com_api component, that does nothing at the moment. An administrator page could show some information about the API. The installation package of that component has an install script, that installs the webservice application in /api. That is to say: the script installs /www/index.php into /api/index.php and it installs the rest that is needed above webroot (later to be extended by installing it elsewhere as fallback, in case someone has no writing rights above webroot or problems like that). The install script can also copy the information that is needed for the webservices application from configuration.php.

The application can be reached via /api/whatever/resources/you/want. If later we would have the API via a com_api component, then the URL won't change. This solution doesn't interfere with other component's routers. The only thing that has to be done too is to add '/index.php' after /api allready in the main .htaccess before the default rules that do that for the CMS (otherwise the routing will go wrong); this is something for the install script to do too.

So you have a simple install along the CMS, but still have a standalone application (that can also be installed "on its own", without the whole CMS). You can have your cake and eat it :-)

[chrisdavenport]

That's similar to what I was thinking. Rather than create a new component at all we could create a package containing a files extension with all the needed files, plus the admin component and the auth plugin. An install script can do any fancy stuff that we might need (such as pulling in default configuration from the CMS) and a pre-flight check can be done to check environment requirements (such as PHP version).

I don't think we can expect any hosts to allow installation of files outside the web root via a web installer. I know my default Ubuntu installation doesn't allow it and I'd be surprised if any correctly configured web server would. Installation outside the web root pretty much requires FTP or SSH access as I understand it.

I don't think it's a good idea to modify the main .htaccess file. That's one reason I would prefer to put the new entry point into a subdirectory.

There is already a Phing build script in the repository. I don't think it works at the moment due to all the changes that have been made, but it could be "fixed" so that it generates the package file automatically. The repo can then continue to have a structure that facilitates those who want to do more advanced installations such installing without the aid of the CMS. As you say, that would give us our cake and we could eat it. Just not sure what it will taste like. :p

[HermanPeeren]

I was into ASP.NET before PHP and writing above webroot is common practice with IIS. I still sometimes use that with PHP (on Windows servers). Sorry, I didn't even know that was not allowed with Linux; thanks for pointing at it. Ayway, conclusion is:

• not to install it above webroot if you want to make use of an install script
• and prevent access with an .htaccess (or similar file for servers that don't use .htaccess).

It is not nice that we would have to modify the main .htaccess file, but I think that is almost unevitable when you want to have an application in a subdirectory that also rewrites urls. Otherwise when the request comes in in the main directory and the url doesn't lead to a file or directory, the rewrite rule in the main directory puts the index.php before the path and the subdirectory won't be reached. But maybe I'm mistaken here? The phprpc-webservice in Joomla 1.5 also was an application in a subdirectory, but didn't use url-rewrites. Just like the administrator-application.

Using api.php, in the subdirectory or in the webroot, doesn't solve anything, for you'll still have to change the .htaccess rewrite rules in the webroot. Of cause you can make a DNS-reference from some api. subdomain to the /api subdirectory, so the application can be reached via api.domainname.ext, but that cannot be done via the installer (and might not be possible with all hosts).

An installable package instead of just a component to install everything in the /api subdirectory looks OK to me. And if at a certain time we would switch to a com_api component that can also be installed with that same package, I always saw the webservices-application in a subdirectory as a temporary solution, to provide the possibility to experiment with new implementations, not bound to the current CMS implementation. This installer thing looks to me like a temporary fix, to use the ease of the installer while keeping the application in a separate subdirectory, but what is wrong with a temporary fix for a temporary solution? So I understand the problem that it might taste a bit odd, but that is only temporarily. And the good thing is: the url doesn't change when the CMS and webservices are united again.

[ronnikc]

Its actually not very accurate.

In CGI/FCGI enviroments you execute as the user (on the Linux system) and not as Apache (or a www/-data) user.

As default you should run with tmp and logging dirs outside the webroot.

A very common approach is /home/user/public_html/

Tmp and logs would often be found in /home/user/tmp and /home/user/logs/

There is nothing to prevent the installation of files into a folder outside the webroot - as long as its withing the scope of the users home directory.

I am sure all hosting solutions today runs like this (or they should be) running apache as mod_php with apache user credentials on execution is not secure at all.

So - you could actually easily deploy files into the users home dir, which would be outside the webroot, but within the scope of the CGI based "jail" where the user operates under and has access to.