Start reworking things into an Standalone Application

[wilsonge]

Implements a rudimentary J! Framework based application structure and namespaces all the library with some service providers. As a result removes the system plugin. Note this isn't in a functional state yet.

This also removes the getInstance method from JApi and moves it to be built in the application and injects the container into the JApi object because it depends on many things including the text, language and application objects.

Large Item Todos:

• [ ] Refactor away from Legacy MVC to some sort of new MVC
• [ ] Decide if we want a JTable replacement or whether we want to drop everything into models (me and @mbabker have discussed this a few times but nothing really ever came out of it)
• [x] Implement a session for enqueing messages into the application. Joomla's session v1 is pretty much unusable. v2 is got finished by @mbabker today: https://github.com/joomla-framework/session/pull/12 but isn't merged yet so I didn't bother
• [ ] Routing (because I couldn't work out at a glance how the current routing system really worked)
• [ ] Work out how the component is going to interact with this app (99% of the CMS classes are not going to be injectable into this thing)
• [ ] Work out how the app interacts with the frontend and backend of Joomla (things like JApplicationCms::isAdmin or the JPATH_SITE/JPATH_ADMIN constants
• [ ] JLayouts don't exist on the framework level so decide if we want to port them into our app or use something like Twig, Plates, Blade or Moustache through the framework Renderer (currently I have ported JLayouts over)
• [ ] Deal with replacing JDocument (cause it's never gonna exist in the framework cause it's a CMS weak link as it is) - as a temporary measure I've ported a simplified version of it into the app structure
• [ ] Structure of the app - currently I only have media + index.php in the root. We might want to rework this in the future as a lot of cheap hosts don't allow this (which is why Joomla is installed by default totally into the root)

Specific Todos:

• [ ] JUri::base() has a dependency on the Joomla config object. When this is now a service provider orientated object suddenly things become really hard really fast because you've got to pass either the configuration or Container in and things don't work. Michael in his Joomla Next! Repo ended up adding in a setBase and setRoot methods (https://github.com/mbabker/joomla-next/blob/master/libraries/core/Uri/Uri.php#L123) - but it doesn't seem ideal either. For now I've removed that section and just used the autodetection but it needs to be fixed before we put things live.
• [ ] Do we want an interface for changing the application config (currently things that were in the params of the system plugin

[wilsonge]

Updated to add in the dev session branch and an implementation

[Kixo]

I have checked the code and it seems as a huge step, Very Good Job @wilsonge TBH I did not tested it yet as it lacks JLayouts, JDocument and some other classes that will not work as it is now We need to think of a deploy plan with this as well... because my phing packager will not work against this PR

[wilsonge]

@javigomez is currently investigating some packaging alternatives for the CMS and 3rd party components (e.g. phing/gulp etc.) so we should make sure we use the same things as them ideally so we are standardized across the projects

[fastslack]

@wilsonge Is there any specified reason to put index.php on www root? I think that put all the application in /api directory could help to simplify lot of things. One reason is to have all independent of the CMS, for example to create a new routing for this. Another good reason is to have clean URL to request resources.

[wilsonge]

That's just the structure we're using for Framework apps. You can only have the entry point and the media inside the web root. For example see https://github.com/joomla-extensions/jstats-server

[fastslack]

Cool i see. What do you think that is the best way to test the webservices application? I mean using the app with CMS to do some initial http request.

[wilsonge]

I'm not sure how the interaction with the CMS is going to work at the moment in all honesty. All the component configuration views are going to have to come from the application (because things like the language package and event package are incompatible with CMS versions). We also have issues that JPATH_ROOT is still required in some areas of the framework and that's also needed for defining JPATH_SITE, JPATH_ADMINISTRATOR etc. in the CMS. So how we're going to get data out the CMS is going to be a real challenge

[fastslack]

I remember to have the same issue with JPATH_ROOT using cli apps. I 'solved' the issue defining JPATH_SITE as www root and let JPATH_ROOT to cli directory. Example:

https://github.com/fastslack/jUpgradePro/blob/master/cli/bootstrap.php-dist#L15

This is an old solution. I'm not sure that adding in a setBase and setRoot methods could help neither. Maybe adding new const like JPATH_API or JPATH_CLI but im not sure.

For that reason is important to define the api in a separate directory, so we can have a bootstrap.php and define the required constant for the api only.

[wilsonge]

Yes but you are defining a JPATH_ROOT for the first time in there. We already have a JPATH_ROOT that is defined for the framework app - nothing to do with the CMS at all.

[fastslack]

Hmmm the only solution i found for this is to replace JPATH_ROOT for JPATH_FRAMEWORK :-| I'm sure @mbabker don't like this idea :-D

[wilsonge]

I mean the filesystem thing is something that we are going to rework anyway - likely it's going to be a wrapper for flysystem. So that removes the need for JPATH_ROOT to be defined (however it depends what functionality gets dropped during that process anyhow)

[wilsonge]

OK So I've added a clone of JDocument but stripped a bunch of stuff out and allowed injection of application and the container object. That allows us to have a basic rendering process in place. Although it should definitely be changed to a better renderer in the final version - JDocument doesn't really cut it.

Also PHPStorm kept throwing errors at me in the namespace Document so I changed the namespace (not the class) to Renderer rather than Document

[gunjanpatel]

@wilsonge

@javigomez is currently investigating some packaging alternatives for the CMS and 3rd party components (e.g. phing/gulp etc.) so we should make sure we use the same things as them ideally so we are standardized across the projects

Can we use something like this? https://www.npmjs.com/package/joomla-gulp-release I can help. :)

[javigomez]

Awesome @gunjanpatel. I'm adding you to the working group. If anyone else wants to volunteer please contact me.

Regarding this:

JLayouts don't exist on the framework level so decide if we want to port them into our app or use something like Twig, Plates, Blade or Moustache through the framework Renderer

I may be wrong but as far as I understood from here: https://github.com/joomla-projects/webservices/issues/4#issuecomment-111760398, the api will have not any UX. If there is any UX like the webservice editor, or the authentication options that goes into a Joomla CMS component. Isn't it?

[Kixo]

we use JLayouts for autogenerating documentation and for the layouts in admin extension

[chrisdavenport]

The admin extension will continue to be a CMS 3.x extension. The auto-generated documentation needs to be in the form of (primarily) machine-readable profiles, such as ALPS or JSON-LD contexts. Additional human-readable documentation can be hosted elsewhere (developer.joomla.org or docs.joomla.org or even GitHub for the core-supported components; some other domain for third-party extensions) and linked to from the profiles. So Javier is correct, there is no requirement for a UX in the API application itself.

[wilsonge]

The 3.x component is still going to need to query the server for all that documentation though and be in charge of rendering it (FWIW all the html is being generated from the layouts https://github.com/joomla-projects/webservices/blob/develop/libraries/webservices/layouts/webservice/documentation.php)

[wilsonge]

OK so the base port to a framework app is done - you can test this now and you will see that you get a HAL and SOAP response (with the appropriate config. Note you will need to set up and populate the db table yourself).

The question now is how you all want to interact with the CMS and legacy MVC. On this the ball is completely in your court cause there's a lot of different hacks around it and I'm not going to try and second guess which one you guys wanna do.

[chrisdavenport]

Can we have some basic instructions on how to get this working? Thanks. :-)

[wilsonge]

You need to manually set up a database and create schemas (as the component doesn't work at this time - it needs a full rewrite).

Then set up a config file (https://github.com/wilsonge/webservices/blob/app/config.json) with the correct values - this is the equivalent of the plugin settings currently.

The urls are the same as those used currently.

Make sure you are using the branch at https://github.com/wilsonge/webservices/tree/split which has proper CMS integration gives you properly valid results (remember to correctly point your path to the CMS at https://github.com/wilsonge/webservices/blob/split/www/index.php#L13

[fastslack]

I'm trying to test it and following yours instructions i got:

{"message":"The \"JPATH_ROOT\" constant must be defined for your application.","code":0,"type":"LogicException"}

The URI that i'm testing is _index.php?option=contact&api=Hal_. Thoughts?

[mbabker]

That comes from the Framework's Filesystem Package. Looking at George's code, he isn't using a JPATH_ROOT constant anywhere and there are still a couple of places in the Framework code that need to be refactored to not use that constant.

[fastslack]

Yes, i saw that, but when i change the constant an empty JSON is returned []. The Framework will be refactored to use what constant instead JPATH_ROOT? What do you think could be the best way to integrate this to CMS?

[wilsonge]

Yah I realised I had commented that line out locally. Eventually we are going to refactor the framework to wrap around flysystem but @HermanPeeren volunteered to do that. You're going to need JPATH_ROOT for the CMS so just comment out the logic exception for now. We don't actually need it as we are only calling Path::find (it's needed if you don't specify a path to that method - but for the purpose of testing this app we will always have a path).

Is your path to the CMS at the right place? Have you added the contact API to the database? Have you set the database correctly?

Also are you sending the correct authentication headers for the CMS? (see https://github.com/wilsonge/webservices/blob/split/src/Integrations/Joomla/Strategy/Joomla.php#L39-L42) as if you aren't then you will return an empty JSON string

[wilsonge]

Btw if you want to cheat and go via a browser for testing purposes then just hardcode in your credentials for the CMS in beneath the line I just linked to

[fastslack]

Seems that authentication headers is the key, i'm writing a cli script to test it using JHttp. I ll test a little more and back :)

[wilsonge]

Closing this in favour of the updated repo that I've been linking to recently in PR #18