Discovery by parsing SCA composite using FraSCAti parser

[mdutoo]

1.

• Start from FraSCAti parser sample provided by pmerle (read-composite.zip)
• in a test, reuse or copy code from existing SCA import so the parser reports discovered services and references to a local Nuxeo (started in the test)
• let it implement the same SCAImport interface as the existing one
• use the parser instead of the existing StAX one in SCA import

2.

• in a test, replace local calls to Discovery API by remote REST calls, while still reusing as much classes as possible
• further work on unifying remote and local Discovery API, so using one or the other is only a matter of using the right implementation, possibly using wrappers over the remote Discovery API so it looks like the local API
• reuse it to develop feature : "Discovery when composite loads up in FraSCAti" by writing a FraSCAti extension, hooking it up at composite load time (ask pmerle), commit (at least a part of) it in FraSCAti

[JGuillemotte]

pmerle code tested OK (displays SCA content). Studied existing SCA import code : difficult to reuse because has too much Nuxeo code in it.

[mdutoo]

About 1. : probably easier to do as a refactoring by someone who already refactored it (e.g. me)

About 2 : to do "replace local calls to Discovery API by remote REST calls", first isolate the bits of code that call Nuxeo APIs, then wrap them in a business (Import) API independent from Nuxeo, then implement it using Nuxeo remote APIs.

This Import business API is part of the EasySOA remote API. LATER see if it overlaps with the Discovery API or other Easy SOA remote APIs, and refactor to make them consistent.

[mdutoo]

In 1., done (see FraSCAtiScaImporter.java) :

• in nuxeo frascati "wrapper" project, added deps so registry-core (including nuxeo and embedded test framework) and frascati parsing (including the four nowadays supported binding extensions - actually, all extensions enriching the parser, such as implementations, should also be added) work
• introduced "SCAImporter" interface common to both impl, started from a copy of the existing parser, merged in it pmerle's read-composite.zip code, using the "wrapper"'s FraSCAtiService
• made it work in an embedded test, impl'ed the WS Service case only for now
• added custom delegate ProcessingContext to get error messages and not to fail on unfound classes
• added "SCA zip" mode (in addition to SCA composite file mode) inspired from SCA contribution, allowing to resolve SCA includes & referenced classes

still TODO :

• other cases (WS Reference, REST Service and Reference) while taking into account the existing visitor architecture and possibly sharing a part of it between both impls if not too complex
• hook it in UI (think about feature first, ex. select upload mode...), build, release
• make the core + FraSCAti test basis usable to the FraSCAti REST project (upcoming).

[JGuillemotte]

Work on registry-core and registry-frascati.

• Common interface for importers :

SCA composites can be imported in two ways : XML sca importer or Frascati SCA importer. these importers works with the same interface : IscaImporter

• Binding info providers for WS and REST :

Binding info providers are specific to way a composite is imported. With a common interface, we have 2 binding info providers for the XML importer and 2 others for the frascati importer. These binding info providers can work with WS or REST bindings.

TODO move impl of ScaImporter.getBindingUrl() in binding info providers

• Visitors for bindings and references :

There are 2 distinct classes to prepare and send data to Nuxeo to register for services and references with a common interface, these classes can be found in the package org.easysoa.sca.visitors in registry-core.

• Nuxeo extension point :

A nuxeo extension point has been added to start the importer automatically. At the moment, The default importer is the Frascati importer. This extension point work with a common ScaImporterComponent.

[JGuillemotte]

Deployment and problems

• If Nuxeo doesn't start :

Be sure that the jar's concerning the registry components are well copied in the directory '.../easysoa-distribution/easysoa/serviceRegistry/nxserver/plugins' and ther is not older version of these jar's in this directory.

• Too many files open error :

On a Linux system, edit with root profile the file '/etc/security/limits.conf'. Add this two lines (* included) :

-    soft    nofile    50000```
  Restart your session (or reboot the computer), the problem will be solved.
- NullPointerException throws by HtmlSanitizerServiceImpl class on lilne 209 :

This error is thrown when a SCA file is imported, however it does not block the import in any way. The issue is known and will be fixed in the Nuxeo 5.4.3 release, see [NXP-7149](https://jira.nuxeo.com/browse/NXP-7149?page=com.atlassian.jira.plugin.system.issuetabpanels%3Achangehistory-tabpanel) and [HtmlSanitizerServiceImpl.java](https://fisheye.nuxeo.com/browse/nuxeo-services/nuxeo-platform-htmlsanitizer/src/main/java/org/nuxeo/ecm/platform/htmlsanitizer/HtmlSanitizerServiceImpl.java?r=df359aba6e4fc97d574cef8f80e0c04bb50b577a&r=61d43a514c2d3c8ebc10ed3dccf42d0f54ee3706&r=30881edf197ff930c6e4cc30b49f70c03d2202ce). The cause of this error is when no filter is specified for a field in the contribution file associated to the sanitizer, see [contribution](http://explorer.nuxeo.org/nuxeo/site/distribution/current/viewContribution/org.nuxeo.ecm.platform.htmlsanitizer.config--sanitizer).
- SCA Importer problem :

The importer works well with a standalone composite file.

In the case of a zip or jar file : the jar containing the HttpDiscoveryProxy is imported without problems. In the same way, an other test made on PureAirFlower sample jar works.
Contrariwise, there is a problem with the Smart travel exemple jar.  A 'NoClassDefFoundError' is generated, because required classes (such as net.webservicex.GlobalWeatherSoap.java) are not in the imported jar or zip but in an other jar or zip file it depends on (such as produced by  the *-meteo-model project) This error stops the import, since we've configured the FraSCAti parser to expect that a zip or jar will be "complete" in that it provides everything it references, whereas SmartTravel is an app with maven dependencies and therefore not "complete".

This impacts the app deployment and development model :
- either we use the SCA application model, whose classloader isolation ensures its zip / jar is complete (besides generic libraries). In this case we have to package in its zip all referenced classes and composites :
  - generic libraries (classes and composites) have to be put in the generic lib/ classloader
  - referenced business classes (ex. interfaces of services consumed) have to be put in the zip, possibly on the fly
  - and the same for "template business" classes and composites. If it were to scale up it would be bad (a lot of identical classes in memory), but given EasySOA Light's prototyping goals it is rather a safe choice. However that also means there is a clear distinction between developing an business app and developing a template app i.e. developing EasySOA itself.
- or we use another model which allows for dependencies (OSGi, maven ??). In this case, those dependencies have to be configured, managed and maintained in the context of a truly modular application - which is far harder, so why not for "traditional" application development (EF, OWT), but not for EasySOA Light.

[mdutoo]

Now on to 2. :

2.1. hook in frascati start : AssemblyFactoryManager ; for now by DiscoveryProcessingContext.getData(Composite, Component), TODO LATER by adding a delegate either provided by us or allowing us to plug in in a pubsub / listener way (ask pmerle) 2.2. discovery impl : reuse FraSCAtiSCAImporter.visit(Composite) (in Api...) by sharing it in a common abstract class (...Base), reimpl both ScaVisitors and their ...Base on top of a nuxeo-free API, like : 2.3. easysoa-registry-api's RestNotificationFactory : impl BindingVisitors on top of it 2.4. LATER (mka) refactor to unify with (I)NotficationService and improve names : notification => discovery... 2.5 LATER start / stop (rather than composite) : from Fractal membrane interceptors, or vice versa on remote introspection / web explorer

[mdutoo]

2 done (using BindingInfoProvider).

Still remains to do :

• maybe some more cleaning on API side ?
• feature in UI : * import SCA ZIP is OK but still to be documented, * design & develop start / stop in Nuxeo (in Web Explorer ? in FraSCAti Studio ? in dedicated UI ? at least after 21 FraSCAti in Nuxeo)

[mkalam-alami]

Works since 1d4dbf1a5, closing.