json-schema-validator as a ready to use OSGi bundle ?

[boly38]

Hi all, I was trying to integrate json-schema-validator as an OSGi bundle and I took so many time to do that. I create a post to explain this : [1].

Please could you check again the OSGi compatibility and tell me if I'm wrong.

[1] http://curiositedevie.blogspot.fr/2014/08/use-json-schema-validator-as-osgi-bundle.html

[aborg0]

I have asked the jopt-simple project maintainer (https://github.com/pholser/jopt-simple/issues/65) to make its builds OSGi bundles, so probably from version 1.8 that will not be a problem. The libphonenumber project already had an issue (https://code.google.com/p/libphonenumber/issues/detail?id=205) for the OSGi support, hopefully that will also be supported soon (probably it worth adding a vote for it). I do not see the org.mozilla.javascript package among the imported packages, so probably that is not required to be modified. The

java.io.IOException: resource /draftv4/schema not found
       at com.github.fge.jackson.JsonLoader.fromResource(JsonLoader.java:91)

problem hopefully be resolved by this issue.

[boly38]

hi firsrt of all thanks @aborg0 for this feedback +1:

• jopt seems to be fixed,
• I just voted for libphonenumber#205

and for the "schema not found" issue: I think this is related to json-schema-core project as draftv4/schema is included into the resource.

the problem hopefully be resolved by this issue.

sorry not sure to understand but what do you mean by "this issue" ?

regards

[aborg0]

I thought you created this issue (#111) in order to get a solution to the resource not found problem,. (Using maven, a configuration is described here to add resources. Probably there is analogous for Gradle too.) Hmm. It seems it seems to be included in the jar. Probably classloading problem? I am going to try this out soon. Thanks.

[aborg0]

Regarding the missing resources: I am not sure for the reason, but it seems the https://github.com/fge/json-schema-validator/tree/master/src/main/resources/draftv4 folder misses the hyper-schema file. (Though it is in https://github.com/fge/json-schema-core/tree/master/src/main/resources/draftv4, so changing the contextclassloader to the com.github.fge.json-schema-core bundle's classloader (during JsonSchemaFactory.byDefault()) solved this problem for me.) Is this the supposed workflow? (It would be nice to also have OSGified services to get the required instances. Probably that was the purpose of this (#111) issue and I just misunderstood.)

[boly38]

@aborg0

yep, fixing "resource not found" pb should fix this issue. (a ready to use OSGi service would be perfect ^^). Sorry for the confusion, this is my fault. My english understanding is a little bit experimental :)

so changing the contextclassloader to the com.github.fge.json-schema-core bundle's classloader (during JsonSchemaFactory.byDefault() solved this problem for me.)

I make some time to switch again on this context and I'm not sure to understand. Could you tell me how you do that ? You patch the json-schema-core implementation ?

As Json validator end-user I used this to get a factory

final JsonSchemaFactory factory = JsonSchemaFactory.byDefault();

Of course my side I use json-validator from a bundle B wich is different from json-schema-core bundle... do you mean I could workaround this "byDefault()" method ?

[aborg0]

@boly38 I have to admit I do not prefer this solution -credits to Marcel Hanser whom I heard from-, but it works:

  //find the bundle you need:
  Bundle selected = null;
  for (Bundle bundle: bundleCtx/**/.getBundles()) {
     if ("com.github.fge.json-schema-core".equals(bundle.getSymbolicName()) selected = bundle;
  }
  ClassLoader specialClassLoader = selected.adapt(BundleWiring.class).getClassLoader();
  ....
  ClassLoader cl = Thread.currentThread().getContextClassLoader();
  try {
    Thread.currentThread().setContextClassLoader(specialClassLoader);
    final JsonSchemaFactory factory = JsonSchemaFactory.byDefault();
    //Do something with the factory.
  } finally {Thread.currentThread().setContextClassLoader(cl);}

So it does not require changes in the json schema validator code, though having OSGi services would make this a lot nicer.

[boly38]

Thanks @aborg0 I will test this tomorrow and give you a feedback. As second way, I would like to try too to remove draftV4 resources from json-schema-core as Cuong Tai do here..

[boly38]

Hi

here are my test results :

• test 1: removing "draftv3/|draftv4/" resources from json-schema-core jar file (src: Cuong Tai) doesn't change anything for me : not found error : KO
• test 2: Trying to translate your workaround on my OSGi framework (Prosyst 7.5). Out of error, Prosyst framework is unable to get classloader using bundle.adapt (following OSGi R5). The following line is not available

   ClassLoader specialClassLoader = selected.adapt(BundleWiring.class).getClassLoader();

I could change this line to

   ClassLoader specialClassLoader = selected.getBundleContext().getBundle().getClass().getClassLoader();

but I always got the same error.

• test 3: using setClassLoader() and local resources If I keep the setClassLoader() and if I put "draftv3/|draftv4/" resources on the OSGi framework binary directory, hyper-schema is found. This means that, in this case, json-bundle resources are not visible but only local resources. This solution works for me but it's realy dirty.
• test 4: using setClassLoader() with a custom classloader I've used the bundle("com.github.fge.json-schema-core")'s classloader to implement my own one src. My change was to override the "getResource()" behavior. cf. code under This solution is not so clean but this one avoid to copy or extract dependencies resources on my local OSGi Framework resource directory.

Hope this helps If somebody found a better fix, please reply. Thanks again to you @aborg0 ^^

    public class MyClassLoader extends ClassLoader {
        Bundle b;
        public MyClassLoader() {
          super(MyClassLoader.class.getClassLoader());
        }
        public MyClassLoader(ClassLoader parent) {
          super(parent);
        }
        public void setBundle(Bundle b) {
            this.b = b;
        }
        @Override
        public Class<?> loadClass(String name) throws ClassNotFoundException {
          return super.loadClass(name);
        }//loadClass
        @Override
        public synchronized Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
          return super.loadClass(name, resolve);
        }
        @Override
        protected Class<?> findClass(String name) throws ClassNotFoundException {
          return super.findClass(name);
        }
        @Override
        protected URL findResource(String name) {
          return super.findResource(name);
        }
        @Override
        protected Enumeration<URL> findResources(String name) throws IOException {
          return super.findResources(name);
        }
        @Override
        protected Package getPackage(String name) {
          return super.getPackage(name);
        }
        @Override
        public URL getResource(String name) {
            if (b != null) {
                return b.getResource(name);
            }
            return super.getResource(name);
        }
        @Override
        public InputStream getResourceAsStream(String name) {
          return super.getResourceAsStream(name);
        }
        @Override
        public Enumeration<URL> getResources(String name) throws IOException {
          return super.getResources(name);
        }
      }//MyClassLoader

    private JsonSchemaFactory getJsonSchemaFactory() {
        // find the bundle you need:
        String targetBundleSN = "com.github.fge.json-schema-core"; // "javax.mail.api"; // com.github.fge.json-schema-validator";
        Bundle selected = findBundleBySymbolicName(targetBundleSN);
        // Bundle selected = findFirstNotNullBundle();
        if (selected == null) {
            LOG.warn("unable to get bundle ");
            return JsonSchemaFactory.byDefault();
        }
        // DO NOT COMPILE // ClassLoader specialClassLoader = selected.adapt(BundleWiring.class).getClassLoader();
        ClassLoader specialClassLoader = selected.getBundleContext().getBundle().getClass().getClassLoader();
        if (specialClassLoader == null) {
            LOG.warn("specialClassLoader is null");
        }
        try {
            URL resUrl = selected.getBundleContext().getBundle().getResource("/draftv4/schema");
            LOG.info("bcontext get draftv4 resource ok :" + (resUrl != null ? resUrl.toString() : "(null)"));
        }catch (Throwable t) {
            LOG.info("bcontext get draftv4 resource KO : " + t.getMessage(), t);
        }

        ClassLoader cl = Thread.currentThread().getContextClassLoader();
        try {
            // Thread.currentThread().setContextClassLoader(specialClassLoader);
            MyClassLoader loaderToUse = new MyClassLoader(specialClassLoader);
            loaderToUse.setBundle(selected.getBundleContext().getBundle());
            Thread.currentThread().setContextClassLoader(loaderToUse);
            final JsonSchemaFactory factory = JsonSchemaFactory.byDefault();
            // Do something with the factory.
            return factory;
        } finally {
            Thread.currentThread().setContextClassLoader(cl);
        }
    }

    private Bundle findFirstNotNullBundle() {
        Bundle selected = null;
        Bundle[] osgiBundles = bcontext != null ? bcontext.getBundles() : null;
        if (osgiBundles == null) {
            LOG.info("unable to get context bundles");
            return null;
        }
        for (Bundle bundle : osgiBundles) {
            String bundleSN = bundle.getSymbolicName();
            if (bundleSN != null) {
                selected = bundle;
                break;
            }
        }
        if (selected != null) LOG.debug("bundle *found* => " + selected.getSymbolicName());
        return selected;
    }

    private Bundle findBundleBySymbolicName(String targetBundleSN) {
        Bundle selected = null;
        Bundle[] osgiBundles = bcontext != null ? bcontext.getBundles() : null;
        if (osgiBundles == null) {
            LOG.info("unable to get context bundles");
            return null;
        }
        for (Bundle bundle : osgiBundles) {
            String bundleSN = bundle.getSymbolicName();
            if (targetBundleSN.equals(bundleSN)) {
                selected = bundle;
                break;
            }
            // LOG.debug("bundle => " + bundleSN);
        }
        if (selected != null) LOG.debug("bundle *found* => " + selected.getSymbolicName());
        return selected;
    }

[boly38]

I just push a first draft of this : bundl'ized version of json-schema-validator

https://github.com/boly38/json-schema-validator-bundle

[boly38]

for rhino deps There is a MANIFEST.MF update required too : I think that related issue is github.com/mozilla/rhino #86

[boly38]

Following json-schema-validator-bundle creation and from my point of view, this issue could be closed. ^^

[fge]

@boly38 thanks a lot for this work, I'll have a look at it; to be honest, OSGi has never been a priority for me and I haven't researched the subject enough to come with a viable solution...

I'll try and understand your work; ideally, I want this to be part of the main distribution although I cannot figure out how to do this yet...

[aborg0]

@fge @boly38 I think the ideal case would be to include OSGi services as well as SPIs* (and not using classes getResource from other bundles). That way we would be sure that there are no classloading problems. (My colleagues reported that they had problems with org.mozilla.javascript's Rhino latest version, so we use the 1.7.2 version from eclipse orbit. Fortunately as it is not a bundle on maven central the restriction on the maven version is not present for this in the schema validator bundle.) Thanks for checking. I am sure @boly38 did a great job to make things work, although I think that way we lost a significant amount of modularity (it is hard to replace the inlined bundles). Unfortunately I'll be too busy for the rest of this year to contribute for a better solution (which imho would involve declaring OSGi services).

*: For OSGi R5 aries SPI fly is also an option, but we are currently on OSGi R4.

[fge]

OK, the thing is that I am not very knowledgeable with OSGi; I cannot fully understand the problems at hand, and even if I do I wouldn't know how to fix them properly in the first place.

I would need guidance on how to approach the problems: what they are exactly, how to fix them; and one step at a time.

More particularly, I would appreciate a way to "unit test" correct OSGi usage. @boly38 @aborg0 is this doable?

[boly38]

@fge @aborg0 I understand your need to get a way to "unit test" correct OSGi usage. I used OSGi since 7 months only, on my project I make some unit tests out of OSGi box, and high level integration tests with my OSGi (non free) platform [impossible to share here].

I never look in detail how to do some kind of "OSGi layer only" integration tests.

Maybe by using OSGi framework like equinox or karaf it's possible to do that with a minimal effort but in the next week I have no time to spend on this. Sorry.

(need to be confirmed by an OSGi expert..) any way OSGi fwk let the way to use different versions of the same package. For example, if you would like to use json-validator bundle and one of the dependency (eg. jackson; inlined or not) with a different version for your need, then you could also make the good osgi import package definition in your own bundle to do that. As CL are separated you wont get class loading issue.

[aborg0]

@fge I think these kind of integration tests are best done by creating a new test project (depending on the original) and execute the tests there using an OSGi container like equinox, knopflerfish, felix or something else. (You need to make all dependent bundles available for the framework, I think the tycho project might help in this regard.)