Differences between UWSServlet and UWSService ?

Which are the main differences between UWSServlet and UWSService ? They're looking to have the same responsibilities, the main difference is that UWSservice has to be embedded into a Servlet, in opposition to UWSServlet which is a ready-to-use servlet.

They have the same responsibilities, but they look to no sharing the same code base or I'm missing something ?

Originally, only UWSService existed. Then, I decided to go for a more Servlet solution in all these libraries (UWS and TAP) and so, UWSServlet was born. So, yes, there code is duplicated and it is very ugly. To have something cleaner, UWSServlet should use UWSService, but I really like the idea to have a do... function for each type of request in UWS and I wanted something as fast as possible....so duplication with few implementation differences due to the Servlet-way to implement it was needed.

However, when I will finish a new feature, it will never be needed to write yourself a Servlet when using UWSService or even to touch UWSServlet: just a simple text configuration file (a .properties file) will be needed (and a custom class for each type of job). This is already more or less working, but I have to finish testing it as well as adding some other properties (generally coming from the configuration file of TAP in order to have some consistency between both of them).

Ok. For my usage, I've decided to wrap UWSService into a Spring RestControler.

Maybe you could take a look to SpringBoot RestControler because of I want to build a huge test suite for everything I do and Spring is a very clever tool to make this easily, like this kind of test

        @Test
    @DisplayName("Create a job")
    public void createJob() throws Exception {
        //__WHEN__

        RequestBuilder request = post("/uws/MOCK-JOB").accept(APPLICATION_JSON).contentType(APPLICATION_JSON);
        MvcResult result = mvc.perform(request).andReturn();

        //__THEN__
        // code 303
        assertThat( result.getResponse().getStatus() ).isEqualTo( 303 );

        // URL location to the new Job
        Pattern urlWithJobIdAtEnd = Pattern.compile("http://localhost/uws/MOCK-JOB/(\\d)*");
        assertThat(result.getResponse().getHeader("Location")).matches(urlWithJobIdAtEnd);

        // No Body
        assertThat(result.getResponse().getContentAsString()).isEmpty();        
    }

In my humble opinion, you have made a impressive but hard to test work, please take a look to SpringBoot RestControler. With this tool, each Action is a method with a simple and obvious REST contract and easy tests.

Exemple

uws.service.actions.AddJob

@RestControler
@RequestMapping("/uws")
public class UWSControler {

   /**
    * Add Job
    */
   @PostMapping("/{jobList}")
   public void  addJob() {
      ...
   }

   /**
    * Lists Jobs
    */
   @GetMapping("/{jobList}")
   public void  listJobs() {
      ...
   }
}

I don't doubt it, but does not it force library users to also use/develop (and so to know) Spring?

Good question :)

I don't have direct answer to this question, but some observations.

SpringFramework is a kind of de facto standard now in the java's web world with a lot very mature tools.
Do you know who are taplib users ? Maybe could we ask the question through a mailing list ?
You need to do big refactoring and lighten the code. You're in what I name the frankestein syndrome : the code is alive, but can easily going out of control. Taplib is rough to maintain (trust me, I' going through the UWS code for a week, code reviewing is my job), it's scary and architecture could be more readable. IMO, the code improvement is a kind of pain and without a good test coverage a kind of an act of faith.
Framework help you to follow the good patterns : . Separate all the layers : Controlers are for the web, services don't have to know they're dealing with web. . A class <==> a test, so every change that broke something put the red light . Tests are like benefits: you can only add them and profit . Use Separation Of Concern with injection

I think it's possible to mitigate a Spring migration vs a better and readable architecture with good test coverage and more contributor to make taplib alive :)

Ok, it's time to take a coffee.

Do you know who are taplib users ?

No. I am from time to time contacted by users who ask for help, improvement suggestion and/or bug correction, but I have no clear and precise idea of who are the taplib users. Besides, taplib including 3 libraries, so users could be interested by one or all of them.

Maybe could we ask the question through a mailing list ?

I already have got the idea to set up up a mailing list. I just needed time and some professionnal stability for that. But I will surely start filling this gap soon.

SpringFramework is a kind of de facto standard now in the java's web world with a lot very mature tools.

I know that Spring is a fairly well known and used JavaEE framework, but from my experience it is not yet the case in the institutes of astrophysics.....it is starting to rise in there, but surely not everybody is using it. Besides, if I am not wrong, it is possible to import a non-Spring library/code in a Spring "project", but not the contrary. That's why I do not want to go for this or that framework. I do not want to push anybody to use a specific technology (that I would have chosen) because of users preferences but also for time consideration (learning and mastering a new framework or more generally a new way to develop takes time that not every people working in science has....especially if these people are on temporary contract).

You need to do big refactoring and lighten the code. You're in what I name the frankestein syndrome : the code is alive, but can easily going out of control. Taplib is rough to maintain (trust me, I' going through the UWS code for a week, code reviewing is my job), it's scary and architecture could be more readable. IMO, the code improvement is a kind of pain and without a good test coverage a kind of an act of faith.

I started the library in 2009. It was my first professional experience and my really first JavaEE-like library. At first, it was very light: 13 classes. It goes less light and a bit more complex in the next versions in order to provide more genericity and some extension ease.

So, I agree. If I have to start from scratch this library, I would certainly do it in a quite different way. The method to deal with HTTPServlet and the method to deal with threads and queues would be way more different and probably better. When I started, Spring was not that famous and surely not that stable and its API was probably different from the current one. What I try to say, is that, from my point of view, the library does not really suffer from a frankenstein syndrom as you said, but from time and Java evolution and more especially from my lack of experience at that time.

Although, I think it is quite strong to say that the code can easily go out of control. Since it exists, no serious issue came out. Except if you modify the library by yourself, I don't think there would be a lot of code going out of control. Yes, the code is lacking a lot of JUnit (I did not know that testing method when I started), but that does not mean it is that fragile.

The code is scary

Believe me the first versions were way worse and I have already seen much worse in other Java code/libraries.

and the architecture could me more readable.

Sure. That's the problem of a library evolving from a not well designed initial API (excuse my lack of experience at that time).

Framework help you to follow the good patterns : . Separate all the layers : Controlers are for the web, services don't have to know they're dealing with web. . A class <==> a test, so every change that broke something put the red light . Tests are like benefits: you can only add them and profit . Use Separation Of Concern with injection

Trust me, I am nowadays aware of all of that. I am respecting as much as I can all those practices when starting new projects....but taplib and especially uwslib are not that new. It will take time to reach the level of excellence you (and I now) desire. And it can not be done all at once, but step by step considering that the library is already in use by several users for live services.

Already before you contacted me on Github, I have started to draft major upgrades to apply on this whole GitHub project. You have just given me new ideas, so thank to you :+1: .

However, it won't happen now. I first need to work on the implementation of the recent versions of ADQL and TAP standards and I can not do that in the same time as a general API modification. So, I postpone that for later but keep all your suggestions on the pile. See the project UWSLib-5.0.

In the meantime, I will probably release one or two minor revisions of uwslib (and one major for taplib and adqllib) just fixing bugs or adding new features.

One of the first thing I want to finish on UWSLib is the configuration file way to create a UWS service. In this way, library users would have much less code to write and they especially would not have to touch the HTTP layer. Once that is done, I can start modifying the API, because they would not use it that much (only for the creation of a job task) ; so the impact on their implementation would be very minimal if I change the major API. The same would also apply to TAPlib.

Do you know who are taplib users ?

No. I am from time to time contacted by users who ask for help, improvement suggestion and/or bug correction, but I have no clear and precise idea of who are the taplib users. Besides, taplib including 3 libraries, so users could be interested by one or all of them.

I already have got the idea to set up up a mailing list. I just needed time and some professionnal stability for that. But I will surely start filling this gap soon.

I could do this for you, I have a pretty kind of experience with this ("réseaux métiers CNRS").

SpringFramework is a kind of de facto standard now in the java's web world with a lot very mature tools.

I know that Spring is a fairly well known and used JavaEE framework, but from my experience it is not yet the case in the institutes of astrophysics.....it is starting to rise in there, but surely not everybody is using it. Besides, if I am not wrong, it is possible to import a non-Spring library/code in a Spring "project", but not the contrary.

That's not really the situation: Spring has several flavors and packaging: you could only use the core for injecting dependencies, use the database module, or security module or another module. You could (that's what I do) use the Spring Boot Framework, a opiniated framework, to build a fully web application stack in few hours.

That's why I do not want to go for this or that framework. I do not want to push anybody to use a specific technology (that I would have chosen) because of users preferences but also for time consideration (learning and mastering a new framework or more generally a new way to develop takes time that not every people working in science has....especially if these people are on temporary contract).

I know, my main developper is on temporaray contract, but he's so happy to learn and use a business valuable technology he could put on his resume. I understand your position, that was my position for a long time, but ... :)

As responsable of software dev., and as concerned security software dev. (see http://devlog.cnrs.fr/anf-nmsaw2016 ), I push people towards frameworks, and it's an almost official recommandation "never ever try to reinvent the wheel. How could one personne do better than a long life, heavily used, tested and evolving library ?"

I understand, sincerely, what are your feelings : a framework is a time consumer, it's heavy, it's scary, but from my experience it's a maturity and collective step, it respects the "principle of least astonishment" for you, but above all for the developers who use taplib. They would not try to understand what is written, where is it and how you have done it, but what you want to do, just this. If they do not understand a tiny detail, a scratchy convention, google is their friend : Spring is such a currently used framework that all basic, annoying, stupid question has been answered. There's book, blog, forum, stackover flow about Spring all over the internet. There's only you to explain how taplib works, from end to end. By using framework, you would only to explain what taplib does, specifically as IVOA protocol implementation, not as java application of an internet web.

Framework is like a spoken langage, it give you the words to exprim what your want to say, no need to build words from the java alphabet, and this spoken langage is shared by a lot of people.

I stop here, promise, for Spring and frameworks :)

And if you don't want to use Spring, at least use REST library as RestEasy or Jersey

You need to do big refactoring and lighten the code. You're in what I name the frankestein syndrome : the code is alive, but can easily going out of control. Taplib is rough to maintain (trust me, I' going through the UWS code for a week, code reviewing is my job), it's scary and architecture could be more readable. IMO, the code improvement is a kind of pain and without a good test coverage a kind of an act of faith.

I started the library in 2009. It was my first professional experience and my really first JavaEE-like library. At first, it was very light: 13 classes. It goes less light and a bit more complex in the next versions in order to provide more genericity and some extension ease.

So, I agree. If I have to start from scratch this library, I would certainly do it in a quite different way. The method to deal with HTTPServlet and the method to deal with threads and queues would be way more different and probably better. When I started, Spring was not that famous and surely not that stable and its API was probably different from the current one. What I try to say, is that, from my point of view, the library does not really suffer from a frankenstein syndrom as you said, but from time and Java evolution and more especially from my lack of experience at that time.

Although, I think it is quite strong to say that the code can easily go out of control. Since it exists, no serious issue came out. Except if you modify the library by yourself, I don't think there would be a lot of code going out of control. Yes, the code is lacking a lot of JUnit (I did not know that testing method when I started), but that does not mean it is that fragile.

Fragile it's not really what I want to say: by going of control I mean, each modification needs a lot of focusing, concentration and expertise from you. You must have all in your brain, or test heavily a lot of use case with a real server. I was in this situation a long time ago, and I couldn't sleep at night : too many classes, tangles everywhere, each modification was a synonym for "OMG, tomorrow I will spend 8 hours to run and check, modifying files on server, install, remove just to make sure my modification doesn't break something". And I was alone to make this, this application was built out of a framework, all libraries, concept, smart way-of-doing were mine.

When the coverage is well done, the tests running at each file saved, you never go away from the guardrail, it's pretty cool to say :"Ok, let's see what will happen if I do this and this"

Ok, lunch time :)

I agree. You do not have to convince me :)

I just say that it will take a bit of time to move the libraries to such kind of framework. I want to do that step by step, carefuly choose the framework (and be sure of the reasons why I chosed it) and not change everything at once one day when I will decide to.

In the meantime, I improve what already exists in order to prepare an as smoother migration for users (and me) as possible. The configuration file (with JUnit tests as it is already done for this part), code cleaning (especially getting rid of external libraries kept in the internal code rater than being outside + adding JUnit tests for major features) and dependency management improvement are the first steps.

First, I start and finish the implementation of the ADQL-2.1 and TAP-1.1 standards with the current API so that users can update their version of the library and immediately support these new standards and then I will work on the whole API improvement of all the libraries (all together considering their resp. dependencies).

I have been using taplib for a while, and I integrated it into a Spring Boot application, because it makes configuration, packaging, deployment and testing much simpler.

The most useful part for me is the ADQL lib: I have been thinking about implementing TAP in Spring Boot, but never found the time.

The current implementation of taplib is pretty rigid when it comes to the web part. If it was built on top of Spring Boot, I think it would be easier to configure it, and you could get rid of a lot of code that is taking care of that part.

gmantele / vollt

Differences between UWSServlet and UWSService ? #86