OpenLiberty / docs

See Open Liberty documentation on https://openliberty.io/docs/
https://openliberty.io/docs/
Other
13 stars 47 forks source link

Testing in a container #953

Closed Charlotte-Holt closed 4 years ago

Charlotte-Holt commented 4 years ago

New topic based on the MicroShed work on how to test in a container. See the resources below. Adapt from the RHR document to add more useful information. It will be remote included in the RHR docs.

From LC: This topic needs to be more of an intro/concept to integration testing with OL - why, what do you achieve, what do you mean by 'integration testing' (as opposed to other types of testing), and what are the challenges that MicroShed resolves? What does MicroShed provide, and what alternatives are there (if any)? Explain how it works with the code snippets but less of a task/guide and more explanatory. Then link to the new(-ish) guide https://openliberty.io/guides/microshed-testing.html for help actually doing it.

Double-check with Alasdair about whether we should also link to Arquillian still.

Contact: Andy Guibert

Resources:

dmuelle commented 4 years ago

Peer review

Define/introduce Testcontainer ?

change "With MicroShed testing you test your application in the same container that you use in production, thus reducing the risk of test failure."

to With MicroShed testing, you reduce the risk of test failure by testing your application in the same container that you use in production.

Why integration testing - possibly reorganize this section? change In integration testing, you can put together individual pre-tested modules and test them as a group. Integration testing follows unit testing. The modules used are previously unit tested, but now they are now tested as a whole to evaluate compliance and functionality.

to Integration testing follows unit testing. After individual modules are unit tested, integration testing looks at the modules together to evaluate their performance as a group.

"You can address this issue by testing your application in the same container that you’ll use in production."

watch for future tense- "You can address this issue by testing your application in the same container that you use in production."

don't use dev/test - from IBM Terminology app dev/test (noun) Usage Do not use "dev/test" unless it is part of the established terminology and is spelled out as "development and testing (dev/test)" on the first occurrence.

nonrestrictive clause needs a comma "test against a running Open Liberty server in non-development scenarios{,} such as CI/CD" pipelines."

"...testing, first add the microshed-testing-testcontainers and junit-jupiter artifact IDs"

This sentence is in a codeblock but doesn't need to be...

In cases where a Dockerfile or container image is not available, use the microshed-testing-liberty

lauracowen commented 4 years ago

Thanks for this draft. I like the general approach of this topic. I think it just needs more detail in the first sections. Also, the section "How it works" doesn't really explain how it works but how to use it.

With regards to what AndyG said about having the guide and the microshed.org website already, that doesn't negate the need for us to have a topic like the one you're writing in the main OL docs. There needs to be something that introduces the concept of integration testing: what it is and why it's important, then introduce the problems that can arise that are then solved by using microshed. Then describe what microshed is and how it does what it does (eg containers etc).

We need this in order to give context to what MicroShed is for (otherwise we'd just be pushing people to a guide or site with no context about why they should care). Also, we need to have this intro in the RHR docs to again give context before pointing them off to the guide and site for worked examples and more detail.

Can you talk to Andy to get a bit more about what integration testing is and why it's important. Can you also check the definition that you have already. Also, ask him if there should also be a discussion of Arquillian and how it fits into this, if he feels it does (when you'd use Arquillian and when you'd use MicroShed - the problem each addresses and how).

Some points to include (if Andy agrees they're correct!):

I think you should remove from this topic the info from "To write a test…" onwards - the topic shouldn't be trying to tell the reader how to use MicroShed. Keep the topic as a conceptual introduction. However, I think that can include a code snippet that illustrates how it works. So you could (unless Andy has a better idea) include the first two code examples (and explanations) from the "What it looks like" section of https://microshed.org/microshed-testing/ but don't include the rest of the section after the second code example - that's too much detail for here. Then remove the code comments from the code example and instead use them in an explanation after the second code example to briefly explain what the integration test (second code snippet) is testing for and how it does it. ie move the code comments out of the code example and into the text of the topic.

If you like, you can use the 'call-out' capability of asciidoc to add markers in the code example so that it's easier to refer to specific lines from the text (https://asciidoctor.org/docs/user-manual/#callouts - like Charlotte's used in the first code example here: https://github.com/OpenLiberty/docs/blob/master/ref/config/serverConfiguration.adoc, which you can see here https://openliberty.io/docs/ref/config/), but you don't have to if it works just to describe it in ordinary text.

Then provide a link to the MicroShed Testing guide to learn more how to use it. And a link to the MicroShed Testing website.

ManasiGandhi commented 4 years ago

Draft link https://draft-openlibertyio.mybluemix.net/docs/ref/general/#microshed.html

lauracowen commented 4 years ago
ManasiGandhi commented 4 years ago

Andrew's comment "so the section "Why integration testing" is still not very convincing -- it's giving more reasons why NOT to use integration testing than it is supporting integration testing" I worked on it

ManasiGandhi commented 4 years ago

@lauracowen Here is the link to the microshed doc after SME review. Can you review it please? https://draft-openlibertyio.mybluemix.net/docs/ref/general/#microshed.html

ManasiGandhi commented 4 years ago

@chirp1 Here is the link to the microshed doc after SME review. Can you review it please? https://draft-openlibertyio.mybluemix.net/docs/ref/general/#microshed.html

ManasiGandhi commented 4 years ago

@dmuelle Can you do a peer review of this Microshed doc draft https://draft-openlibertyio.mybluemix.net/docs/ref/general/#microshed.html

chirp1 commented 4 years ago

@Manasi @dmuelle Hi Manasi, I took a quick look at your topic. Before you have David review, would you go carefully through your topic and make additional changes? You can follow directions in various places, including, but not limited to, these places:

Also, rerun Acrolinx and fix various issues that the tool mentions. Some things you can leave as is such as Java, twelve-factor, dev/prod, methodology, /myservice, NotFoundException.class, and ->. Fix the rest. In some cases, you might need to talk with the developer as to new wording. For example, for the word invoke, you might replace it with different words, depending on the occurrence.

dmuelle commented 4 years ago

peer review

5/1

intro

MicroShed Testing framework analyzes your application from outside the Docker container without accessing the application internals. I don't think you need "framework" iAfter Microshed testing. It seems awkward and also a little confusing since in the previous sentence, you identify microshed as a java library, resulting in two different definitions in back-to-back sentences.

Integration tests are more comprehensive than unit tests. However, integration tests can be more complicated and take longer to run.

I think this sentence should be in the next section. Instead you could have a brief paragraph that explains the benefit of testing from outside the docker container, ie what problem microshed solves.

Why integration testing

Integration tests assess multiple test classes and components, but take longer and can be complicated to set up and configure. You explain what integration tests are, but not unit tests, so the comparison here is not clear.

With shorter development cycles and limited time available, developers often run unit tests because they are easier and faster. instead of "easier", which acrolinx flags, maybe "less complex and faster to run than integration tests" ?

MicroShed Testing framework streamlines your integration tests with the Testcontainers library for a better workflow. In the shortdesc, Microshed isn a library and Testconainers is a framework. Here, they are reversed. I don't think you need framework here, but be consistent in how you refer to testcontainers. Also- "better" is unclear- better how?, maybe "more efficient" ?

Development-production parity

The application might work on your computer but not in production. I don't think this sentence adds anything that's not being said by the sentences before and after it, I'd recommend cutting it

Using MicroShed Testing

"Using" is indistinct- "Writing integration tests with Microshed Testing" might be more specific?

I think the introductory paragraph here can be reworked to explain how Microshed implements testcontainers to support dev/prod parity in testing. As is, it mainly talks about benefits of containers, which is not the main idea of this section. Avoid using "leverage" as a verb.

I think the code example should be a single block, with the description in a paragraph right after. You might have to rework the description a bit to get it to read coherently, but not much.

Make sure to put the text references to annotations (@RESTClient etc) in monospace

Manasi commented 4 years ago

Tagging @ManasiGandhi

ManasiGandhi commented 4 years ago

Andrew's review comments in Slack "hi Manasi, there is a broken link in the "Development-production partiy" section here: ... factors in the link:https://12factor.net/ [twelve-factor app], a methodology ... but other than that it looks good now"

ManasiGandhi commented 4 years ago

@lauracowen Can you review this topic?

https://draft-openlibertyio.mybluemix.net/docs/ref/general/#microshed.html

lauracowen commented 4 years ago

That's great! Thank you. :)

A couple of minor points but otherwise it's almost there:

Let me know when you're done and I'll re-review it quickly. Thanks

lauracowen commented 4 years ago
aguibert commented 4 years ago

yes that sounds reasonable @lauracowen, but for the code example we're going to show I think we need to give a bit more context than just the test method itself. Part of the value of Microshed Testing is the simplicity of setup, and if we just show the reader the test method we're missing out on an opportunity to say "see, this is all it takes"

For the code example I would suggest the code sample from https://microshed.org/microshed-testing/

@MicroShedTest
public class BasicJAXRSServiceTest {

    @Container
    public static ApplicationContainer app = new ApplicationContainer()
                    .withAppContextRoot("/myservice");

    @RESTClient
    public static PersonService personSvc;

    @Test
    public void testGetPerson() {
        // This invokes an HTTP POST request to the running container, which triggers
        // the PersonService#createPerson endpoint and returns the generated ID
        Long bobId = personSvc.createPerson("Bob", 24);

        // Using the generated ID, invoke an HTTP GET request to read the record we just created
        // The JSON response will be automatically converted to a 'Person' object using JSON-B 
        Person bob = personSvc.getPerson(bobId);

        assertEquals("Bob", bob.name);
        assertEquals(24, bob.age);
        assertNotNull(bob.id);
    }
}
lauracowen commented 4 years ago

@aguibert Great, thanks. That's helpful. @ManasiGandhi Okay, what I said in the comment above but use the code example that Andy gives in his comment instead of the one I provided. However, still remove the comments etc (but keep the code example as a single code snippet and put explanation below the whole thing).

lauracowen commented 4 years ago

Thanks. Can you remove the comments from the code snippet (so that the code snippet takes up fewer lines)? Then I think we're done.

ManasiGandhi commented 4 years ago

@lauracowen Removed the comments https://draft-openlibertyio.mybluemix.net/docs/ref/general/#microshed.html

lauracowen commented 4 years ago

Great, thanks. I'm going to sign it off now.

chirp1 commented 4 years ago

Hi Mansi,

If you have any questions, let me know.

ManasiGandhi commented 4 years ago

@chirp1 updated draft link https://draft-openlibertyio.mybluemix.net/docs/20.0.0.9/integration-testing.html

chirp1 commented 4 years ago

Hi Manasi,

I think you did a nice job on the topic! I see a various things that could use a little sprucing:

ManasiGandhi commented 4 years ago

@chirp1 Thanks for your review comments. I worked on your edit suggestions. You can view the changes here https://draft-openlibertyio.mybluemix.net/docs/20.0.0.9/integration-testing.html

chirp1 commented 4 years ago

Hi Manasi, based on our slack just now, would you investigate to do about the blog post on test containers?

ManasiGandhi commented 4 years ago

Hi Karen. I read through the Testcontainers blogpost https://openliberty.io/blog/2019/03/27/integration-testing-with-testcontainers.html and slacked Andrew. He said that the blogpost is not relevant to the topic and that it is an early precursor to Microshed.

chirp1 commented 4 years ago

Hi Manasi, I'm returning this issue so that you make your terms consistent. See my second bullet and the peer review cheat sheet. You seem to have made the two terms more concrete by adding "or integration tests that test your application in an environment similar to production." to the intro instead of making your terms consistent.

ManasiGandhi commented 4 years ago

@chirp1 I updated the draft https://draft-openlibertyio.mybluemix.net/docs/20.0.0.9/integration-testing.html.

chirp1 commented 4 years ago

Hi Manasi,

If you want to discuss any of these points, feel free to slack me.

ManasiGandhi commented 4 years ago

@chirp1 I worked on your edit suggestions:

chirp1 commented 4 years ago

Hi! We're almost done! Here are my latest comments:

ManasiGandhi commented 4 years ago

@chirp1 I worked on your review comments.

chirp1 commented 4 years ago

One last change:

ManasiGandhi commented 4 years ago

@chirp1 I worked on your review comment.

chirp1 commented 4 years ago

The update looks good. We are done. Closing.