Roadmap

[Sandared]

Outline of content planned to be added to this website

• "How to contribute" content

  • How to add a tutorial [FINISHED]
  • How to add a blog post
  • Markdown Cheatsheet [FINISHED]
  • Content Layout explained [FINISHED]
  • Nested Layout explained [FINISHED]

• Create a repository containing a basic GitPod setup for further tutorials [FINISHED]

• Tutorial Series

  • From Hello World in Java to Hello World in OSGi [IN PROGRESS]
    • Starting with an empty workspace (Gitpod should have all necessary tools set up, snippets unfortunately not working yet)
    • Use Maven archetype to create a simple plain Java Hello World Application
    • demonstrate everything works fine
    • show manifest file
    • add bnd-maven plugin to project
    • show manifest file again
    • show that the plain java application still works
    • add necessary bnd-maven plugins to project to create a runnable osgi “fat jar”
    • now start OSGi framework with this bundle installed and that this still “works” (no error message, but also no ouput, as the main method will not be invoked)
    • Comment: Hello World doesn't demonstrate why OSGi is relevant, does it? What about introducing the perspective of composing an application from (initially) independent parts from the start. How can they both start/run? This is an argument for bundles with a life-cycle and BundleActivators. Later, to integrate, we need services and components.
    • explain what OSGi is (Part 1),
    • framework on top of JVM
    • wants Jars with proper manifests and calls them bundles
    • show what manifest entries are necessary to work within OSGi and show that our bundle has those entries
  • Bundles in OSGi
  • Back to the “not working” main method: explain what BundleActivators are and how they correlate to the Bundle Lifecycle
  • Implement the Hello world in the acitvate method of a bundle activator
  • explain why the bundle activator alone does not help us in order to write proper OSGi applications
  • create a second bundle with some classes in it
  • try to instantiate classes from one bundle in another bundle -> this will not work
  • We can explain what private and public packages are
  • OSGi offers on bundle level what Java offers on class level (public/private) link to module layer and what it has to do with this?
  • Now add the respective Bundle annotations for public and private packages to the source code
  • Instantiating classes at runtime will now work
  • Services in OSGi
    • We now can instantiate directly classes from another bundle, but this might not be the best architecture
    • Stop a bundle at runtime (showing dynamism of the bundle lifecycle) and see what happens (both bundles will be deactivated?)
    • Introduce services as a mean to cleanly
  • OSGi development
    • add other needed bnd-maven plugins and explain the final setup, what each plugin does (maybe also maven basics?) and how development works with hot code reload thanks to the bnd-maven-run plugin
    • Introduce an HttpServlet that needs to find an HttpServer and register itself (and possible some static resources.
    • Explain the service registry and how it serves as a central hub for service provided by the different bundles
    • Locating other services is done in the BundleActivator and uses the dynamic services API, and illustrate basic broker-provider-consumer collaboration. Combined with the life-cycle of bundles, we also illustrate the complexities of manually tracking services we depend on.
    • This motivates DS which is more declarative and handles service tracking and injection and "rejection".
    • Explain the two basic OSGi patterns, Extender/Whiteboard
    • show Examples for those patterns, e.g. HttpWhiteboard/EventAdmin or GeckoEMF
    • Show how the Gogo shell can be extended with new commands
    • Explain the component life-cycle and (use of) activate/deactivate methods.
    • Explain component configuration data and the update method
    • Explain the need for and how to create new components during runtime with ConfigAdmin (i.e. manipulate configuration records that triggers creation or updates)
    • Exemplify this with a Gogo shell command that creates new components configured with command arguments
  • Common Pitfalls
    • Explain common pitfalls when developing OSGi applications, e.g. resolver errors and how to solve them (in most of the cases)
  • Testing bundles within a running framework (integration test)
  • Deploying to docker and testing deployed system (system test)

Decisions

• Example case for tutorials:
  • Hello World for bundles
  • REST example app for the rest
• Structuring of tutorials/repos:
  • One repository per tutorial
  • One branch per step in tutorial

[Sandared]

@hallvard : I moved the Roadmap from the site to this issue. If you have anything to add feel free to comment :) If you join the gitter channel on the site we could also chat over gitter.

[hallvard]

I did some editing above, including adding a comment. A crucial point is to what extent we focus on how (things work) vs. why (we design systems this way). A problem with starting with Hello World is the lack of reason for doing it in OSGi. Another problem with Hello World is that most apps are web application anyway, so a Servlet may be a better starting point, that's at least what I start with in my course.

I think we at least need a simple and understandable design case where it is obvious (enough) that having several interacting bundles is natural/useful. And where some kind of extensibility is needed.

[Sandared]

Thanks for your input :) I will first explain why I chose to use Hello World as a starting point and the USP of OSGi, then I spend some words on how we might take a new approach on teaching OSGi (especially to younger people), then add some thoughts on starting with a servlet and then add another option to start with . Additional thoughts on any of those points are highly appreciated, as they are just my condensed experiences and maybe suffer from false perception or survivor bias ;)

Why Hello World My intention of starting with Hello World was to keep the things (technically) as simple as possible, so that a reader can understand what it actually means (technically) to OSGify an application/library/etc., but you are right in that this does not provide a good reason on why to do it in OSGi.

Why modularity might not be the perfect USP for OSGi I think the reason to use OSGi in the first place is a topic on itself, as the rather abstract concept of modularity doesn't sell well to people who just want to get things done fast (a point where OSGi or modular applications at all don't shine). In my experience, most people out there first want to do things fast and then -when the application grew too large to be maintainable anymore- do it right/modular. So in my opinion modularity often comes too late as a pain point in a project to be the USP to start with. However, this does not mean it is not an important aspect that should be teached in the tutorials, just not at the beginning.

A different approach to teach OSGi Therefore, I would tend to another approach in teaching OSGi to newcomers, i.e., shifting the "What" in front of the "Why". I know this contradicts our usual way of narative in tutorials, but I've made the experience that this works better to motivate people to further deal with an issue, because they first see results and get things done fast (and in this case right) and then get the lengthy explanation why. The other thing I would recommend is to break everything down into really small steps as explained on the page on how to add a tutorial. This results from the observation that the attention span becomes shorter the younger the readers are.

A Servlet as starting point for tutorials I think a Servlet might be not the perfect starting point, as to really understand how a Servlet works in an OSGi environment, one has to understand the whiteboard pattern, thus services / components and thus bundles. The amount of background knowledge that has to be provided for such a tutorial is in my opinion too much to begin with, although it might provide a better case for reasoning on why OSGi is needed from a modularity perspective.

Another possible starting point I still would prefer to start really simple, but also would like to see a good overall usecase to show the benefits of modularity later on the tutorial series and Hello World is clearly no fit for this. Therefore, we could take a look at a talk Milen Dyankov gave several times, which dealt with decomposing Java applications: https://www.youtube.com/watch?v=2_v2JybFKUw

@milendyankov : You wished me good luck when I started with the OSGiNews Twitter channel and hoped I have more luck with it than you. So after surviving for close to a year now, this website is my next small step forward. It would be awesome if you could also add your thoughts on this topic, as you already made some experiences with your talk during different conferences. What was the feedback? Is the source code for this talk publicly available somewhere?

Kind regards, Thomas

[hallvard]

I think it's OK to start with how to do a minimal Hello World with OSGi, as this will give them a sense of achievement and their first experience with how the framework controls startup. We could do it in two steps, add another simple bundle and play with the startup order and use the shell.

The advantage of a Servlet is that it is naturally modular, in the sense that you have a server and the servlet as (necessarily) distinct parts. We would rather avoid having to know the specifics of the server, so we use the standard HttpServer service. I.e. we don't need the whiteboard pattern, instead the BundleActivator can look for the HttpServer service and register a HttpServlet and/or a static resource. That's the "old" way of doing it. This can motivate DS, i.e. we can get the HttpServlet injected, and then, as a next step, we can inverse the relationship with the whiteboard pattern, where the server (without us needing to know how) picks up our servlet. Many small steps, as you suggest. And even later, use JAX-RS annotations.

[Sandared]

That sounds good 👍

Just to summarize so far:

• Hello World is used for showing differences between an OSGi and a plain Java application up to the point where services are entering the stage.
• From then on we switch to the Servlet example in "legacy mode" and incrementally turn it to "state of the art mode" in order to use it for services, components/DS, whiteboard.
• somewhere in between we need to explain the general concept of requirements and capabilities and resolving
• We should also do some steps that lead to common errors (e.g., not resolving) and show strategies to solve these errors
• After that we could use Milen's example for motivating OSGi in the context of modularity.
• From there on we could start to show different specs, e.g.,
  • Http-Whiteboard in detail
  • JAX-RS
  • EventAdmin
  • Commands for Gogo shell Those specs can show how with simple patterns like the whiteboard almost every application can be designed modular by default.

[azzazzel]

I tend to agree that a "Hello World" and traditional tutorials "for zero to hero" style may not be the best approach. OSGi tried that with enRoute and while I don't have hard data, I have the feeling it didn't achieve its goals. I think the issue is more psychological that technical. The people who would benefit from a tutorials like those are those who have already chosen to use the technology for other reasons. Being easy and delivering results fasts is secondary goal, giving me what I think (or believe) I need is the primary goal. We tend to look at the technologies like SpringBoot and think that the amount of tutorials and docs and talks and ... are the key to success. But that's only a reinforcing feedback loop. The key is the technology was painted as the one supplying the solution for a great (even if artificially created) demand like "microservices for Java developers". I predict "Quarkus" will soon have the same impact as it is promoted as the one and only serving another great demand "fast and slim Java". The docs, tutorials, blogs, videos, ... make the adoption easier but they don't cause it.

So the way I see it is, the biggest question is "what is the demand that OSGi can supply a solution for?". And I explicitly use the wording "to supply a solution for a demand " and not "to solve a problem". Think about it like how car engines that are easier to fix, tune, improve, ... solve a problem for the car mechanics but they do not create a demand for a particular car model (outside the car's fan club). Without demand, there will be no chance for wider adoption and any effort like yours will only serve the people who have been made to use the technology (as a result of choices made by other decision makers). In that sense those are still useful, but may be not as much as you anticipate.

Given the above, I personally have came to the conclusion that targeting general purpose Java developers with the hope to demonstrate an easy solution, does not work. They already have the easy solution and the reinforcing loops around it that makes it super hard for them to change it to something else. I personally decided to target more senior developers instead. Folks that have enough experience and see a need for better software architecture. It's not a lot of them. And the environment is very competitive with higher level abstractions like containers and Kubernetes. But IMHO that's the space where OSGI still has chance to be relevant. Sadly, the OSGI Alliance does not seem to think the same way.

With that in mind, some time ago I actually started writing a book. The idea was to explain in more details an updated version of my take on the Modularity Maturity Model that I have presented in "What's NOT new in modular Java" talk (https://www.youtube.com/watch?v=gHk_XxRKNiw). But I haven't touched that project for months now as I both don't have enough time and I keep questioning the sense and the possible impact.

As for the Decomposing Java Applications talk, is has been very well received AFAICT. Perhaps because it's not about OSGi but about software design and architecture. The code is available of course: https://github.com/azzazzel/DecomposingJavaApplications Feel free to use it in any way you wish.

[hallvard]

@Sandared After the introduction of DS we could exemplify services by showing off some from the standard (compendium). Then later we could give an overview of the ones we think are most useful and important. Among these, ConfigAdmin deserves extra attention, since it is the easiest way to create components dynamically, e.g. in a gogo command, and shows how nice the component (annotation) types are for making configuration data typesafe. In addition, we need to explain how the life-cycle of (injected) components are tied to the bundle life-cycle, and illustrate how this can support interesting operation scenarios. In my course, I used this as the main selling point of OSGi, the ability to keep (most) services up while updating the system. Only a few students appreciated modularity and extensibility as important qualities. Finally, deploying and testing using docker should be included, and perhaps also remote services (which I lecture about, but don't show them of make the try).

@azzazzel My motivation (for this site) isn't really to win people over to OSGi, since I don't need that for the course (it's mandatory ;-). Still, it is important to motivate the various mechanisms when they're introduced. In the future, I may want to use the material for smaller courses for people from industry, and for such an audience relevant scenarios are important to motivate and "sell" OSGi. I discovered OSGi through Eclipse and plugin development, and I try to explain that such a framework wouldn't work without an underlying OSGi(-like) framework that support modularity and extensibility. However, students are not fond of Eclipse, so I also tell them about other systems built on OSGi to show it's relevance. Still, systems don't generally advertise "OSGi inside" and since they are mostly enterprise systems, students don't know them. Imagine how much easier it would have been if e.g. Android was based on OSGi! Industry seniors, on the other hand, would appreciate such an angle, I imagine.

[hallvard]

Another issue, I suggest using plantuml for illustrating architecture and design, e.g. bundles and packages with dependencies (component diagrams), component collaborations and life-cycle (state and sequence diagrams).

[Sandared]

@azzazzel thanks for this in-depth answer.

The part about only seasoned developers recognize the need for better architecture overlaps with what I meant when I said, that modularity is not a good USP for OSGi to start with. The problem it solves is being hit too late down the road of development in order to be relevant for technology decisions in the beginning for most people. Therefore it's right when you say that we should not try to promote OSGi as the one-fits-all-solution and instead need a specific demand that OSGi supplies a solution for.

This demand could be derived from usecases where OSGi really well fits into, e.g. for (dynamic) plugin systems that don't (initially) need to be distributed. RL usecases fo this are for example runtime updateable monitoring agents like those of Instana. Another example is the plugin system of Atlassian products. AFAIK both use OSGi at its core. Also CLIs for products could be perfectly written OSGi and extended as GoGo Shell demonstrates.

I think the question is: What shall the target audience of this site be? Java-Experts OSGi-Newcomers? Java-Newcomer OSGi-Newcomers? Any other combination?

Also thank you for the link to your code. Is there any chance that some of the content of the book you started might find its way onto this site? ;-)

@hallvard we could use plantuml in markdown preview enhanced, however I assume some sorts of diagrams will be impossible to create witch plantuml and for those I would like to use draw.io.

The proposed ordering sounds reasonable. Could you maybe sketch it out a little bit in the above Roadmap?

Regarding popular examples for the use of OSGi: I think @jbonofre mentioned some time ago that Karaf is used at Netflix. It might be interesting to know if OSGi played a role here or if they just used Karad as a container for a Spring Boot application? :)

[hallvard]

We should move to editing a document describing the steps, rather than editing this issue. How do we structure all this in repos with hierarchical OSGi maven projects? Is it at all possible to have more than one project (with many bundles) in one repo? If not, we will quickly end up with lots of repos, which may be cumbersome to work on independently. E.g. there is no search and replace across repos, is there?

Additionally, we should decide on a case, to make the discussion less abstract. In my course, one option for a small project was a REST service for a small data model, where the data could originate from numerous (other) sources with (slightly) different models. The various source models and how they were "transformed" to the REST service model was handled by different bundles and integrated into the REST service using DS. For each source data model there could be several concrete data sets, so there was a need for having several instantiations of generic components, hence the need for configuration data. When new sources (models and data sets) are identified, they can be added to the system without restarting, since OSGi handles this well.

[Sandared]

Regarding general project organisation I see the following possibilities:

One repository for all tutorials If we use one giant repository for all tutorials and assume one branch per step in one tutorial, then a git history for such a repository might look like this so that people can easily jump between steps, and each step can be updated in a way that the steps built on it can be updated too.

| master (initial commit with gitpod stuff)
|
  \  tutorial-01-step-01               \ tutorial-02-step-01
   |                                    |
     \ tutorial-01-step-02               \ tutorial-02-step-02
      |
      ... 

Such a branching strategy would however prevent us from doing search and replace on the whole repository at once, as in each branch only one tutorial is present.

If we instead decide to do branching for each step based on the tutorial before like this:

| master (initial commit with gitpod stuff)
|
  \  tutorial-01-step-01               
   |                                    
     \ tutorial-01-step-02               
      |
      ...
      \ tutorial-02-step-01 
       |
        \ tutorial-02-step-02

This might work better, but can also become a hassle when an early branch is updated and all branches building upon it need to be updated. This would be easier in the first variant. Contributions from outside the organization can easily be done on the tutorial repository via PR.

One repository per tutorial Each repository would consist of several branches each representing a step. Branching strategy would be like the second one above. This would keep the repositories small and easy to update each, but search and replace over several of them would not work. Contirbutions from outside the organization would need to provide their own repository within the Github account of the contributor This might be good or bad, I don't know.

Maybe third strategy Instead of each step being a branch , each step could also be a seperate project within one repository, i.e., :

tutorial-01-repo
|-- step 01 (contains source of step 01)
|-- step 02 (contains source of step 01 + 02)
...

We also could do this for a giant repository, however, this will lead to a lot of copying code by hand from a changed early step to steps that build upon it.

@hallvard Do you have any thoughts on this or any ideas?

Regarding the example project:

Tutorials covering topics up until bundles I still like the idea of a Hello World tutorial, with which we can show the difference between plain java and osgi jars/bundles and explain the ideas behind it without introducing too much programming, however I also like the example project ideo of your course. So if we could slim this one down enough I would ditch the Hello World example in favour of your example. This might also lead to a better teaching experience, as we only use one running example for the majority of this course. @hallvard : did you have a look at Gecko EMF for the different models? Does it make sense in this context? If so maybe we should invite Jürgen Albert from Data in Motion to this discussion (Creator of Gecko EMF)

**Tutorials covering the rest I think your example with the servlet fits this best.

We should move to editing a document describing the steps, rather than editing this issue.

Either we edit the issue summary at the top each time we reach a consense in the discussion or we open up a wiki page for it. If we use a wiki page we should link it in the issue description. I'm good with both options.

[Sandared]

@hallvard I started with the first tutorial so that we have an example that we can discuss better about. I hope I will find time at the weekend to complete it

[hallvard]

Good discussion about repo, branch and project structure. I didn't think of using branches for steps, that's a good idea. The choice(s) depends a bit on the scope of tutorials and steps. It's a bit like classes (and bundles), if a they get too small and numerous, we need another packaging level. I think one repository per tutorial is best, since we anyway should make them self-contained and consumable individually (while pointing forward and backward). Perhaps that's a good criterion for a good tutorial size: what makes sense to clone and study as a whole.

[Sandared]

I agree :) So one repo per tutorial it shall be ;) Regarding the running example I have come up with another possibility: Gogo commands This would free us from cluttering the teaching of OSGi concepts with additional concepts, like servers/servlets etc.

We could go from a simple HelloCommand class and a main method in a plain jar over to bundles, activators and a command class, over to different bundles for Hello/GoodBye commands, over to separating api and impl in bundles, over to services, to components and finally to whiteboard with Gogo shell.

WDYT @hallvard ?

[hallvard]

I'm not convinced... a web/rest service just seems more relevant and students' motivation quickly drops if they can't relate to the purpose, even though the purpose really is the technology and design.

[Sandared]

Ok. So we stick with the Hello World (bundles) and then REST (for the rest) example.

I started with lesson one yesterday and realized that this will take more time than expected (as it is always the case ;)) I am assuming a novice reader, i.e. someone who can read and write Java code but otherwise has no experience yet. Therefore I explain everything that is needed to understand what is going on when you execute the instructions given in each step. This might be a little overkill, but someone who already knows how the stuff works behind the scenes can just skip those explanations.