[Epic] Spring on Azure

[chenrujun]

This is an Epic to track all Spring on Azure related tasks.

Context

Now for a developer who used Spring and Azure, he may have many problems about documents and samples.

1. It's difficult the find required documents and samples.
2. The document is outdated.
3. etc..

Goal

Investigate the problem and find a solution to solve the problem.

Solution

• [x] #28058 The feature spec result is written on Spring integration Notebook.
• [x] #28056
• [ ] MS Docs
  • [x] Spring on Azure (Create a new landing page)
    • [x] Create a Draft TOC about "Spring on Azure". The result is written in Spring integration Notebook.
    • [x] Create a mockup about "Spring on Azure"
    • [x] Discuss with all stakeholders (1)
    • [x] Collect current MS docs into Spring on Azure Portal.
    • [x] Discuss with all stakeholders (2)
    • [x] #29749
  • [ ] #28017 (Improve current page)
• [ ] Samples
  • [x] Collect all sample repositories in scope
  • [x] List all current problems
  • [ ] Design a solution about how to solve these problems
    • [x] Discuss with all stakeholders about these problems
    • [x] Give a proposal about how to solve these problems
    • [x] List all "Spring on Azure samples". https://github.com/chenrujun/spring-on-azure-samples
    • [x] #29750
    • [x] #32148
  • [x] #29872
• [x] #29895

[jialigit]

Investigate and list the todos and prepare the environments?

enrich the contents: this is the next step after the above one?

[chenrujun]

@jialigit Detailed contents is not decided yet, need more discussion with our PM.

[chenrujun]

Spring on Azure samples problems and solution proposal

Preface

Proper noun explanation

• Spring on Azure samples: Samples that help customer use Azure products in application develop with Spring.
• Spring on Azure docs: Documents that help customer use Azure products in application develop with Spring.

Goal

Solve problems in current Spring on Azure samples.

Samples in scope

All samples are managed by GitHub repository. Repositories in scope are listed in Spring on Azure Repositories.xlsx.

Problem 1: I cannot find samples

Problem 1.1: I don't know where to find Spring on Azure related sample

Problem explanation: For example: I want to find sample about Using Azure Service Bus in Spring application, but I don't know where to find related sample.

Solution proposal:

• (P1) Improve MS docs: Browse code samples page, make sure all samples can be found in that page.

  • Add entry for all samples in scope
  • Improve the tags, make sure related sample can be displayed in the search result when searching related keyword. Make sure all spring-on-azure samples follow these rules:
    • Every spring-on-azure related sample should contain these tags: java, spring, azure.
    • Each sample should contain spring-project related tag: spring-security, spring-cloud-stream, spring-data, etc.
    • Each sample should contain Azure product related tag: service-bus, event-hub, storage-queue, etc.
    • Each sample should contain usage scenario related tag: messaging, secret, storage, etc.

• (P0) Provide a page which list all Spring on Azure samples Here are some potential places to hold this page:

  • MS docs (The new page about Spring on Azure, this page is not implemented yet).

  • A GitHub README like Azure-Samples/java-on-azure-examples or chenrujun/spring-and-azure-product-samples.

  • In each sample repository, put a link to Spring on Azure samples, then after customer used one sample, he will know how to get all other samples. For example:

    This repository is a member of `Spring on Azure samples`, 
    you can refer to [`Spring on Azure samples`](...) to get all `Spring on Azure` samples.

• (P1) Publish some guide on spring.io/guides

  • spring.io/guides is the official site to hold guides about Spring. If we can add some guides on that page, it can help more customer know sprng-on-azure samples.
  • But we need more communication with spring community if we want to put samples on that page.

Problem 1.2: I can't find sample by searching in search engine

Problem explanation: For example: When search spring service bus sample in google, I can't get Azure-Samples/azure-spring-boot-samples/servicebus. Here is the link to google search result

Solution proposal:

• (P0) After investigation, I found that if the repo name contains Azure product name, then the repo is more possible to be list in the search result of search engine. We can make sample repository follow this template: spring-on-azure-samples-xxx.

  • xxx represent an Azure product. All Azure products can be found here: https://azure.microsoft.com/en-us/services/

  • xxx should be lower case. And each word is separated by -.

    Some example repo names:

    • spring-on-azure-samples-azure-active-directory
    • spring-on-azure-samples-key-vault
    • spring-on-azure-samples-event-hubs
    • spring-on-azure-samples-service-bus
    • spring-on-azure-samples-azure-files
    • spring-on-azure-samples-azure-blob-storage
    • spring-on-azure-samples-queue-storage
    • spring-on-azure-samples-azure-spring-apps //

  • Now all sample repositories name should follow spring-on-azure-samples-xxx. Some samples are forked from other notable examples, for example: brewdis, animal-rescue. For such kind of repositories, we follow these rules:

    • Keep the repository name same to upstream repository name. Because the fork upstream repository is famous, If customer already learned the upstream repository, he can know what this repository is used for without running this repository.
    • Keep current repository still forked from upstream repository. And sync from upstream regularly and automatically. By doing this, customer can get more support. For example: If customer have problem that is not related to Azure products, then the upstream repository maintainer should be responsible for it.
    • Make this repository a member of spring-on-azure-samples
      • Add this repository into the page which list all Spring on Azure samples.
      • In spring-on-azure-samples-xxx, add link to this repository. For example: brewdis is used to introduce how to use Redis in Spring application, then in spring-on-azure-samples-azure-cache-for-reais, we can add content like this:

        For more example about using Azure Cache for Redis, please refer to [brewdis](...).

• (P0) Make sure the description and topics are accurate for all GitHub repositories.

  • Follow this link to get more information about repository topics
  • Just like the tags in MS docs: Browse code samples page. make sure all spring-on-azure sample follow these rules:
    • Every spring-on-azure related sample should contain these topics: java, spring, azure.
    • Each sample should contain spring-project related topic: spring-security, spring-cloud-stream, spring-data, etc.
    • Each sample should contain Azure product related topic: service-bus, event-hub, storage-queue, etc.
    • Each sample should contain usage scenario related topic: messaging, secret, storage, etc.

Problem 2: Sample is not easy to follow

Problem 2.1: There is error in the sample

Problem explanation: When I run the steps in the README, I faced some error, I'm not sure it's the problem of this sample or it's the problem of my develop environment.

Solution proposal:

• Add unit-test & integration test for samples, and setup GitHub Actions to run these pipelines in PR.
• In README, tell customer to create issue if he has problem when running this sample.
• Set maintainer for each repository, response to each GitHub issue created by customer.
• Set SLA for these sample repositories. Make sure all customer created issue / PR has response in a few workdays.

(P0) Problem 2.2: Different Prerequisites of running sample.

Problem explanation:

• For example:

  • sample-repository-a requires:
    • Java 8
    • Maven
    • Use Azure cli to create Azure resources
    • etc.
  • sample-repository-b requires:
    • Java 11
    • Gradle
    • Use Terraform to create Azure resources
    • etc.

  It's not easy to run sample-repository-b even if I'm familiar with sample-repository-a. Is it possible to run sample-repository-b without learning terraform?

Solution proposal:

• Make sure all samples have same requirements, so after customer used one Spring on Azure sample, he will use other Spring on Azure samples easily.
  • Java version: Use Java 8 to cover more customer. Because:
    • If a customer is using Java 11, he could run samples developed in Java 8.
    • If a customer is using Java 8, it's possible that he could not run samples developed by Java 11.
  • Build tool
    • Use Maven instead of Gradle to cover more customer. Because almost all Java developer knows maven, but not all of Java developer familiar with Gradle.
    • Use Maven wrapper. So, customer can:
    • Build the sample project without install maven.
    • Avoid the problem caused by different version of maven.
  • Method of creating required Azure resources. (Refs: MS docs about create azure resources) . I think Azure CLI is the best choice according to this table:

	Azure Portal	Azure Tools extension for VS Code	Azure CLI	Azure PowerShell	Infrastructure as Code tools: Bicep, Terraform, Ansible
Seldom change. Change can be detected	:x:	:x:	:white_check_mark:	:white_check_mark:	:white_check_mark:
Can create a bunch of resources without interaction	:x:	:x:	:white_check_mark:	:white_check_mark:	:white_check_mark:
Can be used in all IDE	:white_check_mark:	:x:	:white_check_mark:	:white_check_mark:	:white_check_mark:
Esay to use in all kinds of Operating system	:white_check_mark:	:white_check_mark:	:white_check_mark:	:x:	:white_check_mark:
User can know what he created by interaction	:white_check_mark:	:white_check_mark:	:white_check_mark:	:white_check_mark:	:x:
Can be used without much learnning effort	:white_check_mark:	:white_check_mark:	:white_check_mark:	:white_check_mark:	:x:

(P0) Problem 2.3: Different style of README.

Problem explanation:
For example:

• Azure-Samples/spring-cloud-gateway README's TOC:

  # Deploy Microservices and Spring Cloud Gateway using Azure Spring Cloud and Redis Cache 
  ## What will you experience
  ## What you will need
  ## Install the Azure CLI extension
  ## Clone and build the repo
  ### Create a new folder and clone the sample app repository to your Azure Cloud account  
  ### Change directory and build the project
  ## Provision Azure Spring Cloud service instance using Azure CLI
  ### Prepare your environment for deployments
  ### Login to Azure 
  ### Create Azure Spring Cloud service instance
  ### Load Spring Cloud Config Server
  ## Create Redis Cache
  ## Create microservice applications
  ## Deploy applications
  ### Rate limiting
  ## Next Steps
  ## Credits
  ## Contributing

• microsoft/spring-todo-app README's TOC:

  # Spring Todo App
  ## Requirements
  ## Create Azure Cosmos DB
  ### STEP A - LOGIN to Azure
  ### STEP B - Create Resource Group
  ### STEP C - Create COSMOS DB
  ## Running Spring TODO App locally
  ### STEP 1 - Checkout Spring TODO app
  ### STEP 2 - Configure the app
  ### STEP 3 - Run Spring TODO App locally
  ## Contributing
  ## Useful link

  If Azure-Samples/spring-cloud-gateway's What you will need is same to microsoft/spring-todo-app's Requirements, then it's better to uniform the description. It will help reader easier to use these samples.

Solution proposal:

• Provide a template of README, make all Spring on Azure samples follow the same template. Here is the sample of template README:

  # Manage secrets by Azure Key Vault Sample
  ## Purpose
    This sample is aiming to demonstrate how to how ...
  ## Ask for help
    If you have any questions or suggestions about this sample, please create an issue.
  ## Contributing
    If you want to contribute to this sample, Pull request is warmly welcome.
    Please read COMTRIBUTING.md before creating PR.
  ## Prerequisites of running this sample
    You need to finish these steps before running this sample:
    1. Install Java 8
    2. Install Java IDE
    3. Install Git and clone current GitHub repository
    4. Get Azure subscription
    5. Install Azure CLI and login Azure CLI
    6. etc.
  ## Run this sample
  ### Create required Azure resources
  ### Modify application.yml
  ### Start sample application
  ### Validate application

• We can simplify this process by providing a GitHub template repository.
  • Here is an example of template: chenrujun/spring-and-azure-product-sample-template.
  • With template repository, it will be much convenient to create a new Spring on Azure sample repository.
  • It's possible to update the template repository, so the template should have a version. And this version should be written in the README:

    Spring on Azure sample template version: 1.0.0.

Problem 3. Sample does not support all supported versions.

Problem explanation:

• Current sample repository is using spring-boot:2.5, my project is using spring-boot: 2.7.
  • I'm not sure whether this sample can apply to my project. I need do some investigation to figure it out.
  • If there is difference between spring-boot:2.5 and 2.6, can I find both samples for the different version of spring-boot?

Solution proposal:

• In default branch, use dependabot to make sure all dependencies are using the latest version.
• List all supported versions of spring-boot.
• (P0) If there is difference for different version, it should be written in README or use different branch to demonstrate that usage. Here is an example of fragment in README:

  
  ### Spring Boot version

• This sample is using Spring Boot:2.7.
• This sample is compatible with Spring Boot:2.6. If you want to use Spring Boot:2.6, just change Spring Boot version in pom.xml, no other things need to be changed.
• This sample is NOT compatible with Spring Boot:2.5. If you want to use Spring Boot 2.5. please refer to ... (another branch).

• (P1) Use GitHub actions to run unit test for different version of Spring Boot.

Problem 4: Multiple similar samples.

Problem 4.1: Similar repositories

Problem explanation: For example:

• pet-clinic related sample appears in these places:
  • https://github.com/Azure-Samples/spring-petclinic-microservices
  • https://github.com/Azure-Samples/azure-spring-boot-samples/tree/spring-cloud-azure_v4.2.0/spring-petclinic-microservices
  • https://github.com/Azure-Samples/pet-clinic-jar
• todo-app related sample appears in these places:
  • https://github.com/microsoft/spring-todo-app
  • https://github.com/microsoft/todo-app-java-on-azure

Solution proposal:

• (P0) Deprecate similar repositories, just keep one repository.
• If the repository is mainly used the introduce one azure product, move the sample to azure-product related sample repository. For example, https://github.com/microsoft/spring-todo-app only used to introduce how to use Azure Cosmos DB in Spring application, this sample can be moved to a new repository named spring-on-azure-samples-azure-cosmos-db. Archive original repository, add a link to new repository. Here is an example:

  This repository is archived, to get sample about using Azure Cosmos DB in Spring application,
  please refer to [spring-on-azure-samples-azure-cosmos-db](...).

Problem 4.2: Sample of same feature in different repository

Problem explanation: For example:

• https://github.com/Azure-Samples/cosmosdb-springboot-aks introduced how to use Azure Cosmos and deploy to AKS. But similar sample of using Azure Cosmos appears in many repositories.

Solution proposal:

• (P1) Do not introduce multiple Azure product in azure-product-specific repository. For example:
  • spring-azure-samples-cosmos-db should only contains samples of Azure Cosmos, it should not contain samples about Azure Key Vault. But it can have content like this:

    ## Next step
    - Store the access-key of cosmos-db in Azure Key Vault, please refer to [link]
    - Access Cosmos DB by managed-identity in Azure Kubernetes Service, please refer to [link]

  • spring-on-azure-samples-key-vault should only contains samples of Azure Key Vault, it should not contain samples about Azure Cosmos.

(P0) Problem 5: Not provide the best solution.

Problem explanation: For example: In https://github.com/Azure-Samples/spring-jms-service-bus, use spring-cloud-azure-starter-servicebus-jms is a better option to in this sample.

Solution proposal:

Make sure every Spring on Azure sample is maintained by related expert. The experts should be responsible to provide best solution.

• Set experts of each Azure product and each spring project. For example:

  • Azure products experts. All azure products can be found here: https://azure.microsoft.com/en-us/services/	Azure product	experts
    Azure Key Vault	@ chenrujun
    Azure Service Bus	@ yiliuTo
    Azure Files	@ backwind1233

  • Spring projects experts. All spring projects can be found here: https://spring.io/projects/spring-data. Note: We keep both level-1 project (like spring-data) and level-2 project (like spring-data-jpa).	Spring project	experts
    spring-security	@ chenrujun
    spring-cloud-stream	@ yiliuTo
    spring-data	@ backwind1233

• Get the Azure product list and Spring project list for each Spring on Azure sample.

  • For repositories like spring-on-azure-samples-azure-cosmos-db, it can only have one Azure product: Azure Cosmos DB. But it can have multiple Spring projects. Like: spring-data-jpa, spring-data-jdbc.
  • For repositories like spring-petclinic-microservices, it can have multiple Azure products and multiple Spring projects.

• Make sure all samples are maintained by related expert by setting expert as GtHub Code Owner. If there related file change in a PR, merge is forbidden without the expert's approve.

• Auto fit sample change and expert change

  • What is sample change and expert change?
    • Sample change. For example: new sample repository created, new sample module added in an existing repository, new feature added in an existing sample module. Delete a repository / sample module / feature in module.
    • Expert change. For example: old employees leave the company / team, new employees join and work as related expert.
  • How to auto fit sample change and expert change?
    • Maintain metadata of experts and samples. Here is an example of spring-on-azure-samples-metadata.yml:

      
      spring-on-azure-experts-for-azure-product:

    • azure-product: azure-keyvault artifacts: com.azure.spring:spring-cloud-azure-starter-keyvault-secrets # Used to scan repository, make sure expert is maintainer / codeowner experts: @chenrujun
    • azure-product: azure-service-bus artifacts: ... experts: @yiliuTo
    • azure-product: azure-event-hubs artifacts: ... experts: @yiliuTo spring-on-azure-experts-for-spring-project:
    • spring-project: spring-security artifacts: ... expert: @chenrujun
    • spring-project: spring-cloud-stream artifacts: ... experts: @yiliuTo repositories:
    • url: https://github.com/Azure-Samples/spring-on-azure-samples-keyvault azure-products:
      • azure-product: azure-keyvault path: sample-1/sample-11 spring-projects:
      • spring-project: spring-security path: sample-2/sample-22
    • url: https://github.com/Azure-Samples/spring-on-azure-samples-service-bus azure-products: azure-service-bus
    • url: https://github.com/Azure-Samples/spring-on-azure-samples-azure-spring-apps azure-products: azure-spring-apps, azure-keyvault, azure-service-bus

    • Make sure spring-on-azure-samples-metadata.yml always updated.
    • Update experts manually.
    • Update repositories manually and atomically
      • Manually. In each sample repository's CONTRIBUTING.md file, write content like this:

        If you want to use a new Azure product or Spring project in this repository, 
        please update [spring-on-azure-samples-metadata.yml](...), and [CODEOWNER](...)

        This content can be added into spring-on-azure-samples-template.

      • Atomically. Use GitHub action to scan sample repositories.
      • If a pom file contains dependency of azure-cosmos, then in spring-on-azure-samples-metadata.yml, related azure-product should contain azure-cosmos-db.
      • If a pom file contains dependency of spring-security-oauth2-client, then in spring-on-azure-samples-metadata.yml, related spring-project should contain spring-security.
    • Make sure CODEOWNER always updated according to spring-on-azure-samples-metadata.yml. This can be done by setting up GitHub actions.

(P1) Problem 6: Not using supported version.

Problem explanation: For example:

• Some samples are using Spring Boot 2.2. But 2.2 is not OSS supported.
• Some samples are still using Spring Cloud Azure: 3.x, which is end of life.

Solution proposal:

• Use dependabot to make sure all dependencies are using the latest version.

(P1) Problem 7: Not updated for more than one year.

Problem explanation: Some sample repository is not updated for more than one yeat. For example: https://github.com/Azure-Samples/brewdis is not updated for 2 years.

Solution proposal:

• dependabot can avoid this by always update dependency version.
• Each repository has maintainers, the maintainers will be responsible to merge PRs created by dependency bot.
• Use GitHub actions to check PR/Issues that no response for a long time, notify the maintainer by email / teams.

(P0 inside spring on azure team) Problem 8: Lack of synergy between among Azure products.

Problem explanation: For example: samples from Spring Cloud Apps does not use Spring Cloud Azure libraries.

Solution proposal: This mainly happened in end-to-end-samples. It can be solved by spring-on-azure-experts system designed in the solution of Problem 5: Not provide the best solution.

Problem 9: Repo name does not describe content without contextual information

Problem explanation: Repos with name like brewdws / pet-clinic-jar / animal-rescue / todo-app will not provide enough information. I don't know what the repository is used to without reading the README.

Solution proposal:

Make repository names follow the rule described in Problem 1.2: I can't find sample by searching in search engine: spring-on-azure-samples-xxx.

(P1) Problem 10: Steep learning curve.

Problem explanation: Some samples introduce many features in one application. It will make the sample hard to learn. For example: aad-web-application contains these features:

• login
• client access resource-server
• authorize by app role
• authorize by group name & group id
• on-demand client

Solution proposal:

• In a sample repository, split sample by feature.
• Each sample introduce one feature or a couple of few features. Make sure each sample is easy to learn.
• Provide a learn path of each sample. Make sure one sample is easy to learn after finished preceding samples.

For example, here is a folder structure of a sample repository: There should be a learning path like this: In the learn-path image, sample-A -> sample-B means if customer want to run sample-B, he should run sample-A first.

Next step

• [ ] Create the 2 sample repo in {Azure-Samples](https://github.com/Azure-Samples) :
  • Azure-Samples/spring-on-azure-samples-template
  • Azure-Samples/spring-on-azure-samples-management, which list all samples in its README, and include logic codes to manage all spring-on-azure samples.
• [ ] Write spring-on-azure-samples-template, and release 1.0.0 version.
• [ ] Make current samples follow the template
  • [ ] Make Cosmos DB related samples follow the template
    • [ ] Create repository like spring-on-azure-samples-cosmos-db from spring-on-azure-samples-template, add it to spring-on-azure-samples-management,
    • [ ] Move current spring and cosmos-db related sample into the new repo
    • [ ] Deprecate original repo, add link in original repo's readme, link to new repo
    • [ ] Update spring-on-azure-samples-template, release a new version if necessary.
  • [ ] Make Azure Active Directory related samples follow the template
  • [ ] Make Azure KeyVault related samples follow the template
  • [ ] etc..
• [ ] Create job to manage these repositories. For example:
  • [ ] Create branch spring-boot_3.0 for all managed repositories by a GitHub Action. Or Use GitAction to create the issue: let the repository maintainer do this task mainally.
  • [ ] When spring-on-azure-samples-template released a new version, Use GitHub Action to create issues in each repositories: let the repository maintainer update the template version.

[stliu]

1. open questions
2. bias
3. measure success
4. delivery

[chenrujun]

1. Open questions 1.1. Where to list all samples, as described in Problem 1.1: I don't know where to find Spring on Azure related sample.

2. Bias (Resolved) 2.1. Method to create azure resource. Added a table to comparing all methods of creating azure resources.

3. Measure success (Need more discussion with Sean) 3.1. ROI. Now I'm sure how to calculate the number of RIO. 3.2. OKR: 3.2.1. Subscription (Need to discuss with Sean). Now I I'm not sure how to confirm that how much 3.2.2. The count of visitor and cloner of each sample repository.

   • 3.2.2.1. GitHub traffic page of azure-spring-boot-samples repository
   • 3.2.2.2. Traffic Insights - how can I view it for a longer period? 3.3. Target status, what we delivered. Align the expectation. It's written in 4. Delivery.

4. Delivery 4.1. All samples follow the same template. 4.2. All repositories have a maintainer, publish a document about maintainer's responsibility. 4.3. Build SLA system to make sure all samples can work well. 4.4. Build spring-on-azure-experts system, make sure all spring-on-azure samples are maintained by related expert. And all new samples can be detected.

[fangjian0423]

Updates:

Azure for Spring developers hub page is now live.

[chenrujun]

Closing this issue. Because we decide to use GitHub issue to track Feature or Bug. Epic is not necessary anymore.