Template API "Updating template listing...please wait"

[austinmarten1]

For some context. I am running openrmf in a kubernetes environment. I originally ran into some issues with JWT tokens and authentication but fixed that by building the images manually with some commits that were pushed recently.

However now the only thing that is appearing to know work is the Template API. On the dashboard menu it does read out how many "Templates" that I have but when I go to the Templates section I get stuck on "Updating the template listing"

Screenshots

Additional context I am running openrmf 1.9

[DaleBinghamSoteriaSoft]

Are there any JS errors in the dev console if you can see that? Did you clear cache after updating and retry, just in case some linked in JS updated? We will have to test on K8s on our end to see what we can find out as well.

The latest code is v1.9 and all were pushed to the /master or /main recently after the v1.9 was pushed out. All code in the main branches are 100% up-to-date.

[austinmarten1]

Hello, there is one javascript error that I see in the console.

Edit: Yes I did clear cache and retry

[DaleBinghamSoteriaSoft]

We will have to run through the Source and breakpoint around that to see what the bug is. And then determine whether it is JS or API related.

It is odd it only happens for K8s. We need to duplicate that on our end as well.

[austinmarten1]

Sounds good. Thank you sir.

[austinmarten1]

Reaching out again. Curious if there is any update on this issue.

[DaleBinghamSoteriaSoft]

Not yet, no. I spent all weekend troubleshooting K8s issues and most of last week in meetings, demos, and working on some Professional features. I hope to get to this by the end of the week. But I have to pay bills first.

[austinmarten1]

Thank you, I was more or less curious.

[DaleBinghamSoteriaSoft]

When I look at the JS error in the browser I see "err_incomplete_chunked_encoding nginx". We are using NGINX for the web frontend so it may be something to do with that for K8s. This setup for K8s is old and has not been updated in a bit. We really should be using the MongoDB Community Operator, updated NGINX, updated Prometheus/Grafana, and setting gzip params and such on larger lists like this coming through EKS.

There are no errors in the logs for the template API container. So I am thinking it has to do with the config. I need to compare it to what we do in professional as that works great and is fairly easy to setup and install. We recorded that install video for our video platform. I need to compare that to this OSS setup and probably update the K8s YAML we use to push up. And make sure to use the operator to make the process easier for the DB setup and usage. And check the NGINX configs.

That is no small task to do unfortunately. So it will take a bit of spare time to do that. We do have people running over HTTPS using EC2.

If you have time, I would check on the NGINX settings for this type of timeout thing possibly as well below.

      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"

The MongoDB Community Operator is in GH https://github.com/mongodb/mongodb-kubernetes-operator and is easy to setup, push out, and then update the connection strings to use that as well. You will have to merge all the JS initialization into proper statements for a single DB if you wish. That will suffice for the OpenRMF OSS setup honestly.

[Cingulara]

This runs locally, so have to test on K8s when we deploy the latest version with the latest templates. Working the update now on this and the latest DISA checklist templates as of Nov 1, 2023.

[Cingulara]

@austinmarten1 I got this to run at https://demo.openrmf.io/templates.html with v1.10. There is a trick to it, and still a fix that needs to be done.

In the template YAML, there needs to be a variable for JWTINTERNALAUTHORITY on all APIs in the ENV:

        - name: JWTINTERNALAUTHORITY
          value: https://keycloak.openrmf.io/auth/

That is because we use the internal-to-docker-network URL for validating JWTs when local/on a VM. But in K8s that does not work right because of naming and other things. SO we use this variable. It is basically the JWTAUTHORITY variable w/o the realms/openrmf in it.

---

Also the Template API YAML specifically has to have these resources to get it to run right and not spin OOM:

        resources:
          limits:
            memory: "2250Mi"
            cpu: "500m"
          requests:
            memory: "750Mi"
            cpu: "250m"

[Cingulara]

@austinmarten1 the bug fix:

• remove the template XML raw data from the Template records BEFORE sending them back to the browser...
• make the 'add templates' load for the initial startup use a bulk add versus individual add

[Cingulara]

Code is in the Template Controller to be fixed still. The return Ok(Templates) needs to kill all the "rawChecklist" fields and make them the empty string BEFORE we send it back. We do this in the "list all the checklists for this system package" just missed it for Templates.

  [HttpGet]
        [Authorize(Roles = "Administrator,Reader,Editor,Assessor")]
        [ResponseCache(Duration = 60, Location = ResponseCacheLocation.Any)]
        public async Task<IActionResult> ListTemplates()
        {
            try {
                _logger.LogInformation("Calling ListTemplates()");
                IEnumerable<Template> Templates;
                Templates = await _TemplateRepo.GetAllTemplates();
                _logger.LogInformation("Called ListTemplates() successfully");
                return Ok(Templates);
            }
            catch (Exception ex) {
                _logger.LogError(ex, "ListTemplates() Error listing all Templates and deserializing the checklist XML");
                return BadRequest();
            }
        }

I added this in there and made a cingulara/openrmf-api-template:1.10.02 Template API if you want to use that in your HELM chart:

                _logger.LogInformation("Calling ListTemplates()");
                IEnumerable<Template> Templates;
                Templates = await _TemplateRepo.GetAllTemplates();
                if (Templates != null) {
                    foreach(Template t in Templates) {
                        t.rawChecklist = ""; // clear out contents for the listing, we do not need these here
                    }
                }
                _logger.LogInformation("Called ListTemplates() successfully");
                return Ok(Templates);

[Cingulara]

This code is out there in that container, but needs to be in a packaged up version eventually...so keeping open.

[Cingulara]

This is in the latest v1.10 Template API image and will be done in 1.11 as well. It was sending the XML raw data which it did not need to do.