AngelaFaulding commented 2 years ago

As discussed with James. Several times Kate and I have tried to create a branch, it will show that the process is working: However, it never shows a message to say that the branch has been created. Mauro crashes and usually the branch appears the next time we log in. I created a branch yesterday and it hasn't appeared today. Can this be investigated please? Should we have a message to say that the branch has been created successfully like we do with other items, e.g.

AngelaFaulding commented 2 years ago

I have created a new branch to test this just now: Test Change Request 25/5/22

jamesrwelch commented 2 years ago

I've been through the logs and found a reason for failure - we'll take a look and fix asap. It appears to be related to #442

AngelaFaulding commented 2 years ago

Ok thanks. Just to note, the new branch has failed:

Server Error

We're sorry, but the server responded with an error message.

Details

{
  "headers": {
    "normalizedNames": {},
    "lazyUpdate": null
  },
  "status": 502,
  "statusText": "Proxy Error",
  "url": "https://modelcatalogue.cs.ox.ac.uk/nhsd-datadictionary/api/versionedFolders/5ca6ec5d-f874-4f20-bb66-282b39c522f1/newBranchModelVersion",
  "ok": false,
  "name": "HttpErrorResponse",
  "message": "Http failure response for https://modelcatalogue.cs.ox.ac.uk/nhsd-datadictionary/api/versionedFolders/5ca6ec5d-f874-4f20-bb66-282b39c522f1/newBranchModelVersion: 502 Proxy Error",
  "error": "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>502 Proxy Error</title>\n</head><body>\n<h1>Proxy Error</h1>\n<p>The proxy server received an invalid\r\nresponse from an upstream server.<br />\r\nThe proxy server could not handle the request<p>Reason: <strong>Error reading from remote server</strong></p></p>\n</body></html>\n"
}

AngelaFaulding commented 2 years ago

I tried this on Friday and received: Server Error We're sorry, but the server responded with an error message.

Details { "headers": { "normalizedNames": {}, "lazyUpdate": null }, "status": 502, "statusText": "Proxy Error", "url": "https://modelcatalogue.cs.ox.ac.uk/nhsd-datadictionary/api/versionedFolders/5ca6ec5d-f874-4f20-bb66-282b39c522f1/newBranchModelVersion", "ok": false, "name": "HttpErrorResponse", "message": "Http failure response for https://modelcatalogue.cs.ox.ac.uk/nhsd-datadictionary/api/versionedFolders/5ca6ec5d-f874-4f20-bb66-282b39c522f1/newBranchModelVersion: 502 Proxy Error", "error": "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n\n502 Proxy Error\n\n

Proxy Error

\n

The proxy server received an invalid\r\nresponse from an upstream server.
\r\nThe proxy server could not handle the request

Reason: Error reading from remote server

\n\n" } James said the "The proxy error means it's timed-out - it's been to slow to respond and the UI has given up waiting. The new branch may well appear later. I'd also noticed some slow-down - I'll investigate. The 'asynchronous' option will help here... you won't get the error message as the new branch will be created in the background and you'll be able to carry on working as before."

The branch did appear eventually but will be confusing to new users if this continues when we are working in Mauro.

James said - it shouldn't be taking that long and he will look into it.

@jamesrwelch - I am not moving this to assessed as it still isn't working correctly. Does this need moving back into "To Do"?

AngelaFaulding commented 2 years ago

James agreed that we can't accept this as it is not working correctly and agreed that I would move it back to In Progress.

AngelaFaulding commented 2 years ago

@jamesrwelch - there are still issues with creating branches as below.

I tried creating a branch at 8:15 this morning called "National Eye Colour Data Set Version 1". It is now 3pm and the branch is not there. There is a branch called "Next Release" that wasn't there earlier. The history shows something happened at 8:29:

I have since tried to create a another new branch and received:

Not Found

We're sorry, but the server returned a 'Not Found' error

Details

{
  "headers": {
    "normalizedNames": {},
    "lazyUpdate": null
  },
  "status": 403,
  "statusText": "Forbidden",
  "url": "https://modelcatalogue.cs.ox.ac.uk/nhsd-datadictionary/api/properties",
  "ok": false,
  "name": "HttpErrorResponse",
  "message": "Http failure response for https://modelcatalogue.cs.ox.ac.uk/nhsd-datadictionary/api/properties: 403 Forbidden",
  "error": "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>403 Forbidden</title>\n</head><body>\n<h1>Forbidden</h1>\n<p>You don't have permission to access this resource.</p>\n<hr>\n<address>Apache/2.4.54 (Ubuntu) Server at modelcatalogue.cs.ox.ac.uk Port 443</address>\n</body></html>\n"
}

I will see what is there in the morning after I restart my computer. I have logged off Mauro several times today with no success.

I'm not sure if it is related but I can't get into the Orchestrator:

jamesrwelch commented 2 years ago

I created the 'next release' version this morning. I'll investigate the other issues this evening

AngelaFaulding commented 2 years ago

That is odd as when I first when in there was a test branch and then later on it was next release.

jamesrwelch commented 2 years ago

Yes - I named it ‘test branch’ after I created it, and subsequently remembered I’d just created ‘main’, and not a branch, so I renamed it a little while later

AngelaFaulding commented 2 years ago

That explains that. Still not sure what happened to my branch. It may turn up overnight.

jamesrwelch commented 2 years ago

The orchestrator is now back up and running - some config I forgot to copy across onto my new laptop when I built it and deployed it.
Looking through the logs, I can see the new branch failed for some reason - I'll investigate when I'm back in the office tomorrow.

jamesrwelch commented 2 years ago

(For the orchestrator, you'll need to refresh the page in your browser so it can load the new config)

jamesrwelch commented 1 year ago

New code for branch creation and deletion is in progress in a separate branch of the mdm-core code.

AngelaFaulding commented 7 months ago

Cannot create a branch: { "headers": { "normalizedNames": {}, "lazyUpdate": null }, "status": 500, "statusText": "OK", "url": "https://modelcatalogue.cs.ox.ac.uk/nhsd-datadictionary/api/versionedFolders/5828c961-57c3-4f63-8f46-133f76432224/newBranchModelVersion", "ok": false, "name": "HttpErrorResponse", "message": "Http failure response for https://modelcatalogue.cs.ox.ac.uk/nhsd-datadictionary/api/versionedFolders/5828c961-57c3-4f63-8f46-133f76432224/newBranchModelVersion: 500 OK", "error": { "status": 500, "reason": "Internal Server Error", "errorCode": "DMS01", "message": "Copied DataModel is invalid\n Property [label] with value [Classes and Attributes] must be unique by branch name :: [label]", "path": "/api/versionedFolders/5828c961-57c3-4f63-8f46-133f76432224/newBranchModelVersion", "version": "5.4.0-SNAPSHOT", "exception": { "type": "ApiInvalidModelException", "message": "Copied DataModel is invalid >> Property [label] with value [Classes and Attributes] must be unique by branch name :: [label]", "stacktrace": [ "uk.ac.ox.softeng.maurodatamapper.datamodel.DataModelService.$ttcopyModel(DataModelService.groovy:717)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.datamodel.DataModelService.$ttdataModelService_copyModel(DataModelService.groovy:685)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.core.container.FolderService$ttcopyModelsInFolder_closure65$_closure75.doCall(FolderService.groovy:474)", "uk.ac.ox.softeng.maurodatamapper.core.container.FolderService$ttcopyModelsInFolder_closure65.doCall(FolderService.groovy:467)", "uk.ac.ox.softeng.maurodatamapper.core.container.FolderService.$ttcopyModelsInFolder(FolderService.groovy:458)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.core.container.FolderService.$ttcopyFolderContents(FolderService.groovy:430)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.core.container.FolderService.$ttcopyFolderPass(FolderService.groovy:387)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.core.container.FolderService.$ttcopyFolder(FolderService.groovy:340)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.core.container.VersionedFolderService.$ttcopyVersionedFolder(VersionedFolderService.groovy:747)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.core.container.VersionedFolderService.$ttcopyFolderAsNewBranchFolder(VersionedFolderService.groovy:723)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.core.container.VersionedFolderService.$ttcreateNewBranchModelVersion(VersionedFolderService.groovy:497)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "uk.ac.ox.softeng.maurodatamapper.core.container.VersionedFolderController.$ttnewBranchModelVersion(VersionedFolderController.groovy:192)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "grails.gorm.transactions.GrailsTransactionTemplate$2.doInTransaction(GrailsTransactionTemplate.groovy:94)", "org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)", "grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)", "org.grails.core.DefaultGrailsControllerClass$MethodHandleInvoker.invoke(DefaultGrailsControllerClass.java:223)", "org.grails.core.DefaultGrailsControllerClass.invoke(DefaultGrailsControllerClass.java:188)", "org.grails.web.mapping.mvc.UrlMappingsInfoHandlerAdapter.handle(UrlMappingsInfoHandlerAdapter.groovy:90)", "org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071)", "org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964)", "org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)", "org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:920)", "org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)", "org.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:77)", "org.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:67)", "org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126)", "org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64)", "org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101)", "org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119)" ] } } }

James is working on this.

AngelaFaulding commented 3 months ago

This has been fixed and needs retesting.

AngelaFaulding commented 3 months ago

21/05/24 - it is still taking a long time to create a branch. Can this be checked please?

AngelaFaulding commented 2 months ago

I created a branch before 9am this morning and at 9:53 it still hasn't appeared on Mauro

pjmonks commented 2 months ago

@AngelaFaulding I'm assuming you create the branch as an asynchronous operation, because I've found the reference to it. This is the correct way of doing this, since the operation is very slow and you can immediately go back to the UI and continue other work.

To check the progress of background operations, you can click on your user name (top right hand corner) and select "Jobs":

This then lists the background jobs Mauro has and their current status:

Your branch creation job failed, unfortunately I don't know why from the unhelpful message:

I'd have to try and replicate this myself, please could you list the steps you carried out for me to try myself?

AngelaFaulding commented 2 months ago

These are the steps I followed:

Under the Most Recent Release folder:
Click on the latest finalised trunk
Click on the two pieces of paper to the right of the page, which displays "Create a New Version" which you hover over it
Select the drop down arrow to the right hand side of "Action"
Select New Branch
Add the Branch name, i.e. the CR number, name and version if appropriate, as below, (using the style shown: CR8000 - National Eye Colour Data Set Version 1)
Select Create in the background if you wish to work on another branch whilst the new branch is being created
Select Create Note that it will take several minutes to create the branch (this could be up to 20 minutes). During this time: If the "Create in background" box has not been ticked, a green line will show the process is working: If the "Create in background" box has been ticked, a message will be displayed to show that the branch is being created and you can work on other branches: Once the branch has been created, it will appear in the Most Recent Release folder (where the finalised trunk, that the branch was created from, is held).

I checked jobs and it gives a strange error regarding Classes and Attributes.

AngelaFaulding commented 2 months ago

Peter will look into this.

pjmonks commented 2 months ago

Looking into this, it's possible that I may have to ingest the branches again to fix the version links. Originally, the ingest would have created a versioned folder called "NHS Data Dictionary ([Month] [Year])", but that has been fixed to just "NHS Data Dictionary". Since the test environment now has a mixture of these, the error above is possibly due to that inconsistency.

I will first confirm with the rest of the team on testing status before removing all existing branches and ingesting again to see if that fixes the issue.

pjmonks commented 2 months ago

I have been able to remove and ingest all the branches again to the TEST environment. As I did this, I tested that more branches could be created. As you can see, I was able to now create a new "main" branch and a "pete-test" branch, alongside the in-flight branches:

I believe it is now possible to create branches as before, so testing can continue.

AngelaFaulding commented 2 months ago

1/7/24 - Branch created successfully. Can now test fully.

KatePalmerLilley commented 2 months ago

I created a branch 12/07/2024 branch was still running by 15/07/2024, I cancelled the job and created a new branch which was successful. However the original branch is still showing as running:

AngelaFaulding commented 1 month ago

I had just tried to create a test branch and it failed. I got them following. There is nothing in the job status. I will try to create another branch.

Server Timeout Error

We're sorry, but the server responded with an error message.

Details

{
  "headers": {
    "normalizedNames": {},
    "lazyUpdate": null
  },
  "status": 504,
  "statusText": "Gateway Time-out",
  "url": "https://mauro.dev.dataproducts.nhs.uk/api/versionedFolders/bbd215fc-06a8-4841-9546-f994188f216e/newBranchModelVersion",
  "ok": false,
  "name": "HttpErrorResponse",
  "message": "Http failure response for https://mauro.dev.dataproducts.nhs.uk/api/versionedFolders/bbd215fc-06a8-4841-9546-f994188f216e/newBranchModelVersion: 504 Gateway Time-out",
  "error": "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx/1.24.0 (Ubuntu)</center>\r\n</body>\r\n</html>\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n"
}

pjmonks commented 1 month ago

@AngelaFaulding For your branches, you will have to check the box that says "Create in the background". They are too big to complete a new branch creation in one go, so an asynchronous job must be started. This will mean the response is immediate, then you can wait for background jobs to complete.

At the moment, the branch may still be being created or not, but we can't tell. It usually takes 10 - 15 minutes to create a new branch, so check back then to see if anything happened.

AngelaFaulding commented 1 month ago

Thanks @pjmonks - it has now appeared but took a lot longer that 15 minutes to appear.

AngelaFaulding commented 1 month ago

I have created a branch this morning and it took less than 30 mins:

NHSDigital / DataDictionaryPublication

Creating a Branch #446

Server Error

Details

Proxy Error

Not Found

Details

Server Timeout Error

Details