search

DeployHubProject / DeployHub-Pro

DeployHub Pro Pipeline Status Project
https://www.openmakesoftware.com/application-release-automation-for-continuous-delivery/
Other
11 stars 4 forks source link

DeployHub Reporting from GUI and API #290

Closed piyush94 closed 3 years ago

piyush94 commented 4 years ago

Hi, Is there a way to get some report of the Application deployments with details such as components used, environment info, etc. in JSON and/or excel, CSV or other formats, from DeployHub. Earlier there was a way to export some information from deploy screen, but i am not able to find it anymore.

CC: @svisagan83

sbtaylor15 commented 4 years ago

@piyush94 we changed the home icon button to switch to the domains view and then display the to do list, timeline and global reports. Select the global reports tab for the list of current reports. Let us know if we need to add any additional reports. Please pull the latest image for this update.

CC: @svisagan83

piyush94 commented 4 years ago

@sbtaylor15 - clicking on home is navigating to Domains section but the tabs you mentioned are not visible. Those tabs flash for a second and it goes to Domains screen.

CC: @svisagan83

piyush94 commented 4 years ago

@sbtaylor15 - any update regarding this? Thanks.

CC: @svisagan83

piyush94 commented 4 years ago

@sbtaylor15 - what we are trying to achieve is to extract the information of an application version, such as component details, etc. Initially we are looking at helm chart components. So we need to fetch the chartrepo, chartname and version from the component items. Then if possible extract the container images and tags from the helm chart. Let us know, if you have any suggestions regarding this.

Also if you could let me the know if there is any API docs/explorer for DeployHub, that would be great.

Thanks.

CC: @svisagan83

sbtaylor15 commented 4 years ago

@piyush94 we figured out the flashing screen for the reports. Please grab the latest image.

We have added the following APIs. They are still being refined a bit. Take a look at the data and let us know if this will give you what you want. The APIs will be added to swagger doc once they are finalized. The current API doc is here.

/dmadminweb/API/groups
/dmadminweb/API/users
/dmadminweb/API/categories
/dmadminweb/API/comptypes
/dmadminweb/API/credentials
/dmadminweb/API/endpoints
/dmadminweb/API/environments
/dmadminweb/API/repositories
/dmadminweb/API/datasources
/dmadminweb/API/buildengines
/dmadminweb/API/buildjobs
/dmadminweb/API/notifiers
/dmadminweb/API/components
/dmadminweb/API/applications
/dmadminweb/API/releases

add ?user=<userid>&pass=<password> to the end of each API url. The data returned will be what the userid being used has access to. You will need to pull the latest image for this.

CC: @svisagan83

sbtaylor15 commented 4 years ago

@piyush94 we changed the end point and simplified the data being returned. The data will include the attributes and properties for each object if they have any.

/dmadminweb/API/list/categories /dmadminweb/API/list/comptypes /dmadminweb/API/list/credentials /dmadminweb/API/list/environments /dmadminweb/API/list/groups /dmadminweb/API/list/repositories /dmadminweb/API/list/endpoints /dmadminweb/API/list/users /dmadminweb/API/list/components /dmadminweb/API/list/applications

Please pull that latest image for this update.

CC: @svisagan83

piyush94 commented 4 years ago

@sbtaylor15 - I am not able to find API doc for the above mentioned APIs. Could you let me know, how to use these?

CC: @svisagan83

piyush94 commented 4 years ago

@sbtaylor15 - Is it possible to have an API that returns the calculated YAML file for an application for the associated components and a selected environment. This is same as what is calculated during the Deploy action for an application. We need this to know the deployment configuration of a component. One more thing could be if the file can be stored after running Deploy action and dowloaded at a later time.

Let me know if we need to have a call for this.

CC: @svisagan83

sbtaylor15 commented 4 years ago

@piyush94 we are in the process of updating the swagger API doc for these new end points. Also, we need to have a call on the YAML file.

CC: @svisagan83

piyush94 commented 4 years ago

@sbtaylor15 - Thanks. Are there API endpoints for deleting objects such as application version, component version, environment and endpoint?

CC: @svisagan83

sbtaylor15 commented 4 years ago

@piyush94 there are APIs for deleting endpoints, environments, and credentials.
Rest API Doc

Let me check on application versions and component versions.

CC: @svisagan83

sbtaylor15 commented 4 years ago

@piyush94 The APIs are there for components and applications just not documented. They follow the same format as the other ones.

GET /dmadminweb/API/del/component/{comp_id} GET /dmadminweb/API/del/application/{app_id}

FYI, we are checking the swagger doc since it appears to be cut off.

CC: @svisagan83

piyush94 commented 4 years ago

@sbtaylor15 thanks, delete is working.

CC: @svisagan83

piyush94 commented 4 years ago

@sbtaylor15 Hi, inlist/componentswe are getting below error. Also, does this list/* paths accept query params like domain, etc.? component id 766 doesn't have any componentvars entry.

getDefectsForComponent(766)
In getAttributesForObject - sql = SELECT v.name, v.value, v.arrayid, v.nocase FROM dm.dm_componentvars v WHERE v.compid = ? ORDER BY 1
Finished retrieving rows
Exception caught, e=null
java.lang.NullPointerException
        at dmadmin.model.DMObject.getDomain(DMObject.java:105)
        at dmadmin.model.DMObject.getFullName(DMObject.java:84)
        at dmadmin.model.ComponentItem.toJSON(ComponentItem.java:386)
        at dmadmin.API.exportJSONForComponent(API.java:389)
        at dmadmin.API.doGet(API.java:3704)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1457)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
java.lang.NullPointerException
        at dmadmin.model.DMObject.getDomain(DMObject.java:105)
        at dmadmin.model.DMObject.getFullName(DMObject.java:84)
        at dmadmin.model.ComponentItem.toJSON(ComponentItem.java:386)
        at dmadmin.API.exportJSONForComponent(API.java:389)
        at dmadmin.API.doGet(API.java:3704)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1457)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
{"success":false,"error":"java.lang.NullPointerException"}
sbtaylor15 commented 4 years ago

@piyush94 we changed the code to just get the name of the component item instead of trying to get the domain it belongs to. Component Items are really associated to a Component and not to Domain.

As for the list/* question, there is no filtering other than what the userid making the API call has access to. Something like jq could filter the json output.

Please pull the latest image.

piyush94 commented 4 years ago

@sbtaylor15 Thanks, it's working now. Can you please add the domain field in the JSON schema for list/* where it's missing? It will be easier to parse then.

sbtaylor15 commented 4 years ago

@piyush94 - Do you want the domain as a separate field or part of the object name?

piyush94 commented 4 years ago

@sbtaylor15 - As a field in each object.

sbtaylor15 commented 4 years ago

@piyush94 - I added the domain field, but haven't had a chance to test it. Should be good to go, simple change. Pull the latest image. Thanks, Steve

piyush94 commented 4 years ago

@sbtaylor15 - Thanks for adding the Domain field. Not sure if this is related, but in list/endpoints, I am getting below error. I hadn't accessed this API earlier, so don't know if the error was there or not. It's trying to access a domain to which the user doesn't have access.

Finished retrieving rows
In getAttributesForObject - sql = SELECT v.name, v.value, v.arrayid, v.nocase FROM dm.dm_servervars v WHERE v.serverid = ? ORDER BY 1
Finished retrieving rows
In getAttributesForObject - sql = SELECT v.name, v.value, v.arrayid, v.nocase FROM dm.dm_servervars v WHERE v.serverid = ? ORDER BY 1
Finished retrieving rows
101 is not a valid domain for user 306
Exception caught, e=null
java.lang.NullPointerException
        at dmadmin.API.exportJSONForServer(API.java:4764)
        at dmadmin.API.doGet(API.java:3708)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1457)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
java.lang.NullPointerException
        at dmadmin.API.exportJSONForServer(API.java:4764)
        at dmadmin.API.doGet(API.java:3708)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1457)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
{"success":false,"error":"java.lang.NullPointerException"}
sbtaylor15 commented 4 years ago

@piyush94 - The problem was similar to component item issue where it was trying to derive a domain name for a component/server type. Removed the domain name from the type. Please pull the latest.

piyush94 commented 4 years ago

@sbtaylor15 - Thanks, it's working now.