hygieia / hygieia

CapitalOne DevOps Dashboard
http://hygieia.github.io/hygieia
Apache License 2.0
3.79k stars 1.8k forks source link

Latest jira collector issue with story status #2304

Closed sjalee82 closed 6 years ago

sjalee82 commented 6 years ago

Hi i have an issue with the JIRA collector. when querying the mongodb for the sStatus and sState, all of the tickets are set to Backlog, even though the tickets/issues have other states. Any idea?

nireeshT commented 6 years ago

@sjalee82 when is it last collected data , share jira properties.

subodhbattina commented 6 years ago

@sjalee82 : Please validate the data that you are getting from the JIRA API:

  1. https://jira-server/rest/agile/1.0/board?name= Get the board id from the response, and then use that to get the issue data using the API below

  2. https://jira-server/rest/agile/1.0/board//issue See what the statuses are on the response. If there is a discrepancy, as nireeshT suggested, check your properties file as well. This is what I had at a minimum when I tested it last time.

feature.pageSize=100
feature.cron=0 *
feature.queryFolder=jiraapi-queries
feature.storyQuery=story
feature.epicQuery=epic
feature.jiraProxyUrl=
feature.jiraProxyPort=
feature.jiraBaseUrl=
feature.jiraQueryEndpoint=rest/api/2/
feature.jiraCredentials=
feature.jiraIssueTypeNames=Story
feature.jiraSprintDataFieldName=customfield_10007
feature.jiraEpicIdFieldName=customfield_10003
feature.jiraStoryPointsFieldName=customfield_10004
feature.jiraTeamFieldName=customfield_11248
feature.jiraBoardAsTeam=true

feature.deltaStartDate=
feature.masterStartDate=
feature.deltaCollectorItemStartDate=

sjalee82 commented 6 years ago

i get the same response from both rest queries Below is my configruation:

feature.pageSize=1000 feature.cron=0 * feature.queryFolder=jiraapi-queries feature.storyQuery=story feature.epicQuery=epic feature.jiraProxyUrl= feature.jiraProxyPort= feature.jiraBaseUrl= feature.jiraQueryEndpoint=rest/api/2/ feature.jiraCredentials= feature.jiraIssueTypeNames=Task,Bug,Sub-task,Story,Spike,Production Support,New Feature,Improvement feature.jiraSprintDataFieldName=customfield_10104 feature.jiraEpicIdFieldName=customfield_10100 feature.jiraStoryPointsFieldName=customfield_10106 feature.jiraTeamFieldName=#do not have Team feature.jiraBoardAsTeam=false

feature.deltaStartDate=2018-03-01T00:00:00.000000 feature.masterStartDate=2018-01-01T00:00:00.000000 feature.deltaCollectorItemStartDate=2008-01-01T00:00:00.000000

sjalee82 commented 6 years ago

Excerpt from MongoDB: "_id" : ObjectId("5b2417b836d1a0000b7935d3"), "_class" : "com.capitalone.dashboard.model.Feature", "collectorId" : ObjectId("5b22889230c49e000c0a8cf1"), "sId" : "10126", "sNumber" : "DP-38", "sName" : "deleted", "sTypeId" : "10001", "sTypeName" : "Story", "sStatus" : "Backlog", "sState" : "Backlog", "sEstimate" : "5", "sEstimateTime" : 0, }

response from rest/api/latest/issue/DP-38 "expand": "renderedFields,names,schema,operations,editmeta,changelog,versionedRepresentations", "id": "10126", "self": "http://:8080/rest/api/latest/issue/10126", "key": "DP-38", "fields": { "issuetype": { "self": "http://:8080/rest/api/2/issuetype/10001", "id": "10001", "description": "Created by JIRA Software - do not edit or delete. Issue type for a user story.", "iconUrl": "http://:8080/images/icons/issuetypes/story.svg", "name": "Story", "subtask": false }, "updated": "2018-06-18T13:24:09.960+0000", "status": { "self": "http://:8080/rest/api/2/status/10101", "description": "", "iconUrl": "http://:8080/images/icons/statuses/generic.png", "name": "In-Progress", "id": "10101", "statusCategory": { "self": "http://:8080/rest/api/2/statuscategory/4", "id": 4, "key": "indeterminate", "colorName": "yellow", "name": "In Progress" } },

subodhbattina commented 6 years ago

@sjalee82: This could be a configuration issue in your properties file

Can you please call the api below, to get the story information for your board, and then in that response json, there is a section as below (from what I got last time I worked on it) that lists the field names. Make sure your properties file has the correct field name configuration. You will have to update the query parameters as needed.

https://{jira server domain name}/rest/agile/1.0/board/{board-id}/issue?jql=updatedDate%3E%3D'2016-03-01+00:00'+AND+issueType+IN+('Story')+ORDER+BY+updated+ASC&expand=schema,names&fields=*all,-comment,-watches,-worklog,-votes,-reporter,-creator,-attachment&maxResults=100&startAt=0

"names": { "customfield_15521": "AE", "customfield_15400": "Approvals", "customfield_13100": "Time Scope Variability", "customfield_17700": "Definition of Done", "customfield_15522": "Bank Priority", "customfield_15401": "Satisfaction", "customfield_15520": "Requesting LOB", "customfield_13101": "JSD EazyBI Account Name", "customfield_15525": "Intent Lead", "resolution": "Resolution", "customfield_15526": "Overall Status", "customfield_15402": "Satisfaction date", "customfield_15523": "Complexity", "customfield_15524": "Delivery Model", "customfield_17701": "To Do", "customfield_12800": "System Team Area", "customfield_15518": "Operations Area Goal Alignment", "customfield_15519": "Operations Imperatives", "customfield_12802": "Customer Priority Justification (if Medium, High & Critical)", "customfield_15516": "Key Initiative", "customfield_12801": "Customer/LOB Priority", "customfield_15517": "LOB Impacts", "customfield_12804": "Sprint Requested", "customfield_12803": "EASE WEB Environment (What Environment Does this Exist in?)", "customfield_12806": "Additional Assignee #1", "customfield_12805": "Feature mapping to Test Case", "customfield_12808": "Resolution Area (Where was issue Resolved)", "customfield_12807": "External Dependency", "customfield_12809": "If External Resolve: What External Team or Teams Owned & Resolved?", "customfield_18100": "Target start", "customfield_18101": "Target end", "lastViewed": "Last Viewed", "customfield_14300": "Squad", "customfield_15510": "Estimated Benefit - Baseline", "customfield_12000": "Software Cap Type", "customfield_16600": "CSGE Resource Type", "customfield_14301": "Impediment Reason", "customfield_15511": "Estimated Benefit - Current", "customfield_15514": "Milestone Delta", "customfield_15515": "CR Type", "customfield_15512": "Estimated Total Benefit ($)", "labels": "Labels", "customfield_15513": "Estimated Total Cost ($)", "customfield_15507": "Current End Date", "customfield_15508": "Current Start Date", "customfield_15505": "Benefit Achievement Date Current", "customfield_15506": "Current Due Date", "aggregatetimeoriginalestimate": "Σ Original Estimate", "customfield_15509": "RTG Date", "issuelinks": "Linked Issues", "assignee": "Assignee", "components": "Component/s", "customfield_17000": "Region", "customfield_15500": "Benefit Categories", "customfield_13200": "JSD PluginType", "customfield_15504": "Benefit Achievement Date Baseline", "customfield_15501": "Baseline Due Date", "customfield_17800": "Feature Readiness", "customfield_12227": "Volume", "customfield_10600": "Rank", "customfield_14405": "Requesting IT Director", "subtasks": "Sub-Tasks", "customfield_12220": "Date/Time ISO Lead Review", "customfield_14400": "Software Cap Asset Name", "customfield_12222": "Date/Time Approved", "customfield_12101": "Release Name Introduced", "customfield_12221": "Date/Time ISRM Review Complete", "customfield_12100": "Application", "customfield_14403": "Original Estimate", "customfield_12224": "Point of Contact", "customfield_14404": "Requesting Team Product Owner", "customfield_12223": "Date/Time Retired", "customfield_12102": "Scope", "customfield_14401": "Requesting Team Name", "customfield_12226": "Real Time", "customfield_16701": "R&D Tax Credit Applicable", "customfield_14402": "Accountable Executive/Stakeholder", "customfield_12225": "Batch", "customfield_16700": "Total SG's", "customfield_11248": "Team", "customfield_12219": "Date/Time Peer Review", "progress": "Progress", "issuetype": "Issue Type", "customfield_17100": "Risk Resolution (ROAM)", "project": "Project", "customfield_13300": "JSD eazyBI Description", "customfield_11000": "Feature Label", "customfield_11001": "Testing Work Stream", "customfield_11002": "Testing Cycle", "customfield_13301": "Class of Service", "customfield_17901": "Compliance", "customfield_15600": "BAP CI", "customfield_11246": "Customer Email Address", "customfield_17900": "Legal", "customfield_15601": "Policy", "customfield_13303": "Current Efforts", "customfield_12214": "ISRM Risk Manager", "customfield_11237": "Is the product currently in market?", "customfield_12206": "Peer Review Assignee", "customfield_11238": "Product Lead", "customfield_10700": "Customer Request Type", "customfield_10701": "Request participants", "customfield_10702": "Time to resolution", "customfield_12209": "ISO Lead", "resolutiondate": "Resolved", "customfield_16000": "Incident Number", "customfield_12561": "Process Instructions (3)", "customfield_12560": "Process Instructions (2)", "customfield_12200": "Project Status (RYG)", "customfield_12202": "Forward to FAs", "customfield_12201": "Kanban", "customfield_12564": "LOB (Other)", "customfield_12558": "Date/Time Customer", "customfield_12557": "MS-Project ID", "customfield_10810": "Priority Reason", "customfield_11228": "Cost Center", "customfield_12559": "Process Instructions (1)", "customfield_10811": "Assigned PO", "customfield_10812": "Assigned Team", "customfield_10813": "LOE", "customfield_10814": "Closed Reason", "customfield_10815": "Blocked Reason", "updated": "Updated", "customfield_17201": "Supply Chain Manager", "customfield_17200": "Category Manager", "timeoriginalestimate": "Original Estimate", "customfield_12550": "Gantt Options", "customfield_17205": "Sourcing Intake Number", "description": "Description", "customfield_17204": "Contracting Intake Number", "customfield_12552": "Baseline End", "customfield_11220": "Feedback Rating", "customfield_17203": "Contract Signatory", "customfield_12551": "Baseline Start", "customfield_17202": "Contract Manager", "customfield_12554": "Baseline Effort", "customfield_10012": "Release Version History", "customfield_12553": "Date of Baselining", "customfield_13400": "Software Testing", "customfield_11224": "Is this story being copied from the \"Engagement Stories\" Project templates?", "customfield_11103": "Requested By", "customfield_12555": "Velocity %", "customfield_11104": "Deliverable URL", "timetracking": "Time Tracking", "customfield_12547": "Actual End", "customfield_10005": "Business Value", "customfield_12304": "Lessons Learned", "customfield_11216": "LOB (Unofficial)", "customfield_10006": "Rank (Obsolete)", "customfield_12549": "Latest End", "customfield_10007": "Sprint", "customfield_12548": "Latest Start", "customfield_10800": "Feature", "customfield_10801": "Found by", "customfield_11219": "Type of Feedback", "customfield_10802": "Date", "customfield_10803": "Steps to Reproduce", "customfield_10804": "Expected Results", "customfield_10805": "Actual Results", "customfield_10806": "Client Device", "customfield_10807": "App Build Version", "customfield_10808": "Environments", "customfield_10809": "Impact/Severity", "summary": "Summary", "customfield_16101": "Business Functional Team", "customfield_16100": "Use Case(s)", "customfield_12541": "Target Production go-live date", "customfield_10000": "[CHART] Time in Status", "customfield_11210": "Peer Sign-off", "customfield_12540": "Provide link to project workspace (KL, GitHub, OneDrive, etc)", "customfield_16102": "Model Change Type", "customfield_10001": "[CHART] Date of First Response", "customfield_11211": "Cascading Select", "customfield_12543": "Customer SME", "customfield_11212": "TEST Field Type \"Labels\"", "customfield_12542": "Date/Time AE Review Complete", "customfield_10002": "Flagged", "customfield_11213": "User Field", "customfield_10003": "Epic/Theme", "customfield_12544": "Application name", "customfield_14600": "Gumbo PI Issues", "customfield_10004": "Story Points", "customfield_12536": "Data Elements", "customfield_11204": "QA Completed On", "customfield_12535": "Data Classification", "customfield_11205": "Total Days Spent", "customfield_12414": "Needs Enhancement", "customfield_12538": "Where is application hosted?", "customfield_11206": "End Date", "environment": "Environment", "customfield_15802": "Dependency Due Date", "customfield_12537": "Third Parties Involved?", "customfield_11207": "Testing Notes status", "customfield_15803": "Theme", "customfield_11208": "Link to play", "customfield_12539": "Are there any high-risk transactions involved?", "customfield_11209": "Frustration", "duedate": "Due Date", "customfield_15000": "Program Increment List", "customfield_12530": "ASV Name", "fixVersions": "Fix Version/s", "customfield_12532": "ART PI", "customfield_13500": "Engagement Level", "customfield_12411": "ALM_Severity", "customfield_11200": "Effort", "customfield_15800": "Dependency Status", "customfield_12410": "ALM_Sprint End Date Introduced", "customfield_12531": "ART Name", "customfield_11201": "Incorporated Version", "customfield_15801": "Dependency Delivery Description", "customfield_12534": "Internal or External facing?", "customfield_11202": "Founder Sign-off", "customfield_11203": "Morale", "customfield_12404": "Program Increment", "customfield_12403": "Epic Type", "customfield_12406": "Workstream", "customfield_12405": "Feature/Enabler", "customfield_12408": "ALM_Application", "customfield_10901": "Developer(s)", "customfield_12407": "Planned Start", "customfield_10902": "QA", "customfield_12409": "ALM_Release Name Introduced", "customfield_16200": "Start Date", "customfield_16201": "Progress", "customfield_12400": "Expected Remediation Date", "priority": "Priority", "customfield_12402": "Planned End", "customfield_16206": "Risk probability", "customfield_16205": "Risk consequence", "customfield_15901": "Time to first response", "customfield_12515": "Risk Issue Title", "customfield_12518": "Risk Issue Type", "timeestimate": "Remaining Estimate", "versions": "Affects Version/s", "customfield_12519": "Risk Issue Description:", "status": "Status", "customfield_15100": "TIR Updated?", "customfield_11300": "Phase Found", "customfield_15900": "Approvers", "customfield_13601": "Incident Severity", "customfield_13600": "Incident Type", "customfield_12624": "CLI Project Key", "customfield_11414": "Time in Status", "customfield_11413": "Acknowledged Date", "customfield_12502": "Targeted UATE migration date", "customfield_12625": "Dependencies", "aggregatetimeestimate": "Σ Remaining Estimate", "customfield_16303": "Regulatory Impact", "customfield_16302": "Global Finance Required", "customfield_16301": "IT Required", "customfield_16300": "Audit Driven", "aggregateprogress": "Σ Progress", "customfield_11531": "Date/Time Complete", "customfield_11410": "System Impact: Secondary", "customfield_11530": "Date/Time In Progress", "customfield_14800": "Rank Number", "customfield_11412": "System Impact: Additional", "customfield_12501": "Targeted SITE migration date", "customfield_12622": "Production Deployment Date", "customfield_11411": "System Impact: Tertiary", "customfield_12621": "EIDs Needing EazyBI access", "customfield_11524": "Additional Stakeholders", "customfield_11523": "Primary KPI", "customfield_13701": "Parent Link", "customfield_11526": "Division (ISRM)", "customfield_11525": "Accountable Executive (AE)", "customfield_11407": "VersionOne ID", "customfield_11527": "Task Details", "customfield_11406": "Is issue in VersionOne?", "customfield_11409": "System Impact: Primary", "customfield_11529": "ISRM POC", "customfield_11408": "Prod Deploy Date", "timespent": "Time Spent", "customfield_15200": "Delivery Status", "customfield_17500": "Release Train", "aggregatetimespent": "Σ Time Spent", "customfield_13700": "Team", "customfield_11522": "URL", "customfield_14902": "Primary CTQ", "customfield_12602": "NIST Alignment", "customfield_11634": "Impact", "customfield_11633": "Date/Time Canceled", "customfield_14900": "Portfolio Plan Name", "customfield_12604": "Investment Estimate", "customfield_11515": "Hypothesis", "customfield_14901": "Sub-Portfolio Plan Name", "customfield_12603": "Epic Owner", "customfield_11635": "Next Steps", "customfield_11514": "Problem Statement", "customfield_11517": "Undesired Metrics", "customfield_12606": "WSJF (Weighted Shortest Job First)", "customfield_11516": "Secondary KPI", "customfield_12605": "Funded", "customfield_12608": "Expected Value", "customfield_12607": "Additional Notes", "customfield_11518": "Reporters", "workratio": "Work Ratio", "created": "Created", "customfield_14100": "Exposure", "customfield_16400": "Change Order", "customfield_14101": "Potential Loss Avoided (PLA)", "customfield_11630": "Risk Level", "customfield_11632": "ISRM - Prime ID", "customfield_12600": "Systems Impacted", "customfield_11631": "Risk Response Rationale", "customfield_11623": "Impediment Risk", "customfield_11502": "eID", "customfield_13800": "Flagged reason", "customfield_12835": "All Tickets EM & TDM", "customfield_11504": "JSD How much money and time saved?", "customfield_11625": "LOB (Cyber)", "customfield_11503": "JSD How many teams would use?", "customfield_11506": "JSD Is LOB willing to pay?", "customfield_11627": "Risk Issue Description", "customfield_11626": "Risk ISRM Area", "customfield_11505": "JSD Are there alternatives?", "customfield_11508": "Category", "customfield_11629": "Risk Issue Type", "customfield_11507": "JSD Does JIRA Sponsor Support?", "customfield_11628": "Risk Issue Title", "customfield_11509": "On Behalf Of", "customfield_15540": "Status Summary", "customfield_15543": "Intent Requestor", "customfield_13001": "EASE Web Tech Service Desk Request Type", "customfield_13000": "Deploy/Toggle Type", "customfield_17600": "Acceptance Criteria (AC):", "customfield_15541": "Upcoming Milestones", "customfield_15542": "Intent Approver", "customfield_13002": "Design Skills", "customfield_15545": "Owner", "customfield_15303": "Requirement Status", "customfield_15546": "Delivery Approver", "customfield_11620": "Automation Status", "customfield_12701": "Applicable Stakeholders", "customfield_12822": "NIST Category", "customfield_12700": "Project Category", "customfield_15538": "Objectives", "customfield_12824": "JSD Rename EazyBI account", "customfield_15539": "Resolution Plan", "customfield_12826": "Goals", "customfield_12825": "JSD Group", "customfield_12828": "Stage(s)", "customfield_12827": "Required Resources (ORM)", "customfield_11619": "Client/Device", "customfield_12829": "Division ORM", "customfield_18000": "Risk Impact Advisors", "customfield_15532": "Strategic Alignment", "customfield_16501": "Incident", "customfield_15533": "Sub-Portfolio", "customfield_16500": "Benefit Category", "customfield_15530": "Service Impact", "customfield_15531": "Sponsor", "customfield_15536": "Impacts of Change", "customfield_15537": "Key Accomplishments", "customfield_12820": "Automation Closing Access ticket", "customfield_15534": "Trend", "customfield_15535": "Business Justification", "customfield_10400": "Epic Link", "customfield_15529": "Schedule", "customfield_13900": "Service Type", "customfield_12811": "System Team Comments", "customfield_12810": "External Resolver Team Comments", "customfield_15527": "Portfolio", "customfield_13902": "Sub-LOB / Portfolios (CML TECH)", "customfield_15528": "Resources", "customfield_13901": "Acceptance Criteria", "customfield_12812": "External Resolver Team Priority", "customfield_12815": "System Team Priority", "customfield_13904": "ValidatorBridge", "customfield_13903": "ValidatorNameEID", "customfield_12817": "Pending reason", "customfield_13906": "Type Of Role", "customfield_12816": "System Team Status/Resolution", "customfield_13905": "ValidationSteps", "customfield_12819": "EazyBI Role Request", "customfield_13907": "AWS Account(s)", "customfield_12818": "Reference" }

subodhbattina commented 6 years ago

@sjalee82 : Since we didn't hear back from you, I am assuming this issue is resolved at your end, and therefore closing this issue. Thanks !

sandipbhaumik commented 5 years ago

Hi All, I am new to this api and I have one issue in our project. Currently, I am facing the issue for collecting all jira items for a specific project. Sometimes jira collectors are not able to collect all the data. I want to understand the three parameters here, feature.deltaStartDate=2018-03-01T00:00:00.000000 feature.masterStartDate=2018-01-01T00:00:00.000000 feature.deltaCollectorItemStartDate=2008-01-01T00:00:00.000000

Also, how jira collectors collect data after any exception occurs in midway? Is it atomic? Please help me to understand.

Thanks, Sandip