Inconsistant data between index page and chart with two groups.

What you see may be different as your tracker will have different random data. I am using revision d30409c

Starting with URL:

http://localhost:8080/demo/issue?%40sort0=activity&%40sort1=&%40group0=status&%40group1=priority&%40columns=title%2Cid%2Cactivity%2Cstatus%2Cassignedto&%40filter=id&id=1%2C2%2C3%2C4%2C5%2C6%2C7%2C8&%40pagesize=50&%40startwith=0

I see:

When I click on show stacked barchart or multibar chart, the counts are off. As an example with multibarchart, I see:

The data should be the same as in the index and I should not have a bar for chatting wish, I should have a bar for resolved wish.

can @varshi-123 and @ashrith-UMB also check if the data is inconsistent between chats and index page because in my machine it looks fine and so that I am not able to find the bug in the code. Check it out and let me know if it is still a problem or not.

Hi Pratik:

I have updated my checkout and it now looks like:

team2$ git rev-parse --short HEAD
9da6738
team2$ git status
On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    demo/extensions/config.ini
    demo/html/pygal-tooltips.js

nothing added to commit but untracked files present (use "git add" to track)

I have deleted all cached .pyc files (and pycache directories).

If I run:

python3 ../roundup/scripts/roundup_server.py -p 8080 -n 0.0.0.0 demo=demo

and access the URL's above I still see the same problem.

Can you verify you are running the current main using the commands above and try the URL's and screen capture (one of the few time screen captures are useful) what you see for the URL's.

Also If I drill down by clicking on the bars, I get the expected issues listed in the table. (Note the bug from #86 causes many more issue to be shown. Fix it by adding ,id to the @filter parameter in the URL.)

Ok, here is the problem. Note that I only have one bar for everything except for critical types. At line 328 oc chart.py you have:

                # Add the inner dictionary to the chart
                chart.add(second_prop, [{'value': data['count'], 'xlink': data['xlink']} for first_prop, data in second_prop_dict.items()])

For all cases except critical the list comprehension [] returns only 1 item. not 2. So those bars for bug, feature and wish should be plotted on the resolved side of the chart. But because there is only one data item in the comprehension, it's plotted on the chatting side of the chart.

So for the critical case, add is passed:

[{'value': 1,
  'xlink': {'href': 'http://localhost:8080/demo/issue?@template=index&@columns=title,id,activity,status,assignedto&@sort=id&@group=status,priority&@pagesize=50&@startwith=0&id=1,2,3,4,5,6,7,8&@filter=status,priority&status=chatting&priority=critical',
            'target': '_parent'}},
 {'value': 2,
  'xlink': {'href': 'http://localhost:8080/demo/issue?@template=index&@columns=title,id,activity,status,assignedto&@sort=id&@group=status,priority&@pagesize=50&@startwith=0&id=1,2,3,4,5,6,7,8&@filter=status,priority&status=resolved&priority=critical',
            'target': '_parent'}}]

while bug is passed:

[{'value': 1,
  'xlink': {'href': 'http://localhost:8080/demo/issue?@template=index&@columns=title,id,activity,status,assignedto&@sort=id&@group=status,priority&@pagesize=50&@startwith=0&id=1,2,3,4,5,6,7,8&@filter=status,priority&status=resolved&priority=bug',
            'target': '_parent'}}]

and this produces the broken plot above. Note that the bar still links to the right place: status=resolved, priority=bug. It's just shifted because there is no empty data point for the first plot.

Hmm, it looks like an update to this issue I did last week went missing. In any case, the following code using (nested) defaultdicts for data_dict works and returns a 0 value for any set of indexes that are not present in the data_dict.

Call this with one argument, the path to the tracker home directory.

#! /usr/bin/env python

import pygal
import sys

from collections import defaultdict
from roundup import instance

# gather tracker home as first args
instance_home = sys.argv[1]

# open the instance
instance = instance.open(instance_home)
db = instance.open('admin')

grouping = [('+', 'status'), ('+', 'priority')]

# get the database class for the two group properties
# note this crashes if it's not a link/multilink
group1_db_class = db.getclass(db.issue.getprops()[grouping[0][1]].classname)
group2_db_class = db.getclass(db.issue.getprops()[grouping[1][1]].classname)

# Use a comprehension with lookup of properties
# produces (status = group1 grouping prop and priority:
# link_ids = [
#    (issue1_status_id, issue1_priority_id),
#    (issue2_status_id, issue2_priority_id),
#     ... ]
# so [ ("4","2"), ("1","2"),("3","4") ...]
issue_group_ids = [(db.issue.get(i, grouping[0][1]),
             db.issue.get(i,grouping[1][1]))
            for i in db.issue.filter(
                      ["1","2","3","4","5","6","7","8"], # limit to 8 items
                      {},
                      sort=[('+', 'id')],
                      group=grouping,
            )]

# The data_dict nested dictionary has a primary key of the group2,
# and a secondary key of group1:
#   data[group2 (priority)][group1 (status)]
# as that's the order we need to iterate over when plotting.
# The nested default dicts make sure that any reference to a non-existent
# entry data_dict['9']['99'] == 0 rather than throwing an error.
data_dict = defaultdict(lambda: defaultdict(int))

# group1_dict maps group1 id -> label. E.G.
#  group["1"] = "chatting"
group1_dict = {}  # must have ordered dict

# same but with second group labels.
group2_dict = {}  # must have ordered dict

# build the data dict:
for group1, group2 in issue_group_ids:
     data_dict[group2][group1] += 1

     if group2 not in group2_dict:
          group2_dict[group2] = group2_db_class.get(
               group2,
               group2_db_class.labelprop())
     if group1 not in group1_dict:
          group1_dict[group1] = group1_db_class.get(
               group1,
               group1_db_class.labelprop())

# Problem group2_dict (legend) labels are ordered by the occurance
# of that label within group1.
#
# Say status 'new', id '1' should be the first in legend order.  But
# group1 only has 'open' id '2' items in it. The legend will have
# 'open' before 'new'. If a different search is done and the first
# item in group1 has 'new' items, then the legend will be new, open....
#
# The legend order should be the same (minus statuses that aren't
# used in the dataset (e.g. if no 'resolved' status is shown, it should
# not be in the legend.
#
# Reorder group1_dict and group2_dict by using the group order from
# the filter grabbing all the id's in the class. These will be sorted
# using the direction for the selector and the orderprop.
# Use filter() to get the retired items as well.
#    grouping[X][0] = + or -
all_group1_ids = group1_db_class.filter(
     None,
     None,
     sort=(grouping[0][0], group1_db_class.orderprop()),
     retired=None)

all_group2_ids = group2_db_class.filter(
     None,
     None,
     sort=(grouping[1][0], group2_db_class.orderprop()),
     retired=None)

# reorder dicts
group1_dict = {i: group1_dict[i] for i in all_group1_ids if i in group1_dict}
group2_dict = {i: group2_dict[i] for i in all_group2_ids if i in group2_dict}

# formatting chart
bar_chart = pygal.Bar(show_minor_y_labels=False)
bar_chart.title = "Number of issues per %s per %s" % (
     grouping[1][1], grouping[0][1])
bar_chart.x_title = grouping[0][1]
bar_chart.x_labels = list(group1_dict.values())
bar_chart.y_title = grouping[1][1]

for plot_group2, plot_group2_name in group2_dict.items():
     plot_data = []
     for plot_group1 in group1_dict:
          plot_data.append({"xlink": "https://google.com?%s&%s" % (
               plot_group1,plot_group2),
                            "value": data_dict[plot_group2][plot_group1]})
     print(plot_data)
     bar_chart.add(plot_group2_name, plot_data)

bar_chart.render_to_file('test.svg')

UMB-CS682-Team-02 / tracker

Inconsistant data between index page and chart with two groups. #87