Test the piechart and other chart actions

[rouilj]

Roundup comes with a test suite. The export_csv action (Shown on the issue index page as "Download as CSV") is tested by the testCsvExport class in test/test_cgi.py.

A similar set of tests should be created for the piechart action. There are three steps needed for this:

1) the tracker created for testing needs to include chart.py. For local testing purposes, copy chart.py into share/roundup/templates/classic/extensions 2) insert the code below before the testCsvExport class 3) add the testChartGeneration to the class FormTestCase argument list after testCsvExport

The code below only tests the piechart in forward and reverse sorting on status with 5 issues. The charts under various scenarios test:

• the svg tags at the start/end
• that there is an element with title classes.

These are done using string matching. The rest of the test are done on the parsed XML by matching specific element or elements in specific locations.

• check that the first legend item matches the expected value. For pie charts, the first legend should be the one with the highest count. For bar charts the order is the normal order of the grouped items. The test should make sure that ascending/descending order is correct.
  • check the page title make sure class (issue, user ...) is in the title and the group order is reported.
  • check the values reported for the plotted bar/pie wedge

Extend the testing to include sorting by different properties (e.g. priority) and create the same tests for barcharts. Also check for other chart types (e.g. Horizontal bar ....).

Starting code:

class testChartGeneration(object):
    """Add this class to the argument list for FormTestCase.

       Otherwise the tests won't be loaded.
    """

    def createData(self):
        """Load the database with data to chart."""
        demo_id=self.db.user.create(username='demo', address='demo@test.test',
            roles='User', realname='demo')
        key_id1=self.db.keyword.create(name='keyword1')
        key_id2=self.db.keyword.create(name='keyword2')

        originalDate = date.Date
        dummy=date.Date('2000-06-26.00:34:02.0')
        # is a closure the best way to return a static Date object??
        def dummyDate(adate=None):
            def dummyClosure(adate=None, translator=None):
                return dummy
            return dummyClosure
        date.Date = dummyDate()

        self.db.issue.create(title='foo1', priority="1",
                             status='2', assignedto='4',
                             nosy=['3',demo_id])
        self.db.issue.create(title='bar2', priority="3",
                             status='1', assignedto='3',
                             keyword=[key_id1,key_id2])
        self.db.issue.create(title='bar3', priority="2",
                             status='1', assignedto='3',
                             keyword=[key_id1,key_id2])
        self.db.issue.create(title='baz32', status='4')

    def createClient(self, request_dict, classname='issue'):
        """Create a client with @group and filtering items.
           Pass in a dict like:

           { "@group"': "-status",
             "@filter": "status",
             "status": "2"
           }

           For these tests the key element is @group to handle sorting order.
           Optional argument 'classname' allows changing from 'issue' to
           another class like 'user'.
        """

        cl = self._make_client(request_dict,
            nodeid=None, userid='1')
        cl.classname = classname
        cl.request = MockNull()

        output = io.BytesIO()
        cl.request.wfile = output
        return cl

    def testPiechartForwardSortStatus(self):
        """Test the piechart action.

           Each test consists of creating the data,
           creating the client
           verifying the returned data.
        """
        self.createData()

        cl = self.createClient(
            {'@group': 'status', '@columns': 'id,title'}
            )

        ret = self.instance.cgi_actions['piechart'](cl).handle()

        # basic checks of the returned data
        # Does it start/end with svg tags?
        self.assertTrue(ret.startswith(b'<svg '))
        self.assertTrue(ret.endswith(b'</svg>\n'))
        # is there a class identifying the plot_title
        self.assertIn(b'class="title plot_title"', ret)

        # turn the svg xml into a dom
        # we can then assert things about the structure
        # of the dom that would be difficult to assert
        # using regular expressions
        import lxml.etree as etree
        ret_dom = etree.fromstring(ret)

        # since the order of sorting is positive by status,
        # the database's first issue has status 1 which is unread
        # This should be the first value in the legend.
        # by inspecting the xml, I found the legend in this case:
        # Remeber all lists start at 0.
        # In the 6th element (index 5) under svg
        #   in the 7th child element (index 6)
        #      in the first child
        #         in the first child
        #           extract the text
        self.assertEqual('unread',
            ret_dom[5].getchildren()[6].getchildren()[0].getchildren()[1].text)

        # This is the same test as above, but using a different method
        # Rather than walking the tree from the top, we find the node
        # using xpath to search for matching attributes.
        #
        # Now just to make things confusing, xpath indexes start
        # from 1 not 0 as in Python. Also all xpath results are in a
        # list. Even if it's just a list of one element.
        #
        # Inside () python concatenates strings autmoatically
        # So I have split the xpath query in two strings for
        #  readbility. They make one long string when run.
        #
        # Looking recursively down the tree (//)
        #    find all nodes with any name (*)
        #      that has the class attribute ([@class)
        #      set to legends (="legends"])
        #   return this as a list by surrounding it with ()'s
        #   choose the first element ([1])
        # Looking recursively inside that element (//)
        #    find a node with the id attribute set to
        #        activate-serie-0 (*[@id="activate-serie-0"])
        #        (Because id's ae unique there can be only one.)
        # Then find all direct child nodes (/*).
        # This returns a list and we choose the second child (the first
        #   child is the colored rectangle used to switch graph segments
        #   on and off) index 1 as we are back in python.
        # and grab the text from the node.
        self.assertEqual('unread',
            ret_dom.xpath('(//*[@class="legends"])[1]'
                          '//*[@id="activate-serie-0"]/*')[1].text)

        # This also works since id's are unique
        # Also this id is reserved for the first plotted series (serie-0).
        self.assertEqual('unread',
             ret_dom.xpath('(//*[@id="activate-serie-0"]/*)[2]')[0].text
        )

        # check the title of the chart
        # find the text by using findtext with the namespaced ({...})
        # name of the title node. Not sure why a missing namespace
        # ('.//title') doesn't match.
        self.assertEqual(
            ret_dom.findtext('.//{http://www.w3.org/2000/svg}title'),
            'Tickets grouped by status \n(Roundup issue tracker)'
        )

        # Another way to find it is by namespace css syntax
        # Must use the svg namespace by prepending svg| to the tag.
        # Without the namespace it returns nothing.
        # Look for the title that is the direct child of the svg (root)
        # element.
        from lxml.cssselect import CSSSelector
        css = CSSSelector('svg|svg > svg|title',
                           namespaces={"svg": 'http://www.w3.org/2000/svg'})
        self.assertEqual(
            css(ret_dom)[0].text,
            'Tickets grouped by status \n(Roundup issue tracker)'
        )

         # test the value of serie-0 In formward sort, there are
        # 2 items in status 1
        css = CSSSelector('svg|g.series.serie-0 svg|desc.value',
                           namespaces={"svg": 'http://www.w3.org/2000/svg'})
        self.assertEqual(
            css(ret_dom)[0].text,
            "2"
        )

        '''
        # Remove the surrounding block string marker of three '
        # to uncomment this to dump the xml to a file
        with open('/tmp/f.xml', 'wb') as f:
            f.write(etree.tostring(ret_dom, pretty_print=True))
        '''

    def testPiechartReverseSortStatus(self):
        """Test the piechart action.

           Each test consists of creating the data,
           creating the client
           verifying the returned data.

           In this case, the first element in the legend
           should be the one with the highest count. In case of
           a tie, it should order by the default order of status.

           The status of 1 (unread) has a count of 2, so it
           is first in the legend aka serie-0.
        """
        self.createData()

        cl = self.createClient(
            {'@group': '-status', '@columns': 'id,title'}
            )

        ret = self.instance.cgi_actions['piechart'](cl).handle()

        self.assertTrue(ret.startswith(b'<svg '))
        self.assertTrue(ret.endswith(b'</svg>\n'))
        self.assertIn(b'class="title plot_title"', ret)

        import lxml.etree as etree
        ret_dom = etree.fromstring(ret)
        self.assertEqual('unread',
            ret_dom[5].getchildren()[6].getchildren()[0].getchildren()[1].text)

        # same as above just different method
        self.assertEqual('unread',
            ret_dom.xpath('(//*[@class="legends"])[1]'
                          '//*[@id="activate-serie-0"]/*')[1].text)

        # check the title of the chart
        # find the text by using findtext with the namespaced ({...})
        # name of the title node. Not sure why a missing namespace
        # ('.//title') doesn't match.
        self.assertEqual(
            ret_dom.findtext('.//{http://www.w3.org/2000/svg}title'),
            'Tickets grouped by status \n(Roundup issue tracker)'
        )

You should create methods like: testPiechartPriority, testBarchartForwardSortStatus, testBarchartReverseSortStatus, test BarchartPriority an so forth including multi-group bar charts.

[rouilj]

Varshitha, if you have had no luck getting a clone of the Roundup repo, please attach your test file to this issue.

[rouilj]

Varshitha, where is the file?

[varshi-123]

Hii John , I created a new repository called as roundup.

[rouilj]

Hi Varshitha:

In message @.***>, Varshitha Dinakar writes:

I created a new repository called as roundup.

I only see 1 repository under:

https://github.com/UMB-CS682-Team-02

where did you create the repo? Also did you make it public?

I am not a member of the organization, so I can't see members or private repos.

-- -- rouilj

[varshi-123]

I will add a file here itself and made repo public .

[rouilj]

I can see the repo now, but it doesn't appear as a fork of the main roundup repo.

Compare your roundup repo to the one team 3 created at: https://github.com/UMB-CS-682-Team-03/roundup

Notice how their's shows "Forked from roundup-tracker/roundup".

That's why the email I sent you on this went into the GitHub fork procedure. With a forked repo, you get an easy way of pushing your changes upstream.

If you go to: https://github.com/roundup-tracker/roundup/fork are you able to choose UMB-CS682-Team-02 as the owner?

If so replace the current roundup repo by renaming the current roundup repo to roundup2 and fork the roundup-tracker/roundup repo as c.

Then you can merge your change into the new roundup repo where your work will be identified as a change from the main upstream repo.

If you can't select UMB-CS682-Team-02 as the owner, then just leave it as it is.

In either case, you and the rest of the team will use the UMB-CS682-Team-02 as the owner repo for testing the multibar and other graph types.

[rouilj]

Did you commit your changes to test_cgi.py? I don't se them in the varshitha branch of the repo. The main branch of the repo has commits from you but I don't see a test_cgi.py on the main branch.