Improve the test coverage of the _link_allowed function

[lovisastrange]

Keep a record (copy) of your coverage before you start working on new tests. Furthermore, make sure you add the new tests on a different branch than the one you used for coverage instrumentation.

Having identified “weak spots” in coverage, try to improve coverage with additional test cases.

1. Identify the requirements that are tested or untested by the given test suite.

2. Document the requirements (as comments), and use them later as assertions.

3. Create new test cases as needed to improve branch coverage in the given functions. Can you call the function directly? Can you expand on existing tests Do you have to add additional interfaces to the system (as public methods) to make it possible to set up test data structures?

4. If you have 100 % branch coverage, you can choose other functions or think about path coverage. You may cover all branches in a function, but what does this mean for the combination of branches? Consider the existing tests by hand and check how they cover the branches (in which combinations).

[lovisastrange]

Coverage before new tests for both _link_allowed function and methods in link class:

• Not fully covered, some branches never reached

• 

[lovisastrange]

Requirements identified:

For _link_allowed, the requirements that are found are

• Req 1: Link must be a valid url (must start with http or similar)
• Req 2-3: Link must respectively must not contain the allowed respectively not allowed characters
• Req 4-5: The domain is checked to be in the allowed/not allowed domains if any are given
• Req 6: Checks if url has extensions, only allow if extensions are allowed
• Req 7: Check if the text contains any restriced characters

For the link class, we instead have in the init and eq functions that

• Req 1: Url must be a string
• Req 2: The object we compare a link to must be a link
• Req 3::When comparing two links, they are the same if all attributes are the same.

[lovisastrange]

The untested requirements is Req 1 for _link_allowed and 1 and 2 for the link class

[lovisastrange]

The test on _link_allowed test that the extracted links is an empty list when given a non-valid link. The test test_not_valid_link is created in the file scrapy/tests/test_linkextractors.py. We give the link test.test, and check that

 self.assertEqual(lx.extract_links(response),[])

The coverage after adding this test is

[lovisastrange]

Now, we look at the test added for the link class.

The tests were added in scrapy/tests/test_link.py and cover one of the un-tested cases each. In the first one, test_if_string, we check that the function raises a TypeError when an integer is given instead of a string.

Another test, test_equals_not_link checks that we get a NotImplementedError if we try to compare a link object with a non-link object, more specifically an integer.

The coverage after adding the test is