[Discussion] adjusting the pre-generated json to contain all possible fields

[jknndy]

As title describes the generate json functionality could be adjusted to automatically generate all the possible outputs which would require the developer to remove the unused fields in the json.

a possible improvement, as described by @strangetom in #1124 , is..

An option would be to catch the exception raised if a scraper doesn't implement one of the json keys and output a message saying that the entry should be removed from the testjson if the scraper doesn't implement that function.

for key in OPTIONAL_TESTS:
    with self.subTest(key):
        scraper_func = getattr(actual, key)
        if key in expect.keys():
            try:
                self.assertEqual(
                    expect[key],
                    scraper_func(),
                    msg=f"The actual value for .{key}() did not match the expected value.",
                )
            except Exception as e:
                print(f"If the scraper does not implement .{key}(), then remove the entry from the .testjson file.")
                raise e

Restarting this conversation here so the reordering PR can be merged.