drc/testing/README should document rules coverage

[proppy]

Expected Behavior

https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_pr/tree/main/rules/klayout/drc/testing README should describe each of the tests cases and the rules they cover.

Actual Behavior

https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_pr/tree/main/rules/klayout/drc/testing assume developers/contributors are familiar with the naming scheme of the directories and files to be able to identify which rules are covered.

[atorkmabrains]

@proppy The test coverage are listed in the tracking sheet: https://docs.google.com/spreadsheets/d/1xxxg_VzZWJ1NNysSMcOVzyUin7M2dvtb_E7X-5WQKJ0/edit?usp=sharing

All rules unit tests are found in the GDS files in: https://github.com/mabrains/globalfoundries-pdk-libs-gf180mcu_fd_pr/tree/main/rules/klayout/drc/testing/testcases

Please refer to the test patterns documentation in internal google git repos.

[atorkmabrains]

@proppy BTW, we already have this documented in the testing DRC: https://github.com/mabrains/globalfoundries-pdk-libs-gf180mcu_fd_pr/blob/main/rules/klayout/drc/testing/README.md#regression-outputs

cc @mkkassem

[atorkmabrains]

@proppy Please check the updated version of the documentation. I think that should be better.

https://github.com/mabrains/globalfoundries-pdk-libs-gf180mcu_fd_pr/blob/main/rules/klayout/drc/testing/README.md

[proppy]

@atorkmabrains is there a way to get the mapping between the GDS files in testing and the actual rules being covered?

[proppy]

Alternatively is there a way to make the tracking spreadsheet you referenced public and link it from the README?

/cc @QuantamHD and @mithro

[atorkmabrains]

@proppy Yes, please read the section as mentioned above: https://github.com/mabrains/globalfoundries-pdk-libs-gf180mcu_fd_pr/blob/main/rules/klayout/drc/testing/README.md#regression-outputs

[atorkmabrains]

@proppy Namely this file: final_detailed_report.csv

[atorkmabrains]

cc @mkkassem

[proppy]

@atorkmabrains Thanks for the instructions, I was able to verify the coverage of the rules for the Manual_testcases.gds that @QuantamHD was initially asking about.

See the following notebook: https://colab.research.google.com/gist/proppy/23d03ac768d85db7513d3ec033fc9af5/gf180mcu-klayout-playground.ipynb#scrollTo=vT0g7eE9ISdG

A few following questions:

• it seems that some tests create a lyrdb file and some don't (ex: Manual_testcases), is that intentional?
• is there a way to check which GDS testcase act as a "violator cell" for a given rule (i.e: trigger an excepted failure)

[atorkmabrains]

@proppy Yes, all unit tests for each rule has a bunch of pass test cases, where the DRC for that rule shouldn't highlight and a bunch of fail test cases, where the DRC should highlight and we make sure that locations where it supposed to highlight it highlights and where it supposed not to highlight it doesn't highlight. If the DRC highlight in pass test case, that is false positive test case, and if it highlights in a fail test cases that's a false negative test case. And we investigated all those false positive and false negative test cases before. Please check our original summary of the analysis here:

https://docs.google.com/spreadsheets/d/117Djoc7bjs09jyru4CYNuxXDeraHz8717UGPKOaBA14/edit#gid=0

Also, Please check our slides at: https://docs.google.com/presentation/d/1jMYQMGoihTglgvgFRTApGS_z6e_hdlRhD-zFyHu-65c/edit#slide=id.p https://docs.google.com/presentation/d/1dmM19jXz1AyYz_-JTpomD9hVx87cd8V5PjW5EYrseQk/edit?usp=sharing https://docs.google.com/presentation/d/15kvnHgidU8-gYRN1dv2YV_t_m3IRd5dm6YcchAgA3YA/edit?usp=sharing

Please let me know if you have further questions.

[atorkmabrains]

cc @mkkassem

[proppy]

all unit tests for each rule has a bunch of pass test cases [...] bunch of fail test cases

Is Manual_testcases.gds the only GDS "unit tests" testcase that fit with this definition? If yes, it seems that it only cover a portion of the rules (28) per https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_pr/issues/65#issuecomment-1308668462.

[atorkmabrains]

@proppy check this: rules/klayout/drc/testing/testcases/README.md

Unit tests are covered by all files in the testcases folder. Manual testcases file is just a portion of the tests.

[atorkmabrains]

@proppy I have discussed that with @mithro before. That can't be done. Primitive renaming is a lateral change that affects everything and if you take one part and leave the other parts of the PDK unchanged, it will break the PDK. And there is another issue, doing PR will not work as well as the amount of changes are massive. And github doesn't allow that. I have shown this to @mithro and @mkkassem. That's why I was asking you guys to get the primitives renaming PRs in first before we change anything.

[proppy]

@proppy check this: rules/klayout/drc/testing/testcases/README.md Unit tests are covered by all files in the testcases folder. Manual testcases file is just a portion of the tests.

Yes, but other tests listed in the updated README are labelled as "foundry tests", not "unit tests".

Do they also follow the rules that you mentioned earlier?

all unit tests for each rule has a bunch of pass test cases [...] bunch of fail test cases

If either case, can we make this explicit in the README?

[atorkmabrains]

Yes, Foundry tests are unit tests per rule

[atorkmabrains]

@proppy @mkkassem DRC/LVS are no longer part of this repo.