Vector ACT

[jjscheel]

Technical Group

Vector TG

ratification-pkg

Vector

Technical Liaison

Krste

Task Category

Arch Tests

Task Sub Category

• [ ] gcc
• [ ] binutils
• [ ] gdb
• [ ] intrinsics
• [ ] Java
• [ ] KVM
• [ ] ld
• [ ] llvm
• [ ] Linux kernel
• [ ] QEMU
• [ ] Spike

Ratification Target

4Q2021

Statement of Work (SOW)

SOW: link

SOW Signoffs:

• Task group liaison sign-off date:
• Development partner sign-off date:
• ACT SIG sign-off date (if ACT work):

Waiver

• [X] Freeze
• [X] Ratification

Pull Request Details

No response

[jjscheel]

Update from Xi:

We were working on the exception tests for RVV. We also refined Mask instruction to address the case of non-deterministic mask bits.

We plan to submit the PR somewhere next week and kick off the review process.

[jjscheel]

@allenjbaum and Xi need to work to find a time where the RIOS representative can meeting with key people from ACT group and begin to engage.

This is the key next step, along with submitting PR for work.

[allenjbaum]

We really need to get someone from the Vector TG to be involved with this; I don't know it well enough to understand all but the most basic corner cases that need to be tested.

On Mon, May 1, 2023 at 5:24 PM Jeff Scheel @.***> wrote:

@allenjbaum https://github.com/allenjbaum and Xi need to work to find a time where the RIOS representative can meeting with key people from ACT group and begin to engage.

This is the key next step, along with submitting PR for work.

— Reply to this email directly, view it on GitHub https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-1530665580, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJWBCD2POSYDHE4WAG3XEBH2HANCNFSM6AAAAAAV6UE5A4 . You are receiving this because you were mentioned.Message ID: @.***>

[jjscheel]

I've just added @hushenwei2000 to this topic. Welcome Shenwei!

@allenjbaum, I believe that Xi was working with Krste directly. Your question is a great one and we should see if Krste ever delegated. If not, I'd be inclined to ask the person helping @billmcspadden-riscv with the SAIL work to see if they'd help with ACT too. If not, we can ask Krste and Earl. Sound ok?

[allenjbaum]

I don't think the someone from the Sail team, or helping the Sail team, is what we want. We really want someone who understands the vector spec and its corner cases., That could be someone working on the Sail code if their primary experience is the vector spec, but being conversant with all the intricacies of the vector spec is what we are looking for, and I expect to find that expertise in the vector TG

On Tue, May 2, 2023 at 8:56 AM Jeff Scheel @.***> wrote:

I've just added @hushenwei2000 https://github.com/hushenwei2000 to this topic. Welcome Shenwei!

@allenjbaum https://github.com/allenjbaum, I believe that Xi was working with Krste directly. Your question is a great one and we should see if Krste ever delegated. If not, I'd be inclined to ask the person helping @billmcspadden-riscv https://github.com/billmcspadden-riscv with the SAIL work to see if they'd help with ACT too. If not, we can ask Krste and Earl. Sound ok?

— Reply to this email directly, view it on GitHub https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-1531721490, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJUUMQ2764PL6ROKAY3XEEVEXANCNFSM6AAAAAAV6UE5A4 . You are receiving this because you were mentioned.Message ID: @.***>

[jjscheel]

Yes, @allenjbaum, the SAIL community asked a member of the Vector TG to attend their meetings and help review the SAIL code. Right, @billmcspadden-riscv?

Perhaps this same person would do the same for ACT.

[billmcspadden-riscv]

Victor Moya @.***) was our SME for the review of the Sail vector implementation. Should he be added to this thread?

Bill Mc.

On Thu, May 4, 2023 at 9:57 AM Jeff Scheel @.***> wrote:

Yes, @allenjbaum https://github.com/allenjbaum, the SAIL community asked a member of the Vector TG to attend their meetings and help review the SAIL code. Right, @billmcspadden-riscv https://github.com/billmcspadden-riscv?

Perhaps this same person would do the same for ACT.

— Reply to this email directly, view it on GitHub https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-1534931768, or unsubscribe https://github.com/notifications/unsubscribe-auth/AXROLOBJET6P4K3RVIKXBQDXEO7VDANCNFSM6AAAAAAV6UE5A4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Bill McSpadden Formal Verification Engineer RISC-V International mobile: 503-807-9309

[jjscheel]

Thanks, @billmcspadden-riscv. I'll reach out directly before adding him.

[jjscheel]

@hushenwei2000, please work with @allenjbaum and the Arch Test SIG to begin integrating the Vector ACT. Let me know when you submit your first PR.

[jjscheel]

@hushenwei2000, thanks for attending today's meeting. I'm happy to hear that you and allen are beginning to work together. Please let me know when that first PR is submitted.

As discussed, there likely will be code re-work to split out common testing framework for all vector items. This will be needed by function currently being discussed in the Arch Test SIG such as Vector Crypto and other work not yet discussed, Zvfh/Zvfhmin. So, the key idea is to do integrate the new testing in such a way as to allow for commonality in future vector instructions.

[hushenwei2000]

After the last meeting with test SIG, I learned that my previous tests had a self check and did not comply with the Test Format.

Now, I am changing my test structure to allow the tests to run and compare signatures on RISCOF. I have completed the ACT with vsew=32, and other configurations will be completed as soon as possible.

You can try in my repo: https://github.com/hushenwei2000/rvv-atg/tree/new_test_format

[jjscheel]

Thanks, @hushenwei2000! It sounds like very positive work is underway. This is VERY good!!!

[allenjbaum]

I completely agree, naturally.

On Wed, Jul 19, 2023 at 5:47 AM Jeff Scheel @.***> wrote:

Thanks, @hushenwei2000 https://github.com/hushenwei2000! It sounds like very positive work is underway. This is VERY good!!!

— Reply to this email directly, view it on GitHub https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-1642021051, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJTNAIY5PRLFTBCDIRDXQ7JOHANCNFSM6AAAAAAV6UE5A4 . You are receiving this because you were mentioned.Message ID: @.***>

[hushenwei2000]

I guess I have completed the code modifications to support all configurations. The tests I generated can now generate signature files, and I have tested them in the RISCOF framework.

However, there are sometimes inconsistencies in the signatures between Sail and Spike, which may be a bug in Sail.

You can try it according to the README in https://github.com/hushenwei2000/rvv-atg/tree/new_test_format

I would like to know what I should do next?

[allenjbaum]

Could you give more details about the inconsistencies in the signatures that you found? Can you isolate them to specific instructions?

On Monday, August 14, 2023, HU Shenwei @.***> wrote:

I guess I have completed the code modifications to support all configurations. The tests I generated can now generate signature files, and I have tested them in the RISCOF framework.

However, there are sometimes inconsistencies in the signatures between Sail and Spike, which may be a bug in Sail.

You can try it according to the README in https://github.com/ hushenwei2000/rvv-atg/tree/new_test_format

I would like to know what I should do next?

— Reply to this email directly, view it on GitHub https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-1677139786, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJQNWA66HVUXRMRFXP3XVIC2ZANCNFSM6AAAAAAV6UE5A4 . You are receiving this because you were mentioned.Message ID: @.***>

[hushenwei2000]

Unfortunately, the Sail and Spike signatures of all instructions are inconsistent.

But in my previous test with expected answers, Sail passed all the tests.

Therefore, I suspect that there was an error in Sail about the load/store instruction used for reading/writing signatures.

[allenjbaum]

I was looking for more detail, such as what precisely the mismatch was. Were the signature values offset (so elements were store at signature word X+1 instead of X?) or the signature values were only partially stored or not stored at all (so the background canary wasn't change). Could you cut&paste a bit of the test report so we can see where the mismatches were?

On Tue, Aug 15, 2023 at 6:58 AM HU Shenwei @.***> wrote:

Unfortunately, the Sail and Spike signatures of all instructions are inconsistent.

But in my previous test with expected answers, Sail passed all the tests.

Therefore, I suspect that there was an error in Sail about the load/store instruction used for reading/writing signatures.

— Reply to this email directly, view it on GitHub https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-1678978348, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJQFG2VZ3U7WMSGAVNLXVN563ANCNFSM6AAAAAAV6UE5A4 . You are receiving this because you were mentioned.Message ID: @.***>

[jjscheel]

@hushenwei2000, would you kindly provide a written update here on your progress? Thanks!

[hushenwei2000]

The issue of inconsistent signatures has been resolved by adding initial VLEN configuration in Sail model. And that means the functionality of the test generator is okay.

We will continue to discuss the issue of coverage with Allen.

[jjscheel]

@hushenwei2000, any updates?

[hushenwei2000]

We are working on the coverage things, but they are quite complicated. For example, we need to decide on the numbers and configurations, which may take some time. Allen, James and I are collaborating on that.

[hushenwei2000]

We are working on a formal coverage definition for all instructions. And deal with non-deterministic cases.

[hushenwei2000]

I am kinda busy this month. Quswar Abid from 10xEngineers recently contacted me to help me complete the Vector ACT, much thanks! We are working together to ensure the entire framework correct, especially the signature part.

[hushenwei2000]

I apologize for the inconvenience. Over the past few months, I have been dedicated to completing my master's research project. However, I intend to shift my focus back to ACT in December.

While collaborating with Quswar, I identified some bugs in my signature. Therefore, the immediate priority is to address these issues to ensure the accuracy of the ACT. Subsequently, we will also need to address additional coverage points.

[jjscheel]

Thanks, @hushenwei2000!

[hushenwei2000]

I am working on fixing bugs in the signature, and although I have tried my best to extract common functions, I still need to check each instruction and configuration. I hope to complete the signature part before the next meeting, which will ensure that the ACT code usability.

[hushenwei2000]

I've been continuing to work on fixing bugs and have adapted the signature to all configurations. Next step I will focus on coverage points maybe. Vector ACT Coverage Points(Draft)

[hushenwei2000]

The code has been completed; I think. However, we still need to have a detailed discussion regarding the definition of coverage and make modifications to the source code.

@jjscheel Jeff, as I am about to commence work on my graduation thesis, the ownership of Vector ACT will now be transferred to my colleague, Xieyuan Wu @Oxyw. I will continue to provide assistance as needed. She will be responsible for further development and will be in touch with you regarding any matters related to PR.

[jjscheel]

Welcome @Oxyw! Please send an email to help@riscv.org from your RIOS email asking for a Portal id. We will get you setup with all access you need.

[jjscheel]

@XinlaiWan, @billmcspadden-riscv have we resolved the gating bug for this? It also appears this SAIL PR now has merge conflicts which must be resolved. What is our outlook for completing this?

[Oxyw]

Welcome @Oxyw! Please send an email to help@riscv.org from your RIOS email asking for a Portal id. We will get you setup with all access you need.

Thanks! I've sent the email. btw, my RIOS email is xieyuan.wu@rioslab.org.

[XinlaiWan]

@XinlaiWan, @billmcspadden-riscv have we resolved the gating bug for this? It also appears this SAIL PR now has merge conflicts which must be resolved. What is our outlook for completing this?

Hi Jeff! The bug has been fixed in PR 359. I think it's ready to be merged. And the conflicts are because of the recent update on the main branch about the Sail bitfield syntax. I think it's better to merge it soon, or there will be more conflicts in the future if more updates appear in the main branch.

[jjscheel]

I see that Xinlai. Thanks for the work here.

@billmcspadden-riscv, can you put SAIL PR #359 on your list of issues to try and resolve. I'm holding this open until that gets merged.

[allenjbaum]

Adding more comments about needing a subject matter expert to help define coverpoints for RVV. I was suggesting Krste as the subject matter expert, but Jeff suggested asking Roger Espasa. I was hoping that we could get coverpoints donated from some member company who is implementing RVV (obviously SiFive and Semidynamics is 2 of them) - or even more than one member company, and see if we can get them adapted to the ACT format.

[jjscheel]

Reached out to Roger via email and cc'd Allen.

[Oxyw]

Oops! I realized I mistakenly altered the state button. It's been reverted now. :) Updates: added tests for previously missing instructions (e.g. vmerge, vmv...) and fix some known bugs (value coverage & riscof run code).

[jjscheel]

@vmoya-smd, can you help with coverage on these tests? @allenjbaum, @Oxyw, would you kindly point Victor at the specific files which need review/comments?

[Oxyw]

Great! Here is the link to the draft: RVV Test Coverage Points

[Oxyw]

Updates: cover different mask value (walking_zeros, walking_ones, checkerboard pattern)

[allenjbaum]

It sounds like you're not putting your coverage into the .cgf format , so we can't track it with ISAC spec, which is a bit of a problem we should talk about in the arch-test meeting. This would be the first coverage test that doesn't use ISAC to measure coverage. We allow test generation to to CTG or any other test generator, and even manual tests, but this would be the first time that a different coverage tool is used.

On Tue, Mar 19, 2024 at 2:55 AM WU Xieyuan @.***> wrote:

Updates: cover different mask value (walking_zeros, walking_ones, checkerboard pattern)

— Reply to this email directly, view it on GitHub https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-2006587136, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJQSSBLGCRAEMLWMTDDYZADPLAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBWGU4DOMJTGY . You are receiving this because you were mentioned.Message ID: @.***>

[Oxyw]

Yes, and I think it‘s also hard for covering different vstart to be put into cgf format.

It sounds like you're not putting your coverage into the .cgf format , so we can't track it with ISAC spec, which is a bit of a problem we should talk about in the arch-test meeting. This would be the first coverage test that doesn't use ISAC to measure coverage. We allow test generation to to CTG or any other test generator, and even manual tests, but this would be the first time that a different coverage tool is used. … On Tue, Mar 19, 2024 at 2:55 AM WU Xieyuan @.> wrote: Updates: cover different mask value (walking_zeros, walking_ones, checkerboard pattern) — Reply to this email directly, view it on GitHub <#23 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJQSSBLGCRAEMLWMTDDYZADPLAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBWGU4DOMJTGY . You are receiving this because you were mentioned.Message ID: @.>

[allenjbaum]

I think that problem is getting fixed by adding all?some? CSR updates to the log files,, both explicit and implicit.

On Wed, Mar 20, 2024 at 12:38 AM WU Xieyuan @.***> wrote:

Yes, and I think it‘s also hard for covering different vstart to be put into cgf format.

It sounds like you're not putting your coverage into the .cgf format , so we can't track it with ISAC spec, which is a bit of a problem we should talk about in the arch-test meeting. This would be the first coverage test that doesn't use ISAC to measure coverage. We allow test generation to to CTG or any other test generator, and even manual tests, but this would be the first time that a different coverage tool is used. … <#m6940674792954379929> On Tue, Mar 19, 2024 at 2:55 AM WU Xieyuan @.> wrote: Updates: cover different mask value (walking_zeros, walking_ones, checkerboard pattern) — Reply to this email directly, view it on GitHub <#23 (comment) https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-2006587136>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJQSSBLGCRAEMLWMTDDYZADPLAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBWGU4DOMJTGY https://github.com/notifications/unsubscribe-auth/AHPXVJQSSBLGCRAEMLWMTDDYZADPLAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBWGU4DOMJTGY . You are receiving this because you were mentioned.Message ID: @.>

— Reply to this email directly, view it on GitHub https://github.com/riscv-admin/dev-partners/issues/23#issuecomment-2008956263, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPXVJQ4ZWRTDRCHQT4YHI3YZE4FZAVCNFSM6AAAAAAV6UE5A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBYHE2TMMRWGM . You are receiving this because you were mentioned.Message ID: @.***>

[Oxyw]

updates: 1. compact some macro generator codes; 2. change all mask related tests from vlex to vlm for v0; 3. add floating point dataset

[Oxyw]

updates: 1. extract 12 ibm fp datasets from rpt instead of hand-written in test generator; 2. modify tests for vwred

[jjscheel]

Sounds like good progress. What work remains to have accepted PRs?

[Oxyw]

In addition to the remaining tasks outlined in the coverage point draft, significant code refactoring is also needed. Currently, making minor changes requires modifications across a large portion of the code. The refactoring effort is substantial, and completing it may take some time if I'm working on it alone.

[Oxyw]

Update: save vcsr in signature along with vd, fcsr and xstatus CSRs

[allenjbaum]

Saving xcsr requires eiable.ther running in M/S modes, or transitioning to them prior to saving. Or not running in Umode, which might also be accept

[Oxyw]

updates: fixes (vfwred; some macros) & code refactoring

[jjscheel]

Thanks, @Oxyw. Do you have a date when you think you'll have a first PR for your work?