Question related to Bypass/Global bits in ATS_TRANS_RESP

[zetalog]

Our verification guys noticed a difference between the refence model and the model they developed based on SPEC:

        // only use Untranslated requests to access the implied 4 KiB memory range
        ...
        rsp_msg->trsp.Global = vs_pte.G;
        ...

ATS response uses VS PTE.G which doesn't reflect to the statements of the DTI specification of 5.2.2 DTI_ATS_TRANS_RESP.

Bypass

In the new revision of the DTI specification (edition 3, issue E.b), there is no Global bit defined, but defined Bypass bit:

BYPASS, bit [17]

This field indicates that translation for this StreamID is bypassed. 0 Normal translation 1 Translation bypassed When the value of this field is 1, the VA and the PA of the translation are the same. This bit must be 0 if the value of IA in the translation request is greater than the range shown in the OAS field of the DTI_ATS_CONDIS_ACK message that was received during the connection sequence.

Global

In the old DTI version (edition 3, early release), it forces setting Global to 1 when Bypass is 1 and SSV is 1.

Global, bit[72]

This bit indicates whether this translation appplies to all SubstreamIDs 0 - Non-global. 1 - Global. When the value of the SSV bit in the requesting DTI_ATS_TRANS_REQ message is 0, this bit is reserved, SBZ. When the value of the SSV bit in the requesting DTI_ATS_TRANS_REQ message is 1, and the value of BYPASS is 1, this bit must be 1.

Now we are wiring these bits to 0s both in our design and the verification model which matches the behavior of our PCIe IPs.

Is this a compatibility issue in the reference model?

[ved-rivos]

Mapping to protocols such as DTI should follow the rules provided by those specifications. The IOMMU specification follows the PCIe specification and any such mapping to DTI/LTI or other implementations is implementation defined.

[zetalog]

Sorry for my mistake. I pasted a wrong DTI_ATS_TRANS_RESP figure in the previous comment. Just updated the comment.

[ved-rivos]

The previous picture you posted is for DTI TBU. As noted above, the mapping to implementation specific buses like DTI is implementation specific.

[zetalog]

From PCIe 5.0, there is no Bypass field in CPL, and can see Global bit (Global Mapping) in Table 10-3.

[ved-rivos]

From PCIe 5.0, there is no Bypass field in CPL:

Bypass is a DTI concept and as you noted not a PCIe concept. On other hand Global is a PCIe concept and there is no concept of Bypass in PCIe.

[ved-rivos]

Please check if you see any reference to DTI in PCIe specification.

[ved-rivos]

Such an IP using DTI always receives Global as 0. It is equivalent to such an IP ignoring the Global bit - which is allowed by PCIe - this only results in multiple translations being requested for the same Untranslated Address under different PASIDs - see 10.2.3.8.

[zetalog]

Such an IP using DTI always receives Global as 0. It is equivalent to such an IP ignoring the Global bit - which is allowed by PCIe - this only results in multiple translations being requested for the same Untranslated Address under different PASIDs - see 10.2.3.8.

Does that mean: If an implementation doesn't support ATS.INVAL G field, it can choose to respond G=0 to ATS translation request?

Referring to the following PCIe specification words from 10.2.3.8 (Global Mapping - Global) and 10.5.1.2 (Global Invalidate Supported).

[ved-rivos]

Firstly, I would like to request you to not reproduce information from the PCIe specification that is licensed to you in this GitHub. If you need to refer to a section then providing a section number is sufficient. Would want to avoid any copyright, compliance, antitrust, etc. issues. An implementation that always responds to endpoint with the G bit set to 0 may not run into issues.

[zetalog]

Firstly, I would like to request you to not reproduce information from the PCIe specification that is licensed to you in this GitHub. If you need to refer to a section then providing a section number is sufficient. Would want to avoid any copyright, compliance, antitrust, etc. issues. An implementation that always responds to endpoint with the G bit set to 0 may not run into issues.

Sure, just sorted all of them out. Thanks for the reminds.

The previous picture you posted is for DTI TBU. As noted above, the mapping to implementation specific buses like DTI is implementation specific.

My question is still about the specification/reference model clarification. From the RISCV IOMMU specification, we cannot infer that ATS handlers are allowed to respond G=0. While PCIe spec permits. May we add the implementation dependency of Global bit to the related ATS handling paragraph?

[ved-rivos]

The PCIe specification permits ignoring the response from the IOMMU. The RISC-V IOMMU specification is compliant with the PCIe specification and no further note is required.

[zetalog]

OK, thanks, marking it closed.

[zetalog]

One more thing I'd like to mention is:

For an implementation that is based on DTI-ATS. "Bypass" is a necessary field for ATS response, it combines with "Range" field. Upon receiving such a translation response, the PCIe DTI-ATS bridge which may be resident in PCIe RC is responsible for generating "output address" using "input address", as long as "range", "bypass" fields, with lower bits indicated by "bypass/range" coming from the "input address" in the translation request and higher bits coming from "output address" in the translation response.

As such, "Bypass" is also used in ATS translations. And it likely happens when iosatp.MODE=Bare and iohgatp.MODE=Bare. When bypass=1, output address can be 0, and range indicates the maximum number of supported output address bits.

However, the current reference model seems like a TLP based model, cares less about DTI ATS interfacing, it enforces output address = input address. Normally, a DTI-ATS based PCIe implementation won't send lower bits of input address to IOMMU. It sounds like we may move all of those DTI-ATS bridge side stuffs to implementation specific or test bench specific, and not be a part of IOMMU model to be accommodated with the current IP ecosystem.