Size and permissions returned in the Completion of PCIe ATS translation request

[just-for-fun-too]

Hi

The specification in Section 2.1.3 says:

"If the EN_ATS bit is 1 and the T2GPA bit is set to 1 the IOMMU performs the two-stage address translation to determine the permissions and the size of the translation to be provided in the completion of a PCIe ATS Translation Request from the device."

I just want to confirm that the permissions and the (page) size returned in the response correspond to the second-stage leaf entry and not the first-stage leaf entry. We are providing, in the completion of the PCIe ATS Translation request, the GPA from the leaf entry of the first-stage translation tables, but the permissions and size from the second-stage (against which we checked the request permissions because of doing two-stage translation). Is this correct?

If it is correct, then when we do a translation later on, from GPA to SPA, we will be comparing the permissions that came with the request and were from the second stage translation against the actual second stage permissions at that time. What is the benefit or reason to do this?

Thanks

[ved-rivos]

When T2GPA is on, the IOMMU does the complete address translation and permission checking. However, the response to the address translation request provides the GPA to the device and not the SPA. When the device performs a memory access using the GPA, it is then translated to an SPA and the permissions are verified again as part of that process.

What is the benefit or reason to do this?

Please see the discussion in Section 2.1.3.

[just-for-fun-too]

Hi Thanks. I had read that section. I also read a number of other questions and responses logged here in the past relating to T2GPA, and as such, that part of the issue is pretty clear now.

However, my primary question remains unanswered. The question is, what are the permissions and (page) size that get returned with the GPA in the completion message of the address translation request? Is it from the first-stage or the second-stage when T2GPA is on.

[ved-rivos]

The question is, what are the permissions and (page) size that get returned with the GPA in the completion message of the address translation request? Is it from the first-stage or the second-stage when T2GPA is on.

The permissions and page sizes returned are always the final permission and page sizes i.e. the most restrictive of either stages. The determination of permission to provide does not depend on T2GPA configuration.

[just-for-fun-too]

Okay. Let me restate the question.

As the specification says ...If the EN_ATS bit is 1 and the T2GPA bit is set to 1 the IOMMU performs the two-stage address translation to determine the permissions and the size of the translation. So we: 1- Perform two stage translation. 2- We check the requested permissions against the first-stage translation and second-stage translation. 3- We return the first-stage address (GPA) 4- The question is: Which permissions and (page) size do we return?

You answered above that the "permissions and page sizes are always the final permissions and page sizes" which to me implies in the scenario listed above its from the second-stage translation because that is the final stage. But then you also said that it is "the most restrictive of either stages" which implies it can be from either of the two stages, which ever is most restrictive.

Sorry but this is what I am confused about.

[ved-rivos]

When two-stage address translation is performed the permission and page sizes output by the process are always the most restrictive permission and page size. For instance if the page is writable in VS-stage but not writable in G-stage or vice versa then the accumulated permission is not-writable. If the page size in VS-stage is 2 MiB and in G-stage is 4 KiB or vice versa then page size is 4 KiB and so on. So while the GPA is returned to the device the permission and page size returned are the output of the two stage address translation.

The question is: Which permissions and (page) size do we return?

We return the permissions and page size that is the result of the two-stage address translation. It is the most restrictive of the permissions provided by either stage and is the smallest page size encoded in either stage.

[just-for-fun-too]

Thank you very much for the examples! This answers my questions very clearly.

BTW I did check the RISC-V Instruction Set Manual Volume II: Privileged Architecture also just in case I missed this. But even there I haven't been able to find this concept of restrictive permissions and the smallest page size explicitly called out anywhere. Neither did I find it anywhere in the IOMMU specification.