Test confreq-rs-file-urls - Reading Systems must prevent access to resources referenced via file URLs

[GJFR]

I'd like to add a test for https://www.w3.org/TR/epub-rs-33/#confreq-rs-file-urls:

Reading Systems MUST prevent access to resources referenced via file URLs

This test will attempt to access resources on the user's local file system. If the reading system allows access, the test fails.

However, there's some complications that have to be addressed, and I'd like some input on their solutions such that they are sure to be in line with the general testing guidelines:

• This test has to take into account OS file system differences (e.g., Windows paths look like file:///C:/... while UNIX paths look like file:///...). In this regard, we could make a subtest for every OS variant:

  • pub-file-urls_macos
  • pub-file-urls_linux
  • pub-file-urls_windows

  Or simply put all in one single test, and differentiate over sections.

• Our 2021 study showed that some reading systems load local file:// resources depending on resource type (e.g., text, image) or file extension (e.g., .log, .txt, .mp3). However, testing for all resource types and file extensions might be a bit overkill and complicated. In my opinion, testing for access to local text and image files would already give a representative result.

• By leveraging pre-existing well-known OS files that exist on virtually any OS instance (independent of user settings), I think we can prevent this test to require the tester to manually create files on their OS. Examples of these pre-existing files are:

  • MacOS
  • Text: file:///var/log/system.log
  • Image: file:///System/Library/CoreServices/DefaultBackground.jpg
  • Linux
  • Text: file:///var/log/system.log
  • Windows
  • Text: file:///C:/Windows/setuplog.txt

Let me know if there's any concern about these suggestions.

[iherman]

I'd like to add a test for https://www.w3.org/TR/epub-rs-33/#confreq-rs-file-urls:

Reading Systems MUST prevent access to resources referenced via file URLs

This test will attempt to access resources on the user's local file system. If the reading system allows access, the test fails.

However, there's some complications that have to be addressed, and I'd like some input on their solutions such that they are sure to be in line with the general testing guidelines:

• This test has to take into account OS file system differences (e.g., Windows paths look like file:///C:/... while UNIX paths look like file:///...). In this regard, we could make a subtest for every OS variant:

  • pub-file-urls_macos
  • pub-file-urls_linux
  • pub-file-urls_windows

  Or simply put all in one single test, and differentiate over sections.

I would definitely like to avoid OS variants. The testing environment (reporting, display, etc) is not prepared for this type of granularity, and it would be quite a lot of work for the purpose of a single test. If this can be done in one single test, it would be better.

• Our 2021 study showed that some reading systems load local file:// resources depending on resource type (e.g., text, image) or file extension (e.g., .log, .txt, .mp3). However, testing for all resource types and file extensions might be a bit overkill and complicated. In my opinion, testing for access to local text and image files would already give a representative result.

I agree this would be an overkill.

• By leveraging pre-existing well-known OS files that exist on virtually any OS instance (independent of user settings), I think we can prevent this test to require the tester to manually create files on their OS. Examples of these pre-existing files are:

  • MacOS

  • Text: file:///var/log/system.log

  • Image: file:///System/Library/CoreServices/DefaultBackground.jpg

  • Linux

  • Text: file:///var/log/system.log

  • Windows

  • Text: file:///C:/Windows/setuplog.txt

That sounds good to me. I would keep to the system logs for each cases, to use a similar resource.

Let me know if there's any concern about these suggestions.

We would have to run tests for some of the reading systems, though, because we are late in the process. I do not think we would run into major problems, though, because I expect web based readers to fail to access these files (ie, pass the tests), ie, colibrio would probably give us a positive result right away. But we have to do that before we go to PR.

cc @wareid

[GJFR]

Alright, thanks for the feedback! I will start on the implementation and already do some preliminary testing.