Document the restrictions

[vanillajonathan]

I read the documentation and vaguely get the idea that RestrictedPython is a restricted subset of Python, but the documentation does not explain what is restricted and why.

The documentation should list what is restricted and give a reason as for why that is restricted, as well as list what is not restricted.

Reading the source code gives some hints:

• Matrix multiplication (@) is currently not allowed.
• nonlocal is not allowed.
• await/async is not allowed.
• There is a range function with a limit of 1000.
• list("string") is not allowed. Why?
• tuple("string") is not allowed. Why?

What about...

• infinite loops such as while True?
• recursion?
• nested for loops?
• "foo".zfill(99999999999999999)
• "foo".ljust(99999999999999999, ".")
• 1024**1024**1024
• String repeat "foo" * 1024**1024

[d-maurer]

Jonathan wrote at 2024-1-19 08:58 -0800:

I read the documentation and vaguely get the idea that RestrictedPython is a restricted subset of Python, but the documentation does not explain what is restricted and why.

RestrictedPython is essentially a code transformer checking (and potentially rejecting) syntactic features and converting some syntactic features (such at attribute and item management) into function calls which can be implemented by an application specific policy package to enforce the applications security policy.

RestrictedPython by default rejects syntactic features unless someone has analysed the feature and provided good reasons

1. that it is important and 2. that the risks are not high. For many features, nobody has yet done this preliminary work (e.g. "async/await"). Maybe, you would like to step in?

Apparently, you do not like some predefined definitions (e.g. list, tuple, range, ...). Should you have an application for which those are inadequate, it is very easy to provide your own, better adapted definitions.

For other things, e.g. preventing excessive exponentiation, you would need to extend the code transformation, in order to transform the respective elementary operations into function calls which can be implemented/controlled by the application's policy package.

[vanillajonathan]

Yes, I understand that RestrictedPython uses the ast module to parse the code into an abstract syntax tree and that it uses the ast.NodeTransformer.

I never said that I did not like the list, tuple or range definitions. I pointed out that the documentation does not mention those, and that I had to read the source code to find those, and that I not know why those are not allowed.

I would like the documentation to include a list of features that are not allowed and an explanation why, and a list of features which are restricted and how they are restricted and why.

Furthermore I would like the documentation to include information about what is not restricted that I should be aware of.

[d-maurer]

Jonathan wrote at 2024-1-19 12:54 -0800:

... I would like the documentation to include a list of features that are not allowed and an explanation why, and a list of features which are restricted and how they are restricted and why.

You looked at the code and found out some things you think should get documented. What about signing a contributor agreement and add those things to the documentation?

Furthermore I would like the documentation to include information about what is not restricted that I should be aware of.

Expression evaluation (apart from attribute and item access) is mostly not restricted. Your contributions to the documentation will be welcome.

Providing your own implementation for attribute access, you can control method lookup. In this way, you can e.g. ensure that str.zfill returns a callable imposing a limit on its paramwter or disallow zfill completely.

[vanillajonathan]

You looked at the code and found out some things you think should get documented.

Yes, a few while briefly looking through some of the code, but I wouldn't always know the rationale behind the decisions, even if I perhaps could guess.

What about signing a contributor agreement and add those things to the documentation?

I tend to shy away from contributing to projects where signing a CLA is required. I haven't even used RestrictedPython, just looked at the documentation to try to figure out what it restricts and what it doesn't.

[d-maurer]

Jonathan wrote at 2024-1-19 15:21 -0800:

You looked at the code and found out some things you think should get documented.

Yes, a few while briefly looking through some of the code, but I wouldn't always know the rationale behind the decisions, even if I perhaps could guess.

You can assume mostly that limitations have been introduced based on experience: someone hit a case where a missing limit has posed problems and he introduced the limit.

Example: the limited range function. git annotate tells us that it was introduced by commit ac2fc0f6 and a comment that is has been contributed by Martijn Pieters.

I assume that the actual limits have been chosen "ad hoc". In my view, this is okay. If an application detects that such a limit is inadequate (either too low or too high), it can easily put in its own implementation.

What about signing a contributor agreement and add those things to the documentation?

I tend to shy away from contributing to projects where signing a CLA is required.

Like me.

Unfortunately, this likely means that the documentation will not get improved.

I haven't even used RestrictedPython, just looked at the documentation to try to figure out what it restricts and what it doesn't.

There are several kinds of restrictions:

• syntactic restrictions (implemented by the code transformer)
• conversion of attribute and item management into function calls
• print related conversions
• some predefinitions (safe_builtins, safe_globals) which might be used by an application (if applicable)

The last type of restrictions can easily be changed by an application by providing its own definitions.

The former ones can be changed, too, but it requires changes in the code transformer which is quite hard.