Suggested implementation for the rego API

[r-scheele]

Rego modifier proposal

---

What problem are you trying to solve?

• Ability to write and modify rego rules from the frontend
• compare new rego rules with the old one and swap
• push the latest file to the github repo

Why should we solve it?

• Anybody can write the rego rules without understanding the underlying language

How do you propose to solve it?

• A central json object that represents the state of the file, and contains all the rules to be written. Any changes to the json can then be reflected in the file -

What are your approaches ?

The less complicated method is build the JSON from front-end, from key-strokes and form input to a particular structure that is understood by the API. The API uses this JSON to write the rules, and any update to the rule on the front-end can be done in the file as well

{
    "rules": [
        [
            {
                "command": "input_prop_equals",
                "properties": ["request_method", "GET"],
            },
            {
                "command": "input_prop_equals",
                "properties": ["request_path", ["v1", "collections", "obs"]],
            },
            {
                "command": "input_prop_equals",
                "properties": ["preferred_username", "dev9ine"],
            },
            {
                "command": "input_prop_in",
                "properties": ["company", "data", "name"],
            },
        ],
        [
            {
                "command": "input_prop_equals",
                "properties": ["request_path", ["v1"]],
            },
            {
                "command": "input_prop_equals",
                "properties": ["groupname", "VIEWER"],
            },
        ],
        [
            {
                "command": "input_prop_equals",
                "properties": ["request_path", ["v1", "collections", ""]],
            },
            {
                "command": "input_prop_equals",
                "properties": ["groupname", "GEOCITY_ADMINS"],
            },
        ],
    ]
}

The JSON can be saved in our database for checks and authorization. if any update is made to our JSON in the database, we can immediately write a new rego file.

The schema for the rule is presented below

class Rule(BaseModel):
    command: str
    properties: Union[
        Tuple[str, Union[str, List[str]]], Tuple[str, Union[str, List[str]], str]
    ]

This approach helps us to decide the key on the input object, as well as the value to be checked against in our rego file from the front-end

say;

{
                "command": "input_prop_equals",
                "properties": ["preferred_username", "dev9ine"],
}

result.rego

input.preferred_username == dev9ine

To make the rules writing easier, i propose that we have a trello card where can all add different rules that comes to our mind -> how the rule check below was proposed in the last meeting.

rule.rego

data.items[i].name == input.preferred_username
data.items[i].groupname == "everyone"

Each rule type has a command attached to it from the front-end, the API uses this command property to decide the kind of function to initiate, in order to safely write the rules in the file.

The command is then mapped to individual functions that writes the rule, while providing necessary parameters to the function. The parameters is present in the properties array

In case of further addition to the rules type, we execute 2 or 3 tasks in our API code

• Write a function to handle that rule
• Add the rule command to the command dictionary
• check if there’s need to adjust the schema of the rule

After successfully writing the rego rules in a file from the rego_builder API, we need a way to ensure the opal server uses the latest rego file.

Linking the rules with opal server

In order to allow opal_server use the new rego file, my proposition is to push the file to the remote immediately it is written locally. That way, our github polling system from the opal_server can still work. and updates can reflect

Managing the Git repository linked to our opal-server

On first initialisation of the app, we need to create a git repository that would help contain our opal policy. with this repository, our file changes can be pushed. from python, several steps are needed to appropriately configure git to work for our use case, which are;

• configure git credentials
• check if there is an existing repo - {parameter - path to the remote repo}, and if exits, we clone it and if not, we intialize a new repository[locally, remote]

Updating the existing rego files

In order to have an easy way of modifying the rego files, few highlights should be noted;

• We want to save the current state of the rego file into the database; If any change is made to the document in our database, the rego file is immediately re-written all over again to reflect that change.
• When the user logs in to the app, we want to have the current state of the rules displayed in the UI. This can be queried from the database, giving the user immediately access towards updating the rules, from button clicks and form inputs.

This approach is better since;

• There's no easy way to identify the individual rules in the rego file, making the modification a hard nut to crack.
• We can easily know the current state of the rules, just by querying the database for it.

[Youngestdev]

Good one! I find the example you give very short. Can you give the JSON that'll give us the Rego file here: https://github.com/geobeyond/fastgeoapi/blob/develop/scripts/iam/policy/auth.rego?

@r-scheele ?

[r-scheele]

{
    "rules": [
        [
            {
                "command": "input_prop_equals",
                "properties": ["request_method", "GET"],
            },
            {
                "command": "input_prop_equals",
                "properties": ["request_path", [""]],
            },
        ],
        [
            {
                "command": "input_prop_equals",
                "properties": ["request_method", "GET"],
            },
            {
                "command": "input_prop_equals",
                "properties": ["request_path", ["static", "img"]],
            },
        ],
        [
            {
                "command": "input_prop_equals",
                "properties": ["request_method", "GET"],
            },
            {
                "command": "input_prop_equals",
                "properties": ["request_path", ["static", "css"]],
            },
        ],
        [
            {
                "command": "input_prop_equals",
                "properties": ["request_method", "GET"],
            },
            {
                "command": "input_prop_equals",
                "properties": ["request_path", ["collections"]],
            },
        ],
        [
            {
                "command": "input_prop_equals",
                "properties": ["request_method", "GET"],
            },
            {
                "command": "input_prop_equals",
                "properties": ["request_path", ["collections", "obs"]],
            },
            {
                "command": "input_prop_equals",
                "properties": ["company", "geobeyond"],
            },
        ],
    ]
}

Gives the output below (copied from the result)

allow {
  input.request_method == "GET"
  input.request_path == [""]
}

allow {
  input.request_method == "GET"
  input.request_path == ["static", "img"]
}

allow {
  input.request_method == "GET"
  input.request_path == ["static", "css"]
}

allow {
  input.request_method == "GET"
  input.request_path == ["collections"]
}

allow {
  input.request_method == "GET"
  input.request_path == ["collections", "obs"]
  input.company == "geobeyond"
}

[Youngestdev]

What about the some collection?

[r-scheele]

What about the some collection?

found out it's not necessary. The variable is important only when we need to loop over a list

[Youngestdev]

Hm. I believe we'll still need to implement that at some point.

Also, are there different commands? If there's only one, perhaps we can remove the need to embed the command in a dictionary and pass only the values.

Also, how can we modify an existing rego file? 👀

[r-scheele]

Hm. I believe we'll still need to implement that at some point.

Also, are there different commands? If there's only one, perhaps we can remove the need to embed the command in a dictionary and pass only the values.

There are multiple commands actually. It’s there to ensure the API is calling the right function. Also i can implement the function to handle the variable.

I haven’t gotten to the part that modifies existing rego file, i'll be updating the issue as appropriate

[Youngestdev]

Got it.

[PascalLike]

Thanks @r-scheele

What problem are you trying to solve? Ability to write and modify rego rules from the frontend compare new rego rules with the old one and swap push the latest file to the github repo

I would skip the "compare new rego rules with the old one and swap" part at this stage. I think that for a first iteration the main objective is to:

• To find a consistent way to represent rego files in JSON
• To create a nice way to convert JSON to rego (and vice versa?)

Also from a GUI iteration point of view I see little use to compare with an existing file, I think it's enough to show the current content of the JSON, let the user modify it, and overwrite the initial one on save. Am I missing something?

The grammar to write rego rules in JSON looks simple in a good way. As long as it covers all the expressions we need, it's ok for me. To better test this, I would suggest to use a rego file that fits better the use cases we discussed during our last meeting.

[r-scheele]

Thanks @r-scheele

What problem are you trying to solve? Ability to write and modify rego rules from the frontend compare new rego rules with the old one and swap push the latest file to the github repo

I would skip the "compare new rego rules with the old one and swap" part at this stage. I think that for a first iteration the main objective is to:

• To find a consistent way to represent rego files in JSON
• To create a nice way to convert JSON to rego (and vice versa?)

Also from a GUI iteration point of view I see little use to compare with an existing file, I think it's enough to show the current content of the JSON, let the user modify it, and overwrite the initial one on save. Am I missing something?

The grammar to write rego rules in JSON looks simple in a good way. As long as it covers all the expressions we need, it's ok for me. To better test this, I would suggest to use a rego file that fits better the use cases we discussed during our last meeting.

Thank you, Antonio. You're right! I'm currently working on a way make the testing easy.

[francbartoli]

@r-scheele @Youngestdev @PascalLike what about using tinydb as a persistence layer rather than a json file for the CRUD?

[r-scheele]

@r-scheele @Youngestdev @PascalLike what about using tinydb as a persistence layer rather than a json file for the CRUD?

I plan to integrate mongodb, since it has flexible schema and it can serve our purpose.

[francbartoli]

Hey @r-scheele, do we really need the scalability offered by mongo? Let's keep the default as simplest as possible

[Youngestdev]

@francbartoli, that's a good one. We'll check out each available option and go with the best.

[Youngestdev]

Regarding the persistence, do check this: #5, @francbartoli and @PascalLike.