Validate fields_description is valid json

[djpnewton]

Add a validator to the Utility form that validates that the fields_description parameter is valid json

Also validate that the json root node is a list and that each object in the list has at least the fields 'label', 'description', 'type', 'target'

[djpnewton]

updated utlity spec: https://github.com/zap-me/zap_bill_payment/blob/master/utility_form_dsl.md

[eoliveros]

created a new script before i start editing one of the script that when ran, will tell me whether the value(statically set on the script) is valid or not.

import json

def validateJSON(jsonData):
    try:
        json.loads(jsonData)
    except ValueError as err:
        return False
    return True

jsonData = """[
  {
    "label": "Invoice Number",
    "description": "The invoice number on your bill statement",
    "type": "number",
    "min": 10000,
    "max": 99999,
    "allow_empty": false,
    "target": "particulars"
  },
  {
    "label": "User Name",
    "description": "The user name assocatiated with your account",
    "type": "string",
    "min_chars": 4,
    "allow_empty": false,
    "target": ["code", "reference"]
  }
]"""

isValid = validateJSON(jsonData)
print("Given JSON is", isValid)

When I ran the above script, it tells me the jsonData is True(valid). To in-validate the result, I remove one of the curly brackets and it becomes False. Need to find a way of adapting the script to the app.

[djpnewton]

take a look at https://flask-admin.readthedocs.io/en/latest/api/mod_model/#flask_admin.model.BaseModelView.form_args

and https://stackoverflow.com/a/23068912/206529

so something like:

def no_root_allowed(form, field):
    if field.data == 'root':
        raise ValidationError('"root" is not allowed')

form_args = dict(
    username=dict(validators=[no_root_allowed])
)

[djpnewton]

updated description to add validation of json data as well

[eoliveros]

take a look at https://flask-admin.readthedocs.io/en/latest/api/mod_model/#flask_admin.model.BaseModelView.form_args

and https://stackoverflow.com/a/23068912/206529

so something like:

def no_root_allowed(form, field):
    if field.data == 'root':
        raise ValidationError('"root" is not allowed')

form_args = dict(
    username=dict(validators=[no_root_allowed])
)

trying to understand the example above, i'm trying to validate value for the a key and not the structure of the json itself?

[eoliveros]

also getting the following when visiitng the site:

not sure why though.

[djpnewton]

take a look at https://flask-admin.readthedocs.io/en/latest/api/mod_model/#flask_admin.model.BaseModelView.form_args and https://stackoverflow.com/a/23068912/206529 so something like:

def no_root_allowed(form, field):
    if field.data == 'root':
        raise ValidationError('"root" is not allowed')

form_args = dict(
    username=dict(validators=[no_root_allowed])
)

trying to understand the example above, i'm trying to validate value for the a key and not the structure of the json itself?

It is just an example of validating a form field

[eoliveros]

    with preprocessed source if appropriate.
    See <http://gcc.gnu.org/bugs.html> for instructions.
    error: command 'gcc' failed with exit status 4
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5jwxju_9/gevent/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5jwxju_9/gevent/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-8qrmt5xs/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7m/gevent Check the logs for full command output.
⠸ Building your source code...
{"code":1,"message":"The command '/bin/sh -c pip3 install -r requirements.txt' returned a non-zero code: 1"}
The command '/bin/sh -c pip3 install -r requirements.txt' returned a non-zero code: 1
Build has failed!
----------------------
Deploy failed!
Error: {"code":1,"message":"The command '/bin/sh -c pip3 install -r requirements.txt' returned a non-zero code: 1"}
The command '/bin/sh -c pip3 install -r requirements.txt' returned a non-zero code: 1

Something bad happened. Cannot deploy test-zap-bill-payment at captain-01.

at the caprover dev server. waiitng a bit and trying again

[eoliveros]

and now getting:

Something bad happened while retrieving test-zap-bill-payment app build logs.
Too many wrong passwords... Wait for 30 seconds and retry.
⠧ Building your source code...

Something bad happened while retrieving test-zap-bill-payment app build logs.
Too many wrong passwords... Wait for 30 seconds and retry.
⠏ Building your source code...

hmmmm... not sure whats happening.

[eoliveros]

finally build it.... placing "root" in the fields description

nothing happened. saved it but thats all... i was hoping an error of some sort.

[eoliveros]

doing a pr to see if you could see what's wrong with what im doing that i cant get it to fail.

[eoliveros]

pulled the changes... deploying and this is the message again:

Something bad happened while retrieving test-zap-bill-payment app build logs.
Error: connect ECONNREFUSED 157.245.188.90:443
⠹ Building your source code...

Something bad happened while retrieving test-zap-bill-payment app build logs.
Error: connect ECONNREFUSED 157.245.188.90:443
⠧ Building your source code...

Something bad happened while retrieving test-zap-bill-payment app build logs.
Error: connect ECONNREFUSED 157.245.188.90:443
⠋ Building your source code...

have to wait a bit and try again.

[eoliveros]

been trying lastnight and today.... and still getting:

Something bad happened while retrieving test-zap-bill-payment app build logs.
Error: connect ECONNREFUSED 157.245.188.90:443
⠦ Building your source code...

Something bad happened while retrieving test-zap-bill-payment app build logs.
Error: connect ECONNREFUSED 157.245.188.90:443
⠹ Building your source code...

Something bad happened while retrieving test-zap-bill-payment app build logs.
Error: connect ECONNREFUSED 157.245.188.90:443
⠼ Building your source code...

may need to update the server?

[eoliveros]

updated the caprover server.

[eoliveros]

deploy the latest code and this time its successful, so i think it just needs to upgrade the server.

[eoliveros]

test the fields_description and when i place "root" it appears to not allow it. Placing something before "root" and/or after, this is allowed.

Now need to know what will be placed in the fields_description? and create validators for it?

[djpnewton]

1. look at the spec that I linked

2. Also validate that the json root node is a list and that each object in the list has at least the fields 'label', 'description', 'type', 'target'

So once you have converted the field to json, check that the root object is a list, check that each of the objects in the list has the required fields

[eoliveros]

getting stuck on the deployment again.

[eoliveros]

  {
    "label": "Invoice Number",
    "description": "The invoice number on your bill statement",
    "type": "number",
    "min": 10000,
    "max": 99999,
    "allow_empty": false,
    "target": "particulars"
  },
  {
    "label": "User Name",
    "description": "The user name assocatiated with your account",
    "type": "text",
    "min_chars": 4,
    "allow_empty": false,
    "target": ["code", "reference"]
  }
]

When I swapped the key and the value im getting an error. putting them back to the original, it seems to be ok.

[eoliveros]

trying to add a validator for json... building is failing on gevent again.... will try again in a bit.

[eoliveros]

hmmm.... removing: "reference" records it as successful. as well as "code"

[
  {
    "label": "Invoice Number",
    "description": "The invoice number on your bill statement",
    "type": "number",
    "min": 10000,
    "max": 99999,
    "allow_empty": false,
    "target": "particulars"
  },
  {
    "label": "User Name",
    "description": "The user name assocatiated with your account",
    "type": "text",
    "min_chars": 4,
    "allow_empty": false,
    "target": ["code"]
  }
]

or

[
  {
    "label": "Invoice Number",
    "description": "The invoice number on your bill statement",
    "types": "number",
    "min": 10000,
    "max": 99999,
    "allow_empty": false,
    "target": "particulars"
  },
  {
    "label": "User Name",
    "description": "The user name assocatiated with your account",
    "type": "text",
    "min_chars": 4,
    "allow_empty": false,
    "target": ["reference"]
  }
]

The word "assocatiated" should be "associated".

My test to see if it has the correct json format seems to be correct. changing the fields label also appears to work.