search

minvws / nl-kat-coordination

Repo nl-kat-coordination for minvws
European Union Public License 1.2
123 stars 55 forks source link

Error when manually adding a OOIs #2901

Closed stephanie0x00 closed 4 months ago

stephanie0x00 commented 4 months ago

Describe the bug Various OOIs cannot be added manually as they trigger errors. Screenshots and stacktrace outputs are shown below as well. The following OOIs were identified to be broken:

- Question object a TypeError is returned. 

Exception Value:    

issubclass() arg 1 must be a class

Exception Location:     /app/rocky/tools/forms/ooi_form.py, line 73, in generate_form_fields

- AutonomousSystem number 

Exception Value:    

'OOIForm' has no field named 'name'.

Exception Location:     /usr/local/lib/python3.11/site-packages/django/forms/forms.py, line 400, in add_error

- Config - 'OOIForm' has no field named 'config'.

Exception Value:    

'OOIForm' has no field named 'config'.

Exception Location:     /usr/local/lib/python3.11/site-packages/django/forms/forms.py, line 400, in add_error

'OOIForm' has no field named 'meta_data'.

Exception Location: /usr/local/lib/python3.11/site-packages/django/forms/forms.py, line 400, in add_error


**To Reproduce**
Steps to reproduce the behavior:
1. Go to Object.
2. Add object of type 'Question'
3. See error

**Expected behavior**
The chosen object can be added. 

**Screenshots**

![image](https://github.com/minvws/nl-kat-coordination/assets/9821756/564ff2e6-ec38-462f-a088-a3a1502b6538)

**OpenKAT version**
commit 3588c77f085b5643823477b1636ed33a37d92302 (HEAD -> main, origin/main, origin/HEAD)

**Stacktrace**

Environment:

Request Method: GET Request URL: http://127.0.0.1:8000/en/aa/objects/add/Question/

Django Version: 4.2.11 Python Version: 3.11.9 Installed Applications: ['whitenoise.runserver_nostatic', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.forms', 'django_otp', 'django_otp.plugins.otp_static', 'django_otp.plugins.otp_totp', 'two_factor', 'account', 'tools', 'fmea', 'crisis_room', 'onboarding', 'katalogus', 'django_password_validators', 'django_password_validators.password_history', 'rest_framework', 'tagulous', 'compressor', 'reports', 'knox', 'csp'] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django_otp.middleware.OTPMiddleware', 'rocky.middleware.auth_required.AuthRequiredMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'rocky.middleware.onboarding.OnboardingMiddleware', 'csp.middleware.CSPMiddleware']

Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view return self.dispatch(request, *args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch return handler(request, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/views/generic/edit.py", line 144, in get return self.render_to_response(self.get_context_data()) ^^^^^^^^^^^^^^^^^^^^^^^ File "/app/rocky/rocky/views/ooi_add.py", line 69, in get_context_data context = super().get_context_data(kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/rocky/account/mixins.py", line 103, in get_context_data context = super().get_context_data(kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/views/generic/edit.py", line 74, in get_context_data kwargs["form"] = self.get_form() ^^^^^^^^^^^^^^^ File "/app/rocky/rocky/views/ooi_view.py", line 147, in get_form form = super().get_form(form_class) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/views/generic/edit.py", line 39, in get_form return form_class(**self.get_form_kwargs()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/rocky/tools/forms/ooi_form.py", line 28, in init fields = self.get_fields() ^^^^^^^^^^^^^^^^^ File "/app/rocky/tools/forms/ooi_form.py", line 36, in get_fields return self.generate_form_fields() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/rocky/tools/forms/ooi_form.py", line 73, in generate_form_fields elif self.ooi_class == Question and issubclass(annotation, str) and name == "json_schema": ^^^^^^^^^^^^^^^^^^^^^^^^^^^

Exception Type: TypeError at /en/aa/objects/add/Question/ Exception Value: issubclass() arg 1 must be a class

stephanie0x00 commented 4 months ago

Incident:

image

Environment:

Request Method: POST
Request URL: http://127.0.0.1:8000/en/aa/objects/add/Incident/

Django Version: 4.2.11
Python Version: 3.11.9
Installed Applications:
['whitenoise.runserver_nostatic',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.forms',
 'django_otp',
 'django_otp.plugins.otp_static',
 'django_otp.plugins.otp_totp',
 'two_factor',
 'account',
 'tools',
 'fmea',
 'crisis_room',
 'onboarding',
 'katalogus',
 'django_password_validators',
 'django_password_validators.password_history',
 'rest_framework',
 'tagulous',
 'compressor',
 'reports',
 'knox',
 'csp']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django_otp.middleware.OTPMiddleware',
 'rocky.middleware.auth_required.AuthRequiredMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'rocky.middleware.onboarding.OnboardingMiddleware',
 'csp.middleware.CSPMiddleware']

Traceback (most recent call last):
  File "/app/rocky/rocky/views/ooi_view.py", line 168, in form_valid
    new_ooi = self.ooi_class.parse_obj(form.cleaned_data)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 1118, in parse_obj
    return cls.model_validate(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 551, in model_validate
    return cls.__pydantic_validator__.validate_python(

During handling of the above exception (1 validation error for Incident
meta_data
  Field required [type=missing, input_value={'application': 'Applicat...uo', 'severity': '.u,.'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing), another exception occurred:
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/edit.py", line 153, in post
    return self.form_valid(form)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/rocky/rocky/views/ooi_view.py", line 174, in form_valid
    form.add_error(error["loc"][0], error["msg"])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/forms/forms.py", line 400, in add_error
    raise ValueError(
    ^

Exception Type: ValueError at /en/aa/objects/add/Incident/
Exception Value: 'OOIForm' has no field named 'meta_data'.
stephanie0x00 commented 4 months ago

Autonomous System:

image

Environment:

Request Method: POST
Request URL: http://127.0.0.1:8000/en/aa/objects/add/AutonomousSystem/

Django Version: 4.2.11
Python Version: 3.11.9
Installed Applications:
['whitenoise.runserver_nostatic',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.forms',
 'django_otp',
 'django_otp.plugins.otp_static',
 'django_otp.plugins.otp_totp',
 'two_factor',
 'account',
 'tools',
 'fmea',
 'crisis_room',
 'onboarding',
 'katalogus',
 'django_password_validators',
 'django_password_validators.password_history',
 'rest_framework',
 'tagulous',
 'compressor',
 'reports',
 'knox',
 'csp']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django_otp.middleware.OTPMiddleware',
 'rocky.middleware.auth_required.AuthRequiredMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'rocky.middleware.onboarding.OnboardingMiddleware',
 'csp.middleware.CSPMiddleware']

Traceback (most recent call last):
  File "/app/rocky/rocky/views/ooi_view.py", line 168, in form_valid
    new_ooi = self.ooi_class.parse_obj(form.cleaned_data)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 1118, in parse_obj
    return cls.model_validate(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 551, in model_validate
    return cls.__pydantic_validator__.validate_python(

During handling of the above exception (1 validation error for AutonomousSystem
name
  Field required [type=missing, input_value={'number': 'eoaua'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing), another exception occurred:
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/edit.py", line 153, in post
    return self.form_valid(form)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/rocky/rocky/views/ooi_view.py", line 174, in form_valid
    form.add_error(error["loc"][0], error["msg"])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/forms/forms.py", line 400, in add_error
    raise ValueError(
    ^

Exception Type: ValueError at /en/aa/objects/add/AutonomousSystem/
Exception Value: 'OOIForm' has no field named 'name'.
stephanie0x00 commented 4 months ago

Config image

Environment:

Request Method: POST
Request URL: http://127.0.0.1:8000/en/aa/objects/add/Config/

Django Version: 4.2.11
Python Version: 3.11.9
Installed Applications:
['whitenoise.runserver_nostatic',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.forms',
 'django_otp',
 'django_otp.plugins.otp_static',
 'django_otp.plugins.otp_totp',
 'two_factor',
 'account',
 'tools',
 'fmea',
 'crisis_room',
 'onboarding',
 'katalogus',
 'django_password_validators',
 'django_password_validators.password_history',
 'rest_framework',
 'tagulous',
 'compressor',
 'reports',
 'knox',
 'csp']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django_otp.middleware.OTPMiddleware',
 'rocky.middleware.auth_required.AuthRequiredMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'rocky.middleware.onboarding.OnboardingMiddleware',
 'csp.middleware.CSPMiddleware']

Traceback (most recent call last):
  File "/app/rocky/rocky/views/ooi_view.py", line 168, in form_valid
    new_ooi = self.ooi_class.parse_obj(form.cleaned_data)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 1118, in parse_obj
    return cls.model_validate(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 551, in model_validate
    return cls.__pydantic_validator__.validate_python(

During handling of the above exception (1 validation error for Config
config
  Field required [type=missing, input_value={'ooi': 'DNSAAAARecord|in...53:36', 'bit_id': 'aua'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing), another exception occurred:
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/edit.py", line 153, in post
    return self.form_valid(form)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/rocky/rocky/views/ooi_view.py", line 174, in form_valid
    form.add_error(error["loc"][0], error["msg"])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/forms/forms.py", line 400, in add_error
    raise ValueError(
    ^

Exception Type: ValueError at /en/aa/objects/add/Config/
Exception Value: 'OOIForm' has no field named 'config'.
underdarknl commented 4 months ago

https://github.com/minvws/nl-kat-coordination/blob/abdb9e8821399c5730f02f0bbc571a2252cc4407/rocky/tools/forms/ooi_form.py#L75 is too strict, Fields that are str | None fail this test and since there is no fallback to render fields that are not handled by the above if-tree (which could be a switch), they are never rendered.

https://github.com/minvws/nl-kat-coordination/blob/abdb9e8821399c5730f02f0bbc571a2252cc4407/octopoes/octopoes/models/ooi/network.py#L102 is one such example.

This is broken since: https://github.com/minvws/nl-kat-coordination/commit/f8fc64669b308709f7400c4f235ebf43865dba41

I propose we fix this check to make it more flexible (and check if str is in the Union if its a Union), but also add a default rendered str field for anything unhandled.

underdarknl commented 4 months ago

I also believe our code does not properly handle these unions when it comes to deciding such a union means the field's value is valid even when empty. The current code seems to indicate a Required state for these fields in the form.

underdarknl commented 4 months ago

for 1.16 we will render a default text field using an extra else directive. Make sure to avoid crating a field for the clearance_level.