search

OWASP / OpenCRE

https://opencre.org
Creative Commons Zero v1.0 Universal
79 stars 35 forks source link

[Import] Existing graph contains cycle #552

Closed charifmahmoudi closed 1 month ago

charifmahmoudi commented 2 months ago

Issue

What is the issue?

An error happens when trying to import a mapping after populating the database from upstream

Expected Behaviour

When I run the import on an empty database, it imports correctly

Command to clean the database:

~/OpenCRE$ rm -rf standards_cache.sqlite; make migrate-upgrade

Command to start the import:

~$ curl -X POST http://CRE-LOCAL-SERVER:5000/rest/v1/cre_csv_import -F "cre_csv=@cra_cre.csv"
{"new_cres":["347-352","028-254","820-878","007-274","286-500","732-148","002-202","841-757","240-274","261-010","766-162","731-120","227-045","571-640"],"new_standards":7,"status":"success"}

Result from the server side:

INFO:application.cmd.cre_main:Registering resource CSA CCM of length 1
INFO:application.database.db:knew of node CSA CCM:IVS-06:Vulnerability Remediation:https://cloudsecurityalliance.org/research/cloud-c
                                                               controls-matrix/ ,updating
INFO:werkzeug:161.218.188.108 - - [06/Sep/2024 10:13:14] "POST /rest/v1/cre_csv_import HTTP/1.1" 200 -

Actual Behaviour

Command to clean the database and populate from upstream:

~/OpenCRE$ rm -rf standards_cache.sqlite; make migrate-upgrade; python cre.py --upstream_sync

Command to start the import:

~$ curl -X POST http://CRE-LOCAL-SERVER:5000/rest/v1/cre_csv_import -F "cre_csv=@cra_cre.csv"
<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

Result from the server side:

   return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/csi/OpenCRE/venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/csi/OpenCRE/venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/home/csi/OpenCRE/application/web/web_main.py", line 743, in import_from_cre_csv
    new_cre, exists = cre_main.register_cre(cre, database)
  File "/home/csi/OpenCRE/application/cmd/cre_main.py", line 122, in register_cre
    collection.add_internal_link(
  File "/home/csi/OpenCRE/application/database/db.py", line 1597, in add_internal_link
    cycle = self.__introduces_cycle(f"CRE: {higher.id}", f"CRE: {lower.id}")
  File "/home/csi/OpenCRE/application/database/db.py", line 766, in __introduces_cycle
    raise ValueError(
ValueError: Existing graph contains cycle,this not a recoverable error, manual database actions are required [('CRE: 155-155', 'CRE:
                                                                546-564'), ('CRE: 546-564', 'CRE: 155-155')]
INFO:werkzeug:161.218.188.108 - - [06/Sep/2024 10:08:39] "POST /rest/v1/cre_csv_import HTTP/1.1" 500 -

Steps to reproduce

First clean the database and populate from upstream:

~/OpenCRE$ rm -rf standards_cache.sqlite; make migrate-upgrade; python cre.py --upstream_sync

Second have a CSV file in the correct format to import (name the file cra_cre.csv with the following content)

CRE 0,CRE 1,CRE 2,CRE 3,CRE 4,Cyber Resiliency Act|name,Cyber Resiliency Act|id,Cyber Resiliency Act|hyperlink,NIST 800-53 v5|name,NIST 800-53 v5|id,NIST 800-53 v5|hyperlink,ISO/IEC 27001:2013|name,ISO/IEC 27001:2013|id,ISO/IEC 27001:2013|hyperlink,ASVS|name,ASVS|id,ASVS|hyperlink,CIS v8|name,CIS v8|id,CIS v8|hyperlink,PCI DSS|name,PCI DSS|id,PCI DSS|hyperlink,CSA CCM|name,CSA CCM|id,CSA CCM|hyperlink
,,347-352|Set and confirm integrity of security deployment configuration,,,Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,028-254|Secure auto-updates over full stack,,,Address and remediate vulnerabilities,2.2,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,820-878|Document all trust boundaries and significant data flows,,,Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,,007-274|Patching and updating system components,,Address and remediate vulnerabilities,2.2,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,,286-500|OS security,,Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,732-148|Vulnerability management,,,Address and remediate vulnerabilities,2.2,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,,002-202|Address and remediate,,Address and remediate vulnerabilities,2.2,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,Flaw Remediation,SI-2,https://csrc.nist.gov/publications/detail/sp/800-53/rev-5/final,Management of technical vulnerabilities,A.12.6.1,https://www.iso.org/standard/54534.html,Security Update Verification,V14.4,https://owasp.org/www-project-application-security-verification-standard/,Continuous Vulnerability Management,Control 7,https://www.cisecurity.org/controls/v8/,Ensure that all system components and software are protected from known vulnerabilities by installing applicable vendor-supplied security patches.,6.2,https://www.pcisecuritystandards.org/documents/PCI_DSS_v3-2-1.pdf,Vulnerability Remediation,IVS-06,https://cloudsecurityalliance.org/research/cloud-controls-matrix/
,,,,"841-757|Use approved cryptographic algorithms in generation, seeding and verification of OTPs",Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,,240-274|Log only non-sensitive data,,Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,,261-010|Program management for secure software development,,Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,766-162|Security Analysis and documentation,,,,Address and remediate vulnerabilities,2.2,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,,,731-120|Document requirements for (data) protection levels,Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,,,227-045|Identify sensitive data and subject it to a policy,Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,
,,,571-640|Personal data handling management,,Designed; developed; and produced to ensure cybersecurity,1.1,https://www.european-cyber-resilience-act.com/Cyber_Resilience_Act_Annex_1.html,,,,,,,,,,,,,,,,,,

Third run the command to start the import:

~$ curl -X POST http://CRE-LOCAL-SERVER:5000/rest/v1/cre_csv_import -F "cre_csv=@cra_cre.csv"
northdpole commented 1 month ago

i'm working on this, its not a trivial fix and requires a bit of engineering, will post updates as i get along