search

Nix-Security-WG / nix-security-tracker

Web service for managing information on vulnerabilities in software distributed through Nixpkgs
31 stars 7 forks source link

Bulk CVE ingester is not adapted for production operations #153

Closed RaitoBezarius closed 4 weeks ago

RaitoBezarius commented 10 months ago

I am trying to bulk ingest some CVE on the staging deployment:

WARNING 2023-12-20 13:58:15,142 ingest_bulk_cve 216849 140574326007616 $DATA_CACHE_DIRECTORY was not set. Using the local dir.
Traceback (most recent call last):
  File "/nix/store/2m3lrr252l71c4c55gbh6cr13qgqblls-python3.11-web-security-tracker-0.0.1/bin/..manage.py-wrapped-wrapped", line 23, in <module>
    main()
  File "/nix/store/2m3lrr252l71c4c55gbh6cr13qgqblls-python3.11-web-security-tracker-0.0.1/bin/..manage.py-wrapped-wrapped", line 19, in main
    execute_from_command_line(sys.argv)
  File "/nix/store/vinfkk30jy2560i66q9pk6nsns9y7jhk-python3.11-Django-4.2.7/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/nix/store/vinfkk30jy2560i66q9pk6nsns9y7jhk-python3.11-Django-4.2.7/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/nix/store/vinfkk30jy2560i66q9pk6nsns9y7jhk-python3.11-Django-4.2.7/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/nix/store/vinfkk30jy2560i66q9pk6nsns9y7jhk-python3.11-Django-4.2.7/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/2m3lrr252l71c4c55gbh6cr13qgqblls-python3.11-web-security-tracker-0.0.1/lib/python3.11/site-packages/shared/management/commands/ingest_bulk_cve.py", line 107, in handle
    data_cache_dir, cve_data_cache_dir = self._set_cve_data_cache_dir()
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/2m3lrr252l71c4c55gbh6cr13qgqblls-python3.11-web-security-tracker-0.0.1/lib/python3.11/site-packages/shared/management/commands/ingest_bulk_cve.py", line 102, in _set_cve_data_cache_dir
    mkdir(path.join(data_cache_dir))
OSError: [Errno 30] Read-only file system: '/nix/store/2m3lrr252l71c4c55gbh6cr13qgqblls-python3.11-web-security-tracker-0.0.1/lib/python3.11/site-packages/shared/management/commands/.data_cache'

Ideally, it should make use of the systemd default $RUNTIME_DIRECTORY and just push the cache inside /run/web-security-tracker by "default" cleverly and the delta cron should reuse this fact automagically.

cc @alejandrosame @fricklerhandwerk