contentctl is failing with creating new content.

[yaroslav-nakonechnikov]

Hello,

i've just cloned repository and in main branch did initial setup described there: https://github.com/splunk/contentctl?tab=readme-ov-file#from-source-advanced

then i tried to init it:

(contentctl-py3.11) user@pc:~/onedrive/git/contentctl$ contentctl init
Traceback (most recent call last):
  File "/mnt/c/Users/user/git/contentctl/contentctl/contentctl.py", line 196, in main
    init_func(t)
  File "/mnt/c/Users/user/git/contentctl/contentctl/contentctl.py", line 57, in init_func
    Initialize().execute(config)
  File "/mnt/c/Users/user/git/contentctl/contentctl/actions/initialize.py", line 23, in execute
    (config.path/emptyDir).mkdir(exist_ok=False)
  File "/usr/lib/python3.11/pathlib.py", line 1117, in mkdir
    os.mkdir(self, mode)
FileExistsError: [Errno 17] File exists: 'docs'
  File "/home/user/.cache/pypoetry/virtualenvs/contentctl-zVzUC8o4-py3.11/bin/contentctl", line 6, in <module>
    sys.exit(main())
  File "/mnt/c/Users/user/git/contentctl/contentctl/contentctl.py", line 225, in main
    traceback.print_stack()

it failed. and tried several other times, till i've got that:

(contentctl-py3.11) user@pc:~/onedrive/git/contentctl$ mkdir APP
(contentctl-py3.11) user@pc:~/onedrive/git/contentctl$ contentctl init --app.title APP --app.label APP --path APP
The app 'APP' has been initialized. Please run 'contentctl new --type {detection,story}' to create new content

which is not expected, but understandable.

later i've tried to create new content, and it is failing again:

(contentctl-py3.11) user@pc:~/onedrive/git/contentctl$ contentctl new --type detection --path APP
? enter detection name testing_detection
? what kind of detection is this endpoint
? enter author name yn
? enter author name yn
? select a detection type TTP
? Your data source done
? enter search (spl) _some_search_
? enter MITRE ATT&CK Technique IDs related to the detection, comma delimited for multiple T1003.003, T1003
? security_domain for detection endpoint
Traceback (most recent call last):
  File "/mnt/c/Users/user/git/contentctl/contentctl/contentctl.py", line 204, in main
    new_func(config)
  File "/mnt/c/Users/user/git/contentctl/contentctl/contentctl.py", line 92, in new_func
    NewContent().execute(config)
  File "/mnt/c/Users/user/git/contentctl/contentctl/actions/new_content.py", line 95, in execute
    YmlWriter.writeYmlFile(str(full_output_path), content_dict)
  File "/mnt/c/Users/user/git/contentctl/contentctl/output/yml_writer.py", line 10, in writeYmlFile
    with open(file_path, 'w') as outfile:
         ^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'APP/detections/TTP/testing_detection.yml'
  File "/home/user/.cache/pypoetry/virtualenvs/contentctl-zVzUC8o4-py3.11/bin/contentctl", line 6, in <module>
    sys.exit(main())
  File "/mnt/c/Users/user/git/contentctl/contentctl/contentctl.py", line 225, in main
    traceback.print_stack()

why it asks for file which should be created by the contenctl? as i see, it needs directory APP/detections/TTP, but why it doesn't create if it is not exist?

[yaroslav-nakonechnikov]

also, as it requires to create story, it fails again and again:

contentctl new --type story --path APP
? enter story name convert test
? enter author name yn
? select a category done ######################## there was a choice... why it writes done?
? select a use case Other

and as a result i see next:

story_name: convert test
story_author: yn
category: &id001 []
usecase: Other
name: convert test
id: fdb110c7-fb76-4e94-b144-e2c6cc0cfa6a
version: 1
date: '2024-05-28'
author: yn
description: UPDATE_DESCRIPTION
narrative: UPDATE_NARRATIVE
references: []
tags:
  analytic_story: convert test
  category: *id001
  product:
  - Splunk Enterprise
  - Splunk Enterprise Security
  - Splunk Cloud
  usecase: Other
  cve:
  - UPDATE WITH CVE(S) IF APPLICABLE

so what is that id001?

and validator is failing:

File: stories/convert_test.yml
Error: 2 validation errors for Story
tags.usecase
  Input should be 'Fraud Detection', 'Compliance', 'Application Security', 'Security Monitoring' or 'Advanced Threat Detection' [type=enum, input_value='Other', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/enum
tags.analytic_story
  Extra inputs are not permitted [type=extra_forbidden, input_value='convert test', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden

why it gives option to use Other for usecase, but fails? and same for analytic_story: why it fills it with wrong data? and analytic_story: convert test is also added in story file, where validator doesn't expect it.

[ljstella]

Sorry for the delay in this!

We've got an update in #162 that fixes both the new detection issue, as well as the analytic story category issue, and a few other small sharp edges. It should be merged shortly.

[ljstella]

162 is now merged, and 4.0.5 is now released with fixes for these issues. Thanks for your patience!