Velociraptor gives an "Unknown artifact" error

[WD-ST]

We create custom collectors using an API call and while the collector is created without a problem when we run it, we get an "Unknown artifact" error, This error happens regardless if it is an "offline" collector or a collector that uploads to S3.

Based on some testing it seems we get this error when a collector is created under the following circumstances: • the collector is created to run on Linux • the collector is created through the API • the collector is created to contain three or more custom artifacts.

We created a similar collector for Windows and we had no issues.

We can also manually create a collector through the GUI that contains all 6 of our custom artifacts and the collector will complete successfully without errors.

The Linux collector we were trying to create 6 different custom artifacts and we have tried these artifacts individually and with two custom artifacts bundled and we don't experience this issue, it only occurs when there are three or more custom artifacts when creating a Linux collector through the API.

Thanks

[scudette]

Did you add the custom artifact to the server that created the collector?

When you say create the collector through the API do you mean calling the Server.Utils.CreateCollector artifact?

[WD-ST]

Good Afternoon Scudette,

Yes, I manually created the custom artifacts in the GUI beforehand.

So i can create the collectors manually through the GUI using those artifacts, but we had a python script created for us to automatically automate the collector creation process and discovered the error appears for only Linux collectors which contains three or more custom artifacts when using the script (but Windows works with no problems.)

The python script does use Server.Utils.CreateCollector as part of the collector creation process.

Thanks

[scudette]

As you see the artifact adds dependent artifacts into the collection https://github.com/Velocidex/velociraptor/blob/acc40f08d50d9860d43abcb9f761e9e64859a121/artifacts/definitions/Server/Utils/CreateCollector.yaml#L370

You should be able to see it add the dependent artifacts into the package in the query logs of Server.Utils.CreateCollector - can you attach the output? Also include the parameters to the Server.Utils.CreateCollector artifact (that the python script launched).

[WD-ST]

# Run artefact to create collector, response contains vfs path on server
response = run_server_artifact("Server.Utils.CreateCollector",api,collector_parameters,timeout=9000,verbose=True)

I can see the custom artifacts have been passed through, I have attached two pictures showing this.. one is manually created and one created through the API/Script..

Other than order, the only difference I can see is there is a space between the collector when it is created by the script

You can see it in the request, this is a Linux collector created through the api/script

{ "VQLClientAction": { "principal": "api_user", "env": [ { "key": "OS", "value": "Linux" }, { "key": "artifacts", "value": "[\n \"Custom.Linux.Phase2.LogCollection\",\n \"Custom.Linux.Phase2.ProcessList\",\n \"Custom.Linux.Phase2.Netstat\",\n \"Custom.Linux.Phase2.BashHistory\",\n \"Custom.Linux.Phase2.SSHHosts\"\n]" }, { "key": "template" },

But that space also appears when you create a Windows collector using the script, which works as designed.

I hope this information is what you were after

[scudette]

That's interesting it looks like you are calling the Json API? That is solely for use by the GUI and may be changed in future. For the API we only really support the vql query API using the grpc call.

It should work though 🙂 can you look at the query log for the artifact. You should be able to see the in the log tab of the server Artifact screen

[WD-ST]

I didn't create the script, it was created for us to assist in automating the collector process as at times we need to create approximately 40 collectors at once with different "OutputPrefix", but I will take note that this process may break in future due to changes in the future and advise our project team.

Interesting the logs say "artifact_definitions: Artifact not found" even though it shows the artifacts under "Artifact collection - Parameters" and the "Request" sections.

logs (1).csv

Thanks for the quick responses, Just a note that I wont be able to reply until Monday after this comment,

Have a great weekend.

[scudette]

Can you confirm the custom artifacts are visible in the "View artifacts" screen? Are you sure they are loaded into the server?

[scudette]

You should be able to emulate what the CreateCollector VQL is doing using the following query (run in a notebook)

SELECT name, built_in
FROM artifact_definitions(
   names=["Admin.Client.Upgrade"], 
   deps=TRUE)

Substitute the Admin.Client.Upgrade with your custom artifact. You should be able to get back all your custom artifacts as well as any dependencies (this is how we decide which artifacts to include in the collector bundle).

You can also inspect the generated config file with velociraptor_collector.exe config show to see the included artifact definitions. We dont include the definitions that are built in - did you include your custom artifacts in the config file ?

[WD-ST]

Can you confirm the custom artifacts are visible in the "View artifacts" screen? Are you sure they are loaded into the server?

Sure can

I will test the VQL today and get back to you.

[WD-ST]

So same thing happens using the VQL provided

When I have my 5 customer artifacts I get "artifact_definitions: Artifact not found"

However if I remove three and run the VQL with only 2 custom artifacts, it works

When I run the CLI command I get the artifacts in the config

Using the script collector:

Using the GUI created collector:

[scudette]

Can you check your custom artifact definitions for external dependent artifacts? for example

SELECT * FROM Artifact.Custom.Dependent()

It might be complaining that it can not find one of the artifacts your custom one is referring to instead.

[WD-ST]

Sorry you lost me here,

I ran SELECT * FROM Artifact.Custom.Dependent() and got Plugin Artifact.Custom.Dependent not found.

However I assume I need to change that command to match my custom artifacts, but couldn't work out how.

Apart from the logcollection artifact, most of the others are slightly modified version (or in one case a direct copy) of the inbuilt artifacts, just renamed to make it easier during manual packaging.

[scudette]

No I meant to look for usage similar to the above. If the artifact calls another artifact then it will select from Artifact.OtherArtifactName and that must be resolvable and exist.

Maybe we are barking up the wrong tree because you said that using the GUI properly gives the correct result but only when calling via the API you are getting artifact not found. I suspect this is something that is wrong with the python script calling the API - can you attach the python scipt?

Alternately you can use the proper API (via VQL) e.g. using pyvelociraptor to call the collect_client() VQL. See https://docs.velociraptor.app/docs/server_automation/server_api/ for help with using the API

[WD-ST]

Sorry for the delay, just getting approval to attach the python script as I'm unsure whose IP the script is.

Will also looking into the proper API

[WD-ST]

Sorry for the delay...

This is the script we call from command line using python3

#!/usr/bin/python
import os
import shutil
import csv
import argparse
import pyvelociraptor
import boto3
from botocore.exceptions import ClientError
import configparser
from generate_emails import create_eml
from pyvelociraptor_wrappers import run_server_artifact, run_vql, fetch

#Load Configuration File
config = configparser.ConfigParser()
config.read('config.ini')

# Load Velociraptor Configuration File 
api = pyvelociraptor.LoadConfigFile(config['APP']['API'])

def update_YaraRules():
    """
    Updates the Velociraptor "tool" for YaraRules with the supplied URL in config.ini 
    Arg: 
        None
    Returns:
        None
    """
    tool_name = "YaraRules"
    # Craft VQL Query
    query = 'SELECT inventory_add(tool="{}", url="{}") FROM scope()'.format(tool_name,config['APP']['YaraURL'])
    run_vql(query,api)

def create_collector(OS,agency_tag,artifact_list):
    """
    Create collections based on predefined arguments and tags.
    Arg: 
        OS {str} - Operating System
        agency_tag {str} - Tag per agency
        artifact_list {arr}- List of Artifacts to be included in the collector
    Returns:
        response {obj} - obj that includes the vfs path of the collector
    """
    # Craft VQL Query - TODO modify local results option
    if config["S3-Results"]["OutputLocal"] == "TRUE":
        print("[+] Collector will output data to a local directory")
        collector_parameters = "dict(OS='{}',artifacts={},opt_output_directory='{}_{}_',target='ZIP',target_args=dict())".format(OS,artifact_list,config['S3-Results']['OutputPrefix'],agency_tag)
    else:
        print("[+] Collector will output data to AWS S3")
        target_arg_str = "dict(bucket='{}',credentialsKey='{}',credentialsSecret='{}',region='{}')".format(config['S3-Results']['Name'],config['S3-Results']['Key'],config['S3-Results']['Secret'],config['S3-Results']['Region'])
        collector_parameters = "dict(OS='{}',artifacts={},opt_output_directory='{}_{}_',target='S3',target_args={})".format(OS,artifact_list,config['S3-Results']['OutputPrefix'],agency_tag,target_arg_str)

    # Run artefact to create collector, response contains vfs path on server
    response = run_server_artifact("Server.Utils.CreateCollector",api,collector_parameters,timeout=9000,verbose=False)

    print("[+] {} - {} collector created: {}\n".format(agency_tag,OS,response[0]['Binary']['Path']))

    return response

def download_collector(vfs_path, size, phase, agency_tag, output=config['APP']['Output']):
    """
    Download the specified collector executable from the Velociraptor server.
    Implements Velociraptor's "fetch" server artefact, only available in Velociraptor versions >(0.6.4? TBC)
    Arg:
        vfs_path {str} - Path on the Velociraptor server from which to download the collectors
        agency_tag {str} - Tag per agency
        output {arr} - Directory on the local file system to write the downloaded collectors to
    Returns:
        Local path of downloaded collector
    """
    if not os.path.isdir(output):
        os.makedirs(output)

    # Use same filename for the local file as VFS
    filename = phase + "_" + agency_tag + "_" + vfs_path.split('/')[-1]
    output_path = os.path.join(output, filename)

    # https://github.com/Velocidex/velociraptor/issues/1587 (FIXED)
    print("[-] Downloading collector to path " + output_path + "...")
    fetch(api,vfs_path,size,output_path)
    print("[-] Download complete")
    return output_path

def upload_collector(collector_file):
    """
    Upload the specified local collector file to AWS
    Most details read from configuration file
    Arg: 
        collector_file {str} - Path of collector file on the local machine
    Returns:
        Link to uploaded collector
    """
    # If an error has occurred, exclude this collection from the upload
    print("[-] Uploading specified file " + collector_file)
    if not collector_file:
        print("[-] No file was uploaded as no path was given to upload from.")
        return False
    bucket_name = config["S3-Collectors"]["Name"]
    aws_region = config["S3-Collectors"]["Region"]
    aws_key = config["S3-Collectors"]["Key"]
    aws_secret = config["S3-Collectors"]["Secret"]

    # Perform the upload
    upload_name = os.path.basename(collector_file)
    s3_client = boto3.client('s3',
                             aws_access_key_id=aws_key,
                             aws_secret_access_key=aws_secret,
                             region_name=aws_region)

    try:
        s3_client.upload_file(collector_file, bucket_name, upload_name)
    except ClientError as e:
        print("[-] Upload to S3 failed, please attempt manually")
        print(e)
        return False

    # upload has finished, generate URL for download
    url = "https://s3-{location}.amazonaws.com/{bucket_name}/{key}".format(location=aws_region, bucket_name=bucket_name, key=upload_name)
    print("[-] Upload complete, URL {}".format(url))
    return url

def main():
    """
    Main function. Creates, downloads, and uploads collectors, then creates email templates to send.
    Arg: 
        None
    Returns:
        None
    """
    parser = argparse.ArgumentParser(
        description="Tool to generate offline collector agents to scan yara.",
        epilog='Example: generate_yara_collectors.py')

    if config['APP']['YaraURL'] != '':
        print("[-] Setting Yara URL to: {}".format(config['APP']['YaraURL']))
        update_YaraRules()
    else:
        print("[-] Using Yara URL from Velociraptor Server")

    # Phase1 Arifact List - Windows

    p1_artifact_list_win = ['Custom.Windows.Search.Yara',
                            'Custom.Windows.EventLogs.Chainsaw']

    # Phase1 Arifact List - Linux

    p1_artifact_list_nix = ['Custom.Linux.Search.Yara']

    # Phase2 Arifact List - Windows

    p2_artifact_list_win = ['Custom.Windows.Phase2.LogCollection',
                            'Custom.Windows.Phase2.ProcessList',
                            'Custom.Windows.Phase2.StartupItems',
                            'Custom.Windows.Phase2.Drivers',
                            'Custom.Windows.Phase2.Timeline']

    # Phase2 Arifact List - Linux

    p2_artifact_list_nix = ['Custom.Linux.Phase2.LogCollection',
                            'Custom.Linux.Phase2.ProcessList',
                            'Custom.Linux.Phase2.Netstat',
                            'Custom.Linux.Phase2.BashHistory',
                            'Custom.Linux.Phase2.SSHHosts']

    # Define whether results are to be saved locally or in S3
    local_results =  config["S3-Results"]["OutputLocal"]

    with open(config['APP']['Tags'], encoding="utf8") as f:
        csv_reader = csv.DictReader(f)
        # Iterate through agency tags and call collector functions
        for tag in csv_reader:
            # Phase1 - Create Collectors
            links = { "windows": False, "linux": False }
            if config['APP']['Phase1Collectors'] == "TRUE":
                print("[-] Generating Phase 1 Collectors for " + tag['agencytag']) 
                # Create OS specific collectors
                if config['APP']['Windows'] == "TRUE":
                    response = create_collector("Windows",tag['agencytag'], str(p1_artifact_list_win))
                    collector_path = download_collector(response[0]['Binary']['Path'],response[0]['Binary']['Size'], "Phase1", tag['agencytag'])
                    links["windows"] = upload_collector(collector_path)
                if config['APP']['Linux'] == "TRUE":
                    response = create_collector("Linux",tag['agencytag'], str(p1_artifact_list_nix))
                    collector_path = download_collector(response[0]['Binary']['Path'],response[0]['Binary']['Size'], "Phase1", tag['agencytag'])
                    links["linux"] = upload_collector(collector_path)
                # send emails
                if (not links["windows"] and not links["linux"]):
                    print("[!] No Phase 1 collectors were created as no operating system was selected, or an error reading the configuration file occurred. Please check the configuration file to address this.")
                else:
                    create_eml("Phase1",config["S3-Results"]["OutputLocal"], tag['agencytag'], tag['agencyemail'], links["windows"], links["linux"], config['APP']['OutputEmails'])   

            # Phase2 - create collectors
            links = { "windows": False, "linux": False }
            if config['APP']['Phase2Collectors'] == 'TRUE':
                print("[-] Generating Phase 2 Collectors for " + tag['agencytag'])

                 # Create OS specific collector
                if config['APP']['Windows'] == "TRUE":
                    response = create_collector("Windows",tag['agencytag'], p2_artifact_list_win)
                    collector_path = download_collector(response[0]['Binary']['Path'],response[0]['Binary']['Size'], "Phase2", tag['agencytag'])
                    links["windows"] = upload_collector(collector_path)
                if config['APP']['Linux'] == "TRUE":
                    response = create_collector("Linux",tag['agencytag'], p2_artifact_list_nix)
                    collector_path = download_collector(response[0]['Binary']['Path'],response[0]['Binary']['Size'], "Phase2", tag['agencytag'])
                    links["linux"] = upload_collector(collector_path)
                # send emails
                if (not links["windows"] and not links["linux"]):
                    print("[!] No Phase 2 collectors were created as no operating system was selected, or an error reading the configuration file occurred. Please check the configuration file to address this.")
                else:
                    create_eml("Phase2", config["S3-Results"]["OutputLocal"], tag['agencytag'], tag['agencyemail'], links["windows"], links["linux"], config['APP']['OutputEmails'])

if __name__ == '__main__':
    main()

Some of the triggers are reliant on a config file, where you can set certain parameters

 # Generate Windows Collectors
Windows = TRUE
# Generate Windows 32bit Collectors
Windows_x86 = FALSE
# Generate Linux Collectors
Linux = TRUE
# Create Phase 1 collectors
Phase1Collectors = TRUE
# Create Phase 2 collectors
Phase2Collectors = FALSE

[scudette]

I am not sure if you still have an issue with it but these days it is easier to create the offline collector using the commandline as described here

https://docs.velociraptor.app/knowledge_base/tips/automate_offline_collector/

Please reopen if you still have issues with it