Deploy YARA rules remotely and privately

[zayhanlon]

User story
As a detection & response engineers,
I want to deploy YARA rules to agents remotely and privately from a server I host myself (separate from Fleet)
so don't have to write rules to disk (too large of scope and too slow) or host rules on a non-private webserver (what osquery supports today).

• Too large of scope: If they wrote the rules to disk, any engineering (including outside detection & response team) could read these rules because they'd live in the organization's monorepo. If an engineer's account gets compromised then they can read these rules.
• Too slow: It takes months to get these rules reviewed.
• Privately: Viewing rules must require authentication. Currently, osquery doesn't provide a way to authenticate YARA rules.

Context

• Product designer: @zwass

Changes

Product

• [ ] No Fleet UI/CLI/API or fleetd changes
• [ ] osquery changes: Support deploying YARA rules to agents remotely behind authenticated endpoints

Engineering

• [ ] Reference documentation changes: TODO
• [ ] Usage documentation changes: TODO
• [ ] Database schema migrations: TODO
• [ ] Load testing: TODO

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Requires load testing: TODO
• Risk level: Low / High TODO
• Risk description: TODO

Manual testing steps

1. Step 1
2. Step 2
3. Step 3

Testing notes

Confirmation

1. [ ] Engineer (@____): Added comment to user story confirming successful completion of QA.
2. [ ] QA (@____): Added comment to user story confirming successful completion of QA.

[noahtalerman]

Feature fest: We have a yara table in Fleet: https://fleetdm.com/tables/yara

Does that address the need?

[zayhanlon]

Related to an ask from customer-domon re: #19553

[JoStableford]

Related to a Slack conversation

[nonpunctual]

The yara & yara_events' tables do not have the capabilities to access sigrules from a remote server.

[zayhanlon]

@noahtalerman let me know if you or design team would like to chat with the customer during or after this air guitar. thanks!

[noahtalerman]

Hey @zayhanlon! Would love some help setting up a call w/ the customer.

[noahtalerman]

Containers supported by YARA (Specifically Kubernetes): Brock: Need context → Adding namespace column by PID would allow you to use YARA rules in containers kubequery in the repo may be helpful May be a blocker since they don’t have direct Kubernetes API Brock: Could pass along to infra team to use Ben: Potentially, could be hard though Concern was if they could use osquery to run YARA rules in containers that are Docker → Now use Kubernetes so want that same ability for Kube osquery does not support containerd → this would resolve the issue TODO Brock: Bring this to eng to discuss Found article about osquery support for containerd https://developer.ibm.com/articles/monitoring-containers-osquery/

[nonpunctual]

Hey @noahtalerman I did reach out to IBM on their GitHub to see if they had open-sourced anything in relation to this. Got no response. That said, I think we could duplicate what was built based on the article.

[noahtalerman]

Hey @zayhanlon, I updated this air guitar to the user story format and moved our original issue description below.

Bringing this one back to feature fest to weigh whether we'll commit to solving this one next sprint or in a later sprint.

---

Original issue description:

Ideal workflow: Customer would have a repo of YARA rules that only detection & response team can access. Deploy these per team (production servers and workstations).

Some tables are blocked by not having YARA

Future: How to displace CrowdStrike? Event monitoring. osquery evented tables aren't as good and performant as CrowdStrike.

Problem

Background:

• Currently not using YARA scanning at all today
• One option - put the rules on disk, it would take forever. If there was an incident and they needed to add new rules, that timeline wouldn’t work. Not a tenable option.
• Another option - pull from a web server, but then it would need to be publicly accessible. Then attackers could then see what their rules are.
• DART team currently has a task to build a YARA scanning capability. Osquery provides this capability but the methods to implement are not ideal for our environment

Potential solutions

1. Build the feature for Fleet to be given a set of YARA rules like Osquery query packs. Next, build the feature for Osquery to pull those Osquery rules from the TLS server.
2. Where would this live in Fleet?
   • Same as query packs
   • Would be able to be configured as ‘YARA events’ scheduled to run at a recurring configurable interval of frequency
3. Benefit to them: YARA rules would be the method of distributing out the rule and detecting the presence of new malware signatures out in their environment
   • Crowdstrike doesn’t support YARA rules today
   • Would make it easier for other DART teams to use this feature (on a moments notice, be able to write a rule for new

[zayhanlon]

Thank you @noahtalerman

[noahtalerman]

Hey @zwass, I'm passing this story to you and tagged you as the product designer.

Please bring this story + your proposed changes to the design review ritual so that we can provide a second pair of eyes on changes and give feedback.

Thanks!

cc @zayhanlon

[noahtalerman]

Hey @zayhanlon because this is being worked on by customer success I think it makes sense to move this issue to the #g-customer-success board. I moved it.

When @zwass brings changes through design review and we decide that it's ready to estimate, I think at that point we add the issue back to the drafting board (:product) so that we can bring it through a product group's estimation session.

Please let me know if you disagree!

[zayhanlon]

sounds good! i was tracking it separately, but we can follow your guidance. this will be zach's next issue after he's back from leave.