(optionally) Remove signatures from removed sources - or bulk edit them

[kam193]

Is your feature request related to a problem? Please describe. I've added a new source of Yara rules, but then I decided they are not good enough, so I removed the source. However, the signatures stayed, and I need to remove it manually, one by one.

Describe the solution you'd like It would be nice if removing a source would also remove (or at least disable) related signatures.

Describe alternatives you've considered

1. We may want to ask on removing a source if it should also remove or disable signatures.
2. A nice solution would also be a good bulk editing of signatures. If I could apply a change (remove/disable) to all results of a query, it would be more than enough. Just bulk selecting rules would help, but then we still may have several pages to process

Additional context

[cccs-rs]

Based on the APIs, whenever you delete a signature source, it should consequently remove all the signatures related to that source:

delete_signature_source (Update Source Managment page): https://github.com/CybercentreCanada/assemblyline-ui/blob/20a98c26f9e00804870ca01074c44f52acfd9fcd/assemblyline_ui/api/v4/signature.py#L329

set_service (Services page): https://github.com/CybercentreCanada/assemblyline-ui/blob/20a98c26f9e00804870ca01074c44f52acfd9fcd/assemblyline_ui/api/v4/service.py#L200

Could you provide any further details on this issue if this still persists in the latest stable release?

[kam193]

I've just tested, and it indeed works as expected. Maybe it was some glitch (like failing line 326?). Anyway, thanks for taking a look at this!

[kam193]

Okay, an update: the signature is no longer in the database, but... the Yara service still uses it :) I've just got a hit for the rule I used for the test yesterday, although I cannot search it. I think the service isn't updated after removing rules.

[cccs-rs]

It's possible the service didn't get the new bundle of signatures from the updater at the time of the task.

Services usually ask if there's a new bundle after each task so it's possible it still had the old signature set until you sent another task, meanwhile the updater already had the new bundle ready to send but someone had to ask for it.

That being said if the old signature set is being used across multiple new samples, then it means there's something wrong with the updater getting the message that the signatures have been deleted and could be caching bug if it continues to send the old bundle.

Checking the contents of /tmp/tmp* in the updater container should show what's being sent to services and you can compare against the contents of /updates/tmp* in a service instance.

[kam193]

Yes, this is what I suspect. It's about the Yara service, and I see that meanwhile other sources didn't have any changes detected by the updater. I've checked the folders you mentioned, my test rule source was still in both.

On the service:

On the updater:

However, when I clicked on updating all sources for the Yara service in the AL configuration, one actually got updates (it's a little weird as the last update in the source was 2 weeks ago, but anyway), and thereafter, my test source was gone from the file system:

As so, I suspect that removing a source is effective when another source is changed.

[cccs-rs]

This should be patched in the 4.5.0.28 release (and all services rebuilt from this release).