Enhancement: service aliases to workaround case-sensetive ServiceNames

[markus-lassfolk]

Possibility to add Service Aliases for the API, so we can call for alternate names/casings in addition to the real name as the API is case sensetive.

Context:

ELJeffe: When submitting a task via the api, we have "services": {"selected": ["MetaDefender", "...", "..."]}. Seems the services are case sensitive (at least on my initial tests). Would be nice if they weren't as instead of passing various tool names, I now have to specify if "metadefender" then "MetaDefender" for any particular service. I expect the same is true of others, like VirusTotal.

cccs-sgaron: It is case sensitive, you can have two service with the same name with different cases... cccs-sgaron: Was that a good design choice, probably not, but that prevents us from removing the case sensitiveness in the API

markus-lassfolk: Would it maybe be possible for you to add Service Aliases in the API so we can call for alternate names/casings in addition to the real name?

cccs-sgaron: It's possible. Create a ticket in the Assemblyline ui repo. This might not make it as an high priority item though.

[eljeffeg]

Perhaps a quick resolution might be a "try again" approach by checking for the case-sensitive name first. Then if False, instead of failing, check case-insensitive. This might preserve the backward compatibility of allowing case-sensitive service names when there is duplication.

[cccs-rs]

How far would this need of service name aliasing need to go exactly? Just for ingests/submits to the system?

[eljeffeg]

That would cover my use case. I think the better target might be the API endpoints to cover all integrations. I can't recall if there are any other queries that take the service name, but I'd consider those as well if it's not a heavy lift.

[cccs-sgaron]

Can we tackle this a completely different way and just reject the API call if you've provided the name of a service that does not exist? That way, we dont have to implement a list of service aliases and manitain that list and if you submit a file to a service that does not exist, the call will be rejected and you will know that you've messed up. Cause in the end, your in a the same boat if you typo the name even when aliases exists...

[eljeffeg]

My use case was case-sensitivity conflicts, as we have a client application that uses lowercase virustotal, metadefender, etc. If you're checking if a service exists, than for this case, I expect it could replace it. For example:

servicename = getServiceName(servicename)

def getServiceName(servicename):
    for service in services:
        if service.lower() == servicename.lower():
            return service
    return None

I expect a replace of spaces, underscores, dashes could be handled similarly if desired. A full alias beyond that would probably be a lot more involved. @markus-lassfolk what was your use case? Not sure if I'll use it or not, but I did like the trick of being able to submit a binary without running a service for the purpose of just adding it to the archive for retro hunting.

[markus-lassfolk]

It was mostly that it felt strange to have to memorize the casing for all services to call them. And then using casing to distinguish between two different services sharing the same name feels prone for error. My imagination is too limited to find a scenario where I would like to call MetaDefender or MEtaDefender or metadefender and consider that 3 different services in the backend.

[cccs-sgaron]

The only reason for them being case sensitive is that we've used the service name as the document key in elastic and the document key is case sensitive there. And now we're kind of stuck with it for legacy reasons. Even if we where to provide service Aliases this would only apply at input because if you where to do searches for those services results afterwards you'd still have to use their case-sensitive value... That is kind of why I'm reluctant to this change as I am right now without a solution to move to case-insensitive everywhere without reindexing all the data on currently running systems.

[cccs-rs]

Status on issue? Is this still something we're interested in pursuing?

[eljeffeg]

I'm still interested in the feature - making the service name case insensitive when doing a ingests/submit via API.