tenable / pyTenable

Python Library for interfacing into Tenable's platform APIs
https://pytenable.readthedocs.io
MIT License
354 stars 173 forks source link

The "workbenches.assets" method returns "BadRequestError [400]" when filters use tag values with whitespaces #554

Closed fatzyyy closed 1 year ago

fatzyyy commented 2 years ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Using iPython3 (or standard interactive shell of python):

  1. First we test with workbenches.vulns method

In [14]: from tenable.io import TenableIO In [15]: tio = TenableIO() In [16]: filters = [("tag.Agents", "set-has", "Windows Servers")] In [17]: response = tio.workbenches.vulns(*filters, age=7) In [18]: len(response) Out[18]: 394

We received 394 assets.

  1. How we switch to workbenches.assets method using the same tuple in the filters list.

response = tio.workbenches.assets(*filters, age=7)

[400: GET] https://cloud.tenable.com/workbenches/assets?filter.0.filter=tag.Agents&filter.0.quality=set-has&filter.0.value=Windows+Servers&date_range=7&all_fields=full body=b'{"statusCode":400,"error":"Bad Request","message":"Error"}'

BadRequestError Traceback (most recent call last) Input In [19], in <cell line: 1>() ----> 1 response = tio.workbenches.assets(*filters, age=7) File ~/repos/sca/sca-venv/lib64/python3.8/site-packages/tenable/io/workbenches.py:104, in WorkbenchesAPI.assets(self, *filters, **kw) 101 else: 102 query['all_fields'] = 'full' --> 104 return self._api.get('workbenches/assets', params=query).json()['assets']

File ~/repos/sca/sca-venv/lib64/python3.8/site-packages/restfly/session.py:603, in APISession.get(self, path, **kwargs) 577 def get(self, 578 path: str, 579 **kwargs 580 ) -> Union[Box, BoxList, Response]: 581 ''' 582 Initiates an HTTP GET request using the specified path. Refer to 583 :obj:requests.requestfor more detailed information on what (...) 601 >>> resp = api.get('/') 602 ''' --> 603 return self._req('GET', path, **kwargs)

File ~/repos/sca/sca-venv/lib64/python3.8/site-packages/restfly/session.py:559, in APISession._req(self, method, path, **kwargs) 557 continue 558 else: --> 559 raise error_resp 561 elif status in range(200, 299): 562 # As everything looks ok, lets pass the response on to the 563 # error checker and then return the response. 564 resp = self._resp_error_check(resp, **kwargs)

BadRequestError: [400: GET] https://cloud.tenable.com/workbenches/assets?filter.0.filter=tag.Agents&filter.0.quality=set-has&filter.0.value=Windows+Servers&date_range=7&all_fields=full body=b'{"statusCode":400,"error":"Bad Request","message":"Error"}'

We tested few other tags and error only occurs when workbenches.assets is used in combination with tags that have whitespaces in their values.

Expected behavior The workbenches.assets should work with tags where values have whitespace characters.

Screenshots N/A

System Information (please complete the following information):

Additional context pyTenable==1.4.3

csanders-git commented 2 years ago

Here is the issue, I actually JUST ran into this same thing and dug a bit deeper.

The tenable servers won't accept a '+' sign as URL encoding for a space which requests/restify is going to use as the default URL encoding method. They will respond with {"statusCode":400,"error":"Bad Request","message":"Error"}

Yes, I understand that this is against spec, but I assume no one here is surprised by tenable violating spec.

What you need is to match what the UI does, which is use %20 for space chars.

However, there is no option to replace these. Even monkey-patching can run into stickey issues as i'll show,

for instance, you might be tempted to pass something like (example) ... {'filter.0.filter': 'tag.Scan%20Type', 'filter.0.quality': 'set-has', 'filter.0.value': 'Office%20Public%20IPs'}

Obviously this will fail the parse_filters() checks https://github.com/tenable/pyTenable/blob/6661f516fb2be54f95ea6e39b8970bb05357649b/tenable/io/workbenches.py#L30 - as Scan%20%Type != Scan Type - however even if you were to bypass that with something like

tenable.io.WorkbenchesAPI._workbench_query = (lambda w,x,y,z: {'filter.0.filter': 'tag.Scan%20Type', 'filter.0.quality': 'set-has', 'filter.0.value': 'Office%20Public%20IPs'}) you'll STILL fail this because requests will encode the '%' as %2520 (double encoded) so you'll end up with something like ... /workbenches/assets?date_range=30&filter.0.filter=tag.Scan%2520Type&filter.0.quality=set-has&filter.0.value=Office%2520Public%2520IPs .

As you're likely aware the real crux of the issue therefore is how https://github.com/tenable/pyTenable/blob/6661f516fb2be54f95ea6e39b8970bb05357649b/tenable/io/workbenches.py#L104 works (return self._api.get('workbenches/assets', params=query).json()['assets'])

There are two short term solutions here, and both are going to involve NOT using workbenches.assets directly. As I mentioned before, the real fix should be to fix the Tenable Web Servers to accept '+' in compliance with RFCs.

1) you can ignore the tio_handler.workbenches.assets function and call

self.tio_handler.get('/workbenches/assets?date_range=30&filter.0.filter=tag.Scan%20Type&filter.0.quality=set-has&filter.0.value=Office%20Public%20IPs')

you'll have to generate your own filter string and affix it to the URL (you can internally call tio_handler.workbenches._parse_filters() which will return a list of these filters) - but as the UI will make this request anyway, you can just inspect the request to determine the filter formatting.

2) you can ignore tio_handler.workbenches.asset function and call tio_handler and treat the args as a JSON body. I haven't read all the API docs, but I doubt this is 'truly' supported, but it does work. The advantage here is you can directly pass the output of

query = tenable.io.WorkbenchesAPI._workbench_query(part_target_filter, {}, self.tio_handler.filters.workbench_asset_filters())

into tio_handler.get(). so it'd look like the following

self.tio_handler.get('workbenches/assets', json={'filter.0.filter': 'tag.Scan Type', 'filter.0.quality': 'set-has', 'filter.0.value': 'Office Public IPs'})

or all at once

query = self.tio_handler.workbenches._workbench_query([("tag.Scan Type", "set-has", "Office Public IPs")], {}, self.tio_handler.filters.workbench_asset_filters())
self.tio_handler.get('workbenches/assets', json=query)

This will actually work (not sure why it accepts a body value with JSON but not '+' urlencoding sigh, but whatever -- the output result is

GET /workbenches/assets HTTP/1.1
Host: cloud.tenable.com
User-Agent: python-requests/2.27.1
Accept-Encoding: gzip, deflate
Accept: */*
Connection: close
X-APIKeys: REMOVED
Content-Length: 132
Content-Type: application/json

{"date_range": "30", "filter.0.filter": "tag.Scan%20Type", "filter.0.quality": "set-has", "filter.0.value": "Office%20Public%20IPs"}
csanders-git commented 2 years ago

btw, normally i'd make a PR to fix this, but i firmly believe the right fix for this is for Tenable to fix their webservers instead of manually crafting GET URLs in situations where spaces can be passed.

aseemsavio commented 2 years ago

You might find the exports.assets() function helpful.

aseemsavio commented 1 year ago

@fatzyyy @csanders-git Exports API does what you're trying to achieve in a much better way. Instead of workbenches, you can use the export APIs.