[FEATURE] Implementing a custom baseline request to examine interesting behaviors

[j3ssie]

Dear Team,

I'm eager to witness BCheck incorporate a tailored baseline request feature to enhance its ability to detect intricate behaviors. I strongly believe this enhancement would enable BCheck to scrutinize a broader spectrum of issues and vulnerabilities compared to its current functionality.

Here's a simple scenario illustrating what I have in mind:

1. Suppose the original URL/request sent from Burp to BCheck is https://example.com/api/settings/public.

2. Now, I aim to conduct a content discovery scan for the specified list of SQL files.

define:
    secret = 
        "back.sql",
        "backup.sql",
        "accounts.sql",
        "backups.sql",
        "clients.sql",
        "customers.sql",
        "data.sql",
        "database.sql"

3. I really want to send a custom baseline request to the URL like https://example.com/hopefullygot404.sql . Let's call this custom_1 request/response

4. Now, I can employ certain conditionals (it would be advantageous to have feature #171 implemented as well) to perform comparisons for each execution, as demonstrated below.

if wordcount({custom_1.response.body}) - wordcount({check.response.body}) > 1000 then
    report issue:
        severity: info
        confidence: certain
        detail: `SQL file disclose {check.request.url}`
end if

Naturally, I believe it would be beneficial for you to integrate utility functions such as length, wordcount, and stringCount('anystring', check.response.body) to enhance the effectiveness of the checking process

[Michelle-PortSwigger]

Hi

Would the differs from conditional help with the type of BCheck you are trying to create? https://portswigger.net/burp/documentation/scanner/bchecks/bcheck-definition-reference#conditionals

[j3ssie]

Hi @Michelle-PortSwigger, yeah the differs from is exactly what I need to check. May I ask what is this function check for? because sometimes the response is different in length but it is the same word count. Particularly noteworthy are wildcard 404 pages that respond with a 200 status code, echoing back the path within their content.

Finally, I really want to do a differs from function with a custom baseline request. like what I said in the initial comment with the URL https://example.com/hopefullygot404.sql

[Michelle-PortSwigger]

differs from checks for quite a few things. It's a checksum of the response body and some relevant headers (such as content type, content length, status code, ETag, and last modified). This isn't an exhaustive list, but should give you a good idea of how it's working.

You can then use it to compare requests using the following format: if {latest.response} differs from {base.response}

I hope this helps. Please let us know if you have any questions.

[j3ssie]

Thank you for the info @Michelle-PortSwigger. Not sure if I should ask in the issue but I'm doing the scan for checking for sensitive files like below:

metadata:
    language: v2-beta
    name: "Sensitive Dot File Exposed"
    description: "Tests for exposed dot file config in current path and at the root directory of site"
    author: "j3ssie"
    tags: "exposure", "sensitive", "config", "dot-file"

define:
    garbage_path = `/.hopefullyget404`
    issueName = `Sensitive Dot File`

run for each:
    # this list of file is actually much more
    sensitive_path =
        "!.gitignore",
        ".htaccess",
        ".CVS",
        ".DS_Store"

given path then
    # replace the potential path with the last path
    send request called check:
        method: "GET"
        replacing path: `{regex_replace({regex_replace({base.request.url}, "^.*?\/.*?\/.*?\/", "/")}, "([^/]+)$", "")}{sensitive_path}`

    # send a random request in the hopes of effectively filtering out false positives
    send request called garbage:
        replacing path: `{garbage_path}`

    if {check.response.status_code} is "200" and
        ({check.response.headers} matches "text/plain" or
        {check.response.headers} matches ".*octet-stream.*") then
        if not({garbage.response.headers} matches "text/plain") and
            not({garbage.response.headers} matches ".*octet-stream.*") then
            # actually check the differs in the response
            if {garbage.response.body} differs from {check.response.body} then
                # make sure some HTML is not present in the response
                if not({check.response.body} matches "(?i)(\<\!doctype|\<html|\<head|\<body)") and
                    not({check.response.body} matches "(?i)(Oops!|Whoops!|AutodiscoverService|not\sfound|Request\sRejected|Access\sDenied|a\sbad\sURL|has\sbeen\slocked)") then
                    report issue and continue:
                        severity: low
                        confidence: tentative
                        detail: `{issueName} found at {check.request.url}. Manual investigation is advised`
                        remediation: `Ensure the file {sensitive_path} are not exposed from your webroot`
                end if
            end if
        end if
    end if

The scan is working fine and can find the sensitive file but there is one problem. The request in the part {garbage} gonna send the request https://example.com/.hopefullyget404 every time the new path send (you can see the picture below.

[Michelle-PortSwigger]

Hi @j3ssie

If this BCheck was tested against the following two URLs, what requests do you want it to send?

https://ginandjuice.shop https://ginandjuice.shop/blog

Are you looking for it to send requests for both https://ginandjuice.shop/.htaccess and https://ginandjuice.shop/blog/.htaccess

or just the one request to https://ginandjuice.shop/.htaccess?

[j3ssie]

Hi @Michelle-PortSwigger, For this validation, I specifically aim to examine solely the final component of the path. Assuming our fundamental request is https://ginandjuice.shop/, I intend to dispatch a request to https://ginandjuice.shop/.htaccess. This shouldn't pose an issue.

However, my inquiry pertains to whether there exists a method to transmit the {garbage} request just once via the URL https://example.com/.hopefullyget404, and subsequently utilize it as a reference point for comparison with other requests.

The actual behavior with my rule is this {garbage} request is every time the new sensitive_path. Please refer to this rule and picture (https://github.com/PortSwigger/BChecks/issues/172#issuecomment-1932586306)

[Michelle-PortSwigger]

Hi

If you're just wanting to send requests to the main URL for the host, you might find a host BCheck a better option.

You can also potentially reduce the number of times the https://example.com/.hopefullyget404 request is sent by looking at where you position it in the flow of your BCheck.

If you use this example as a guide (https://github.com/PortSwigger/BChecks/blob/main/examples/exposed-backup-file.bcheck) then you can see that the {garbage} request is only sent if certain conditions are met, so you could apply the same logic to your BCheck and only send the garbage request if you will need to use it.

I hope this helps.