search

OWASP / wstg

The Web Security Testing Guide is a comprehensive Open Source guide to testing the security of web applications and web services.
https://owasp.org/www-project-web-security-testing-guide/
Creative Commons Attribution Share Alike 4.0 International
7.28k stars 1.32k forks source link

Standardizing capture output #583

Closed victoriadrake closed 3 years ago

victoriadrake commented 4 years ago

Discussion in #573 brought up the point of adding a standard format for capture output to the contribution guide. This would exemplify how much information from the capture to provide, as well as the format to provide it in.

The goal is to present the information in an easy-to-read format, with a level of accuracy that is appropriate for educational purposes.

The last iteration of the proposed format is:

Request:

Request URL: https://site-under.test/securityRealm/createAccount
Request method: POST
...
POST data:
username=user456
fullname=User 456
password1=My-Protected-Password-808
password2=My-Protected-Password-808

Response:

Response headers:
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID.a7731d09=node011yew1ltrsh1x1k3m6g6b44tip8.node0; Path=/; Secure; HttpOnly

Originally posted by @victoriadrake in https://github.com/OWASP/wstg/pull/573#issuecomment-703983146

Any further discussion points or iterations that better achieve the goal are welcome! Otherwise, this issue is for the implementation of the new standard form in the contribution guide and test case examples.

patrickceg commented 4 years ago

I won't have time until later this week to even know if I'll have the time for this task, but I think the first step would be to know which of the articles either have captures or should have captures. This is both for follow-up tasks and to know if that format fits all the other articles.

For example, that format may make less sense if you have multiple captures in a row (since you'll take up more vertical space). It would be good to know if any test cases have a chain of requests (for example if the attack contains intentionally invalid items like multiple redundant request headers: I've seen that in some attacks that fool content delivery networks - CDN).

ThunderSon commented 4 years ago

I prefer having the raw HTTP requests and responses, instead of cleaning them up for readers. That's how they'll be seen everywhere. Everything that is not relevant to the test scenario at hand is removed, except the first 2 lines (GET / .... HTTP/1.1 and Host) for example. If let's say this is testing auth, a cookie or a token should be present, or a certain service fingerprint.

kingthorin commented 4 years ago

I'm in agreement with @ThunderSon.

patrickceg commented 4 years ago

We should also pick a way to cut out excess information, preferably some way that doesn't mess with all the format scripts. I think ellipsis with no surrounding brackets, so ... seemed to work for the issue I worked on.

Things I think won't work

github-actions[bot] commented 3 years ago

Please comment if you are still working on this issue, as it has been inactive for 30 days. To give everyone a chance to contribute, we are releasing it to new contributors.

patrickceg commented 3 years ago

You can assign me to this, given I want to try to get my personal projects up again over the next few weeks. (Enough virtual vacations in video games 🎮 😆)

My plan is as follows:

patrickceg commented 3 years ago

Hmm there were more articles with captures than I thought... like last time, I am using a GIST scratchpad https://gist.github.com/patrickceg/79900464dfd23ca349970f2d8efd51ae#file-wstg583-md

Question: I noticed different forms of request, some of them using tooling including Telnet, cURL, or ncat. Should we standardize on a tooling call for making the request?

example: switch all of the articles to cURL unless there's a compelling reason not to

kingthorin commented 3 years ago

I’d say the whole “more than one way to skin a cat” thing is good.

Don’t get caught up on tool output, focus just on raw requests/responses.

victoriadrake commented 3 years ago

I'd add that if it helps clarity, we can say "all examples in this book use X tool to show output" but only if it makes it less complicated, not more.

patrickceg commented 3 years ago

(The CMOS battery of the machine I primarily use for this project failed, so I'm waiting on the special CR2032 with welded tabs to come in.)

Ok, I'll dig in to each of the uses with calls to tooling to see how their request/response is used in the context of the article. My first pass was really just seeing where are there elements that look like an HTTP request or response.

kingthorin commented 3 years ago

Thanks @patrickceg, please keep in mind:

  1. The repo/project are in a bit of a lull still.
  2. We don't want this to be a huge time sink for you (or anyone) so stay in-touch, and think about whether or not it's practical to break into chunks.
patrickceg commented 3 years ago

No problem: I barely spent 2 hours so far, and half of that was setting up all my Github stuff for the first time in a few months.

patrickceg commented 3 years ago

Do we have any other examples of specific recommendations for pieces of content?

I notice https://github.com/OWASP/wstg/blob/master/template/999-Foo_Testing/2-Template_Explanation.md has the general template recommendations. However, blindly adding something as specific as the format for an HTTP response would be out of place.

I was thinking of a blurb similar to the following, but I'm not sure where to put it for pull request:

HTTP Requests and Responses

If the test case requires examples of HTTP requests and responses, format the requests and responses as a capture in an HTTP code block.

Example request:

GET /html/rfc7231 HTTP/1.1
Host: tools.ietf.org
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36 Edg/88.0.705.81
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Referer: https://tools.ietf.org/html/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,fr;q=0.8
If-None-Match: "aaaaaa-aaaaa-aaaaaaaaaaaaa;aaaaaaaaaaaaa"
If-Modified-Since: Sun, 21 Feb 2021 08:13:25 GMT

Example response headers:

HTTP/1.1 200 OK
Date: Tue, 02 Mar 2021 21:34:03 GMT
Server: Apache/2.2.22 (Debian)
Expires: Tue, 09 Mar 2021 21:34:03 GMT
Cache-Control: max-age=604800
Vary: negotiate,Accept-Encoding
Content-Location: rfc7231.html
TCN: choice
Last-Modified: Sun, 21 Feb 2021 08:13:25 GMT
ETag: "aaaaaa-aaaaa-aaaaaaaaaaaaaa;aaaaaaaaaaaaa"
Accept-Ranges: bytes
Content-Encoding: gzip
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Clacks-Overhead: GNU Terry Pratchett
Content-Type: text/html; charset=UTF-8

(TODO: What about the body?)

Use ellipsis ... if it is necessary to cut existing content if you plan to only highlight parts of the response. Example:

HTTP/1.1 200 OK
...
Content-Type: text/html; charset=UTF-8
victoriadrake commented 3 years ago

@patrickceg I think that what you've put together so far makes sense. I agree about

However, blindly adding something as specific as the format for an HTTP response would be out of place.

Since we may have more examples for things like captured output, code samples, etc, do you think it would make sense to add a new document or directory under OWASP/wstg/blob/master/template/ to house these?

patrickceg commented 3 years ago

Sorry for the late reply: I'm going to have to check notifications settings because I didn't get one for that message.

Since it sounds like we have no structure for specific examples, I'll try to make one under the template directory and see how it looks / how to link to it from the style guides or other parts of the contribution guide.

ThunderSon commented 3 years ago

By checking the provided examples, this still has the issue of having all headers. What's the purpose in examples that contain all possible headers in a request? I can send an HTTP/1.1 request with barely the host and the path after connecting to the server's port.

patrickceg commented 3 years ago

Noted: I'm finally getting my study routine back, so I'll make sure the example shows something about cutting out items. (...assuming my keyboard stops glitching as it did just as I was typing this 😒)

EDIT: In case there's context misinterpretation, I've been doing front-end and articles for my work project for the past few weeks, and it's been messing with my concentration to do front-end wiki articles here. I'm not bottling up a 100 page thesis for my pull request 😁 In fact, I'm hoping the instructions I make for this fit on one tablet's screen.

patrickceg commented 3 years ago

Since the format is settled, here's a list of the pages that deviate enough that it's worth spending time to change.

Reformat: These articles could use a big reformat, but since there's already issue #598 to put them together, the format of blocks can just be mentioned in that task

Minor changes that could all be batched into a single pull request since they're just find and replace:

Note:

github-actions[bot] commented 3 years ago

Please comment if you are still working on this issue, as it has been inactive for 30 days. To give everyone a chance to contribute, we are releasing it to new contributors.