Coordinate flag submission responses?

[Trolldemorted]

cc @MarkusBauer @F30 @ldruschk @MMunier @domenukk

Interviews with unsupervised (external) bambi teams that participated in BambiCTF#5 have shown that an exploit parallelization and submission tool would have been super convenient, as they struggled to scale their exploits against all teams. Since Bambi CTF targets newcomers and casual players, we decided to develop bambiXploit, which will run your exploit once per team per round (parallel, but with staggered starts), collect flags from stdout, submit them to our gameserver (reconnects etc. included), and display some statistics.

Would you be interested in unifying our flag submission responses to make it work for the majority of A/D CTFs? We have private flag submission tooling with more advanced features, but we'd be happy to establish an open-source minimum standard for new CTF teams.

To achieve that, we would propose to harmonize our responses so that bambiXploit can identify the semantics of the result while leaving room for ctf-framework-specific details and flair. Do you think we can agree on certain prefixes like OK, DUP, OWN, OLD, INV and ERR?

If you want to keep the possibility to respond asynchronously without strict ordering (I think all of us are using HMACs and can reject certain submissions without waiting for a database), we can include the flag into the prefix like FOO{BARBAZ} OK, so bambiXploit (and your own tooling) can attribute responses to flags. However, for this we have to agree on a delimiter char/byte (unicode flags ptsd incoming) which all of us will not use in our flags. Maybe a whitespace?

If you know who to tag from the ructf(e) and ictf teams please do so :)

[domenukk]

I summon @Lukas-Dresel (if that works?)

[F30]

To achieve that, we would propose to harmonize our responses so that bambiXploit can identify the semantics of the result while leaving room for ctf-framework-specific details and flair. Do you think we can agree on certain prefixes like OK, DUP, OWN, OLD, INV and ERR?

I guess we can do that, though not in time for this year's FAUST CTF.

If you want to keep the possibility to respond asynchronously without strict ordering (I think all of us are using HMACs and can reject certain submissions without waiting for a database), we can include the flag into the prefix like FOO{BARBAZ} OK, so bambiXploit (and your own tooling) can attribute responses to flags. However, for this we have to agree on a delimiter char/byte (unicode flags ptsd incoming) which all of us will not use in our flags. Maybe a whitespace?

I don't think we have a need for unordered responses, but "no whitespace in the flag" seems fine for me.

Speaking of whitespace/delimiters, should we probably agree on a more complete protocol than just the responses? Stuff like "one flag per line ('\n' separated)", welcome banner separators etc.?

[Trolldemorted]

I guess we can do that, though not in time for this year's FAUST CTF.

No hurries, I would not want to touch prod infrastrucuture 4 weeks before showtime either :)

Speaking of whitespace/delimiters, should we probably agree on a more complete protocol than just the responses? Stuff like "one flag per line ('\n' separated)", welcome banner separators etc.?

You are right, if someone wants to have a welcome banner we must define a delimiter for it. I guess we all agree on "one flag per line, linux linefeeds", do you have a proposal how we should detect the end of the banner? \n\n maybe?

[F30]

You are right, if someone wants to have a welcome banner we must define a delimiter for it. I guess we all agree on "one flag per line, linux linefeeds", do you have a proposal how we should detect the end of the banner? \n\n maybe?

That's what I had in mind, yes.

[Trolldemorted]

I have moved this issue to our specification repo, since our flag submission endpoint should be defined here anyway.

Regarding exploit parallelization: We will be shipping bambixploit-dotnet on the bambictf6 vulnboxes, and have changed our flag submission endpoint to send a welcome banner, delimited with \n\n, and we include the flag in the response to make both the clients and server easier to implement.

Since most of us disclose flag hints with complex structures, we included bambictf/enowars flag hint parsing in our template(s). Are your flag hint structures sufficiently stable that we should add code for them too?

[Trolldemorted]

CCing @Lukas-Dresel because it looks like he has taken over the development of iCTF's infrastructure :)

[F30]

We have now implemented this in the FAUST CTF Gameserver and wrote a spec for it, which can be found at https://ctf-gameserver.org/submission/. Looking forward to any feedback.

[Trolldemorted]

Awesome! Will you have testing occasions without the real services for this year's faustctf? If you want to include bambixploit into your vulnboxes we will gladly help, if you have an HTTP endpoint that serves a \n-separated list of teamips it should be a walk in the park (fingers crossed).

[Lukas-Dresel]

Holy what is this magic of cross-repo summoning? Unfortunately I don't really check my github notifications all that often ...

[Lukas-Dresel]

@domenukk thanks for the ping :)

[Lukas-Dresel]

I think a standard like this would be great to have, however, for the iCTF at the very least having a tcp-style interface as described above is not straightforward since submission of flags is available on the open internet, not from the VPN only (by design). Therefore, we would at the very least need support for some form of authentication since we cannot otherwise determine who the submitting team is before we could do this.

[Lukas-Dresel]

For the response prefixes themselves, I see no reason why we could not change those, however, ours are provided via authenticated REST API requests, not a TCP interface unfortunately, so I'm not sure how much value doing so would add to the above use-case of a unified flag submitter

[Lukas-Dresel]

We could abuse the banner idea for the same thing? But in that case I'd suggest modifying the proposed delimiter, since human-readable prompts can include \n\n quite frequently (any blank lines following text for example)

[F30]

Awesome! Will you have testing occasions without the real services for this year's faustctf? If you want to include bambixploit into your vulnboxes we will gladly help, if you have an HTTP endpoint that serves a \n-separated list of teamips it should be a walk in the park (fingers crossed).

The submission server will be available before the start of the CTF in principle (without much functionality, though). Don’t expect there to be a testing instance.

Afaik there are currently no plans to include an exploiter with the Vulnbox. Team numbers/IPs will be provided in form of the usual teams.json.

For the response prefixes themselves, I see no reason why we could not change those, however, ours are provided via authenticated REST API requests, not a TCP interface unfortunately, so I'm not sure how much value doing so would add to the above use-case of a unified flag submitter

Yeah, I don‘t think this proposal is compatible with a REST API (or should be). The only thing that seems applicable are the response codes.

We could abuse the banner idea for the same thing? But in that case I'd suggest modifying the proposed delimiter, since human-readable prompts can include \n\n quite frequently (any blank lines following text for example)

What do you mean by "same thing"? The \n\n was born out of a desire for simple banners (see above) and I think it currently serves our purposes well.

[Lukas-Dresel]

What I mean is, if iCTF were to adopt this text-based format for submission (or just expose it additionally), it would have to support authentication of some kind, since the authentication is how we distinguish who is submitting a flag. iCTF submissions happen (or at least can happen) from outside the game network, so we can't use the source IP to distinguish who is submitting the flag.