Issues with importing challenges

[ZetaTwo]

We encountered issues when trying to import challenges. The situation was something like this:

• We had 12 challenges already import
• We tried to import 19 challenges (7 new)
• Parsing worked fine
• Clicking "import" sends a request with the correct data
• The request eventually times out with a 504.

I know this isn't super useful debug info. I'm in the middle of a CTF but afterwards I'll try to recreate it and see if I find any error messages or logs.

[SakiiR]

Hey @ZetaTwo !

How are u doing ?

We had a similar issue (#17) earlier, do you think it can be related to that somehow ?

This commit should fix the bug https://github.com/TFNS/CTFNote/commit/0abdc24c7618b9d5c2e35a9122ef105ec063e3f3

Are you sure to be using the last version of CTFNote ?

That would be helpfull to have a typical json or raw import to reproduce. ( You can also try with the JSON file provided in #17 against your version just to be sure )

[ZetaTwo]

Hello! I'm good! How are you? :D

I installed the CTFNote instance on Wednesday last week by cloning the repo (main branch) and launching it so I'm pretty sure it's a recent version. Also I did not see multiple requests for adding the challenges. All challenges were sent in one request in an array so it does not seem to be that issue.

This was the json I tried to import:

{"success": true, "data": [{"id": 1, "type": "dynamic", "name": "Welcome", "value": 50, "category": "Web", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 4, "type": "dynamic", "name": "diveinternal", "value": 50, "category": "Web", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 5, "type": "dynamic", "name": "Your Note", "value": 50, "category": "Web", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 9, "type": "dynamic", "name": "babycrypto1", "value": 50, "category": "Crypto", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 10, "type": "dynamic", "name": "babycrypto2", "value": 50, "category": "Crypto", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 11, "type": "dynamic", "name": "babycrypto3", "value": 50, "category": "Crypto", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 12, "type": "dynamic", "name": "babycrypto4", "value": 50, "category": "Crypto", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 13, "type": "dynamic", "name": "atelier", "value": 310, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 7, "type": "dynamic", "name": "babysandbox", "value": 428, "category": "Web", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 8, "type": "dynamic", "name": "babyweb", "value": 428, "category": "Web", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 15, "type": "dynamic", "name": "bank", "value": 428, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 23, "type": "dynamic", "name": "Sakura", "value": 428, "category": "Reversing", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 14, "type": "dynamic", "name": "babychrome", "value": 445, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 25, "type": "dynamic", "name": "SQG (rev)", "value": 445, "category": "Reversing", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 3, "type": "dynamic", "name": "Janken", "value": 460, "category": "Web", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 22, "type": "dynamic", "name": "pprofile", "value": 460, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 2, "type": "dynamic", "name": "3233", "value": 472, "category": "Web", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 17, "type": "dynamic", "name": "pwnbox", "value": 496, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 19, "type": "dynamic", "name": "Damn Vulnerable Box", "value": 496, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 6, "type": "dynamic", "name": "doublecheck", "value": 499, "category": "Web", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 18, "type": "dynamic", "name": "SQG (pwn)", "value": 499, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 21, "type": "dynamic", "name": "Query Firewall", "value": 499, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 16, "type": "dynamic", "name": "FLAG Trust", "value": 500, "category": "Pwnable", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}, {"id": 24, "type": "dynamic", "name": "ultrushawasm", "value": 500, "category": "Reversing", "tags": [], "template": "/plugins/dynamic_challenges/assets/view.html", "script": "/plugins/dynamic_challenges/assets/view.js"}]}

Note that the client side parsing did seem to work without issues and I got the same problem when I manually extracted the name/category and did "raw parser" instead. Also the request looked fine but it stalled and eventually timed out.

[ZetaTwo]

What I did instead was to manually just import 1-3 challenges at a time and that worked. Could it be that there's some accidental O(n^3) action going on or something?

[ZetaTwo]

Ok, I tried updating to latest stable commit: ef4e45354f6ea7d8fb28a5a70b02dc08a045e84b and restarting with docker-compose down && docker-compose up --build -d and tried adding the same tasks as above again.

This is the full request (slightly censored):

POST /api/ctf/test/task HTTP/1.1
Host: example.com
Connection: keep-alive
Content-Length: 1574
Pragma: no-cache
Cache-Control: no-cache
sec-ch-ua: "Google Chrome";v="89", "Chromium";v="89", ";Not A Brand";v="99"
DNT: 1
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36
Content-Type: application/json
Accept: */*
Origin: https://example.com
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://example.com/
Accept-Encoding: gzip, deflate, br
Accept-Language: sv,en-US;q=0.9,en;q=0.8,ko;q=0.7
Cookie: [CENSORED]

{"tasks":[{"title":"Welcome","category":"Web","Exists already":false},{"title":"diveinternal","category":"Web","Exists already":false},{"title":"Your Note","category":"Web","Exists already":false},{"title":"babycrypto1","category":"Crypto","Exists already":false},{"title":"babycrypto2","category":"Crypto","Exists already":false},{"title":"babycrypto3","category":"Crypto","Exists already":false},{"title":"babycrypto4","category":"Crypto","Exists already":false},{"title":"atelier","category":"Pwnable","Exists already":false},{"title":"babysandbox","category":"Web","Exists already":false},{"title":"babyweb","category":"Web","Exists already":false},{"title":"bank","category":"Pwnable","Exists already":false},{"title":"Sakura","category":"Reversing","Exists already":false},{"title":"babychrome","category":"Pwnable","Exists already":false},{"title":"SQG (rev)","category":"Reversing","Exists already":false},{"title":"Janken","category":"Web","Exists already":false},{"title":"pprofile","category":"Pwnable","Exists already":false},{"title":"3233","category":"Web","Exists already":false},{"title":"pwnbox","category":"Pwnable","Exists already":false},{"title":"Damn Vulnerable Box","category":"Pwnable","Exists already":false},{"title":"doublecheck","category":"Web","Exists already":false},{"title":"SQG (pwn)","category":"Pwnable","Exists already":false},{"title":"Query Firewall","category":"Pwnable","Exists already":false},{"title":"FLAG Trust","category":"Pwnable","Exists already":false},{"title":"ultrushawasm","category":"Reversing","Exists already":false}]}

And this is the response:

HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Date: Tue, 30 Mar 2021 22:25:46 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 12
Connection: keep-alive
Cache-Control: no-cache
Etag: W/"c-vYhJ+637j7hGzLiQJl8ECNUD9Wk"
Pragma: no-cache
X-Powered-By: Express

{"tasks":[]}

The request no longer hangs but instead no tasks are added at all. I have tried to reduce this to a simple test case and it seems that a space in the challenge name is causing issues. If add only the following:

your note|web

in the raw parser. It does not add a challenge and if you input:

abc|web
Your Note|web

it adds the "abc" challenge but not "Your Note".

[SakiiR]

I could not reproduce the issue. However, there's a known issue with the tasks unicity.

So if you don't burn ur database and keep the test database as is ( containing multiple CTF with multiple tasks ), there is a chance the tasks could not be added because they already exist in another CTF.

Can you retry please ? but after removing the database volume ( if you did not do it already ^^ ).

This still is an issue and should be considered. However, I just wanna make sure this import fails because of this.

[ZetaTwo]

Yes, this seems to indeed be the cause. I retried the original payload from above and no challenges were added. Then I prefixed all the names with "test-" and tried again and all of them were added. So yeah as you say it seems that challenge names are unique instead of the (challenge name, event) pair being unique.

[SakiiR]

Hey @ZetaTwo

The bug should now be fixed on the dev branch !

We are planning to do an update of the main branch soon, and i'll close this issue when it's done :)

[ZetaTwo]

Awesome! Thanks!

[XeR]

Fixed in v2.0.0 Check MIGRATION.md to upgrade