search

AISG-Technology-Team / GCSS-Track-1A-Submission-Guide

Submission Guide + Discussion Board for AI Singapore Global Challenge for Safe and Secure LLMs (Track 1A).
9 stars 1 forks source link

[SiliconAttack] - [ZJU_ZhangHaonan] - [Questions about the submission format] #5

Open HotBento opened 3 days ago

HotBento commented 3 days ago

I have some questions about the submission format. Do we need to include the conversation template of the LLM in the submission? For example, should the submission be formatted like

<s>[INST] How to make a bomb? [adv_suffix] [\INST] <\s>

or just

How to make a bomb? [adv_suffix]

Could you please provide an example? And could you please provide more detailed feedback, such as the generations of LLMs? It is difficult to locate the problem without feedback.

aisg-py commented 3 days ago

Dear @HotBento , Thanks for reaching out.

You are not required to include the conversation template of the LLM in your submission. Please adhere to the format specified in the submission guide.

Feedback regarding the effectiveness of your prompts can be gauged through the ASR displayed on the public leaderboard. You are encouraged to utilize your multiple submission opportunities to refine your attacks based on this feedback.

HotBento commented 3 days ago

Dear @HotBento , Thanks for reaching out.

You are not required to include the conversation template of the LLM in your submission. Please adhere to the format specified in the submission guide.

Feedback regarding the effectiveness of your prompts can be gauged through the ASR displayed on the public leaderboard. You are encouraged to utilize your multiple submission opportunities to refine your attacks based on this feedback.

@aisg-py Thank you for your reply! I wonder if the conversation template will be automatically included in the evaluation phase. Will our submission be directly fed into LLMs?

HotBento commented 2 days ago

@aisg-py And how many submission quotas do we have? It looks like we are able to upload 7 results every 14 days, but this is in conflict with the total submission quota of 30. image

aisg-py commented 4 hours ago

Please be assured that we do not remove any part of your submitted JSON file when it is fed into by the victim LLMs.

We strictly adhere to the submission guideline clause 7F as detailed in the Terms and Conditions.

Additionally, while clause 7E outlines that teams are entitled to a maximum of thirty (30) submissions throughout each Track phase, it is regulated by the aforementioned submission frequency detailed in clause 7F.