search

Skybox-Technologies / csgo-ai-competition

CSGO.ai competition
34 stars 16 forks source link

Submission guidelines feedback #15

Closed kristoff3r closed 3 years ago

kristoff3r commented 3 years ago

Hey everyone,

I've drafted up some submission guidelines on the submission-info branch. I'd like to get some feedback about the process and documentation before I make it final and merge it to master. Please reply to this issue if you find bugs or stuff that can be done smarter.

Before the weekend I'll put up the submission site itself, where you will be able to upload your code. It will also run some sanity checks so you can see if it will work in the evaluation environment.

Nilanshrajput commented 3 years ago

How are we supposed to upload the model for generating test results? and are there some limitation over model sizes?(maybe we can add model weights along with all files)

Nilanshrajput commented 3 years ago

And where do we upload this submission(zip format?

kristoff3r commented 3 years ago

@Nilanshrajput the uploading site will be added in 1-2 days, it isn't done yet. There isn't any hard limit on model sizes. Model weights can be added in the zip file.

kubilay-yavuz commented 3 years ago

Hi, the test set which you will evaluate our models has all the attributes that are in the train set, right? To give an example, will there be a different map_crc value in the test set which is not available in the train set?

kristoff3r commented 3 years ago

@kubilay-yavuz the evaluation data will have the same format/columns, but I cannot guarantee that there won't be a few map_crc / patch_version values that weren't in the training set. I haven't received the data set for evaluation yet, but there is a lot of different values for those, and if the games are played after the training set was made then those values will of course be new.

kubilay-yavuz commented 3 years ago

@kristoff3r Hello, again, the train data was sorted in a time manner, will that be the same for the test data? Thanks in advance

kristoff3r commented 3 years ago

@kubilay-yavuz yes

nperraud commented 3 years ago

@kristoff3r Here is an idea... Why do you just provide the input data and our submission would be the solution? This seems much more simple to me.

kristoff3r commented 3 years ago

@nperraud I agree that would be much simpler, but

  1. for the finals we need to be able to run the models anyway
  2. I haven't received the complete evaluation data set yet

I'm trying to make it as low friction as possible given the constraints.

andrew-son commented 3 years ago

@nperraud Since the demos are taken from the pro matches, there is a possibility that you can do some reverse engineering and find matches on hltv which would defeat the purpose of the competition but you would still win it

kristoff3r commented 3 years ago

Just a quick update: I'm waiting for green light from one of our partners before putting the site up, but because we're getting close to the deadline I've asked them to extend the deadline a bit. I'll let you know when I get a response.

kristoff3r commented 3 years ago

The submission site is up now: https://ai.skybox.gg

It will stay up until the 21st. Let me know if you experience any problems.

kkhrystosenko-morebis commented 3 years ago

Hello, can I upload the code on the 21st of August? @kristoff3r

Nilanshrajput commented 3 years ago

@kristoff3r any chance the submission website will remain live for one more day, after the deadline? or deadline gets extended by a day, would really help if possible.

kristoff3r commented 3 years ago

@kristoff3r any chance the submission website will remain live for one more day, after the deadline? or deadline gets extended by a day, would really help if possible.

@Nilanshrajput I'll let the submission site stay up for another day so you can upload, but I need to check with the other parties in the competition if we can still accept it. I'll get back to you.