openreview / openreview-scripts

12 stars 7 forks source link

Need for a tutorial / user-oriented documentation #1096

Open tvercaut opened 5 years ago

tvercaut commented 5 years ago

It's unclear how the samples should be used and if any specific permissions are required for the user.

I tried installing openreview-py through pip and the run python samples/python/get-reviewers.py --baseurl https://openreview.net MIDL.io/2019/Conference/ But this led to empty results:

submissions:
[]
notes:
[]
Reviewers:

I would be interested in understanding what's wrong with how I am trying to use the tool.

melisabok commented 5 years ago

This script is out of date, you can see the openreview-py documentation here:

https://openreview-py.readthedocs.io/en/latest/

There are some examples how to get different types of notes:

Get notes by invitation:

notes = c.get_notes(invitation='MIDL.io/2019/Conference/-/Full_Submission')

Get notes by a regex invitation:

notes = c.get_notes(invitation='MIDL.io/2019/Conference/-/Paper.*/Official_Review')
tvercaut commented 5 years ago

Is there an explanation of the construct of the invitation field? For example, How do I get only the accepted papers. I tried the following but can't see anything related to acceptance.

#!/usr/bin/env python3

## Import statements
import argparse
import csv
import sys
from openreview import *
import httpimport

## Import statements and argument handling
parser = argparse.ArgumentParser()
parser.add_argument('--cpath', help="conference path ex. MyConf.org/2017", default="MIDL.io/2019/Conference/Abstract")
parser.add_argument('-o','--output', help="The directory to save the output file")
parser.add_argument('--baseurl', help="base url", default="https://openreview.net")
parser.add_argument('--username', required=True)
parser.add_argument('--password', required=True)
args = parser.parse_args()

## Initialize the client library with username and password
client = Client(baseurl=args.baseurl, username=args.username, password=args.password)

## get conference directory
base_path = "https://raw.githubusercontent.com/iesl/openreview-scripts/master/venues/"+args.cpath
config_path = base_path+"/python/"
config_url = config_path+"config.py"

## load conference specific data
print("Loading "+config_url)
with httpimport.remote_repo(['config'], config_path):
    import config

conference = config.get_conference(client)
#print(conference)

iterator = tools.iterget_notes(client, invitation=conference.get_submission_id())

notes = {}
for note in iterator:
    #print(note.number)
    notes[note.id]= {
        'submission': note,
        'reviews': [],
        'metareview':[],
    }
    print(note)

print("\nRetrieved: " + str(len(notes)) + " entries")
carlosmondra commented 5 years ago

We just added an example in the docs that does exactly this. Please follow the link below to see it: https://openreview-py.readthedocs.io/en/latest/workshop.html#retrieving-all-accepted-submissions-for-a-conference

tvercaut commented 5 years ago

Thanks. This answers my questions for now. I haven't closed the issue as I still feel that a high-level documentation / tutorial would be beneficial but feel free to close it if you think this is a too vague task.

Additionally, I just also saw that you added a similar functionality here: https://github.com/iesl/openreview-py/blob/a3b3b90c17ad1fffd48b9cafaedda6d344427007/openreview/conference/builder.py#L302

This doesn't work for me but I have filed a specific issue on this: https://github.com/iesl/openreview-py/issues/316