OpenGVLab / ChartAst

ChartAssistant is a chart-based vision-language model for universal chart comprehension and reasoning.
Other
64 stars 5 forks source link

About the functions defined in numerical QA #4

Closed zhangliang-04 closed 5 months ago

zhangliang-04 commented 5 months ago

Thank you for the great work! After reading the paper carefully, I have the following questions:

  1. How many functions are defined when constructing COT over numerical QA? It seems that they are numpy functions (Fig.2). Could you provide a list of them?
  2. Do you invoke these numpy functions during evaluation for numerical questions?
FanqingM commented 5 months ago

Hi, Zhangliang Most of the functions used are functions provided by numpy, including max, min, substract, etc. The specific list is as follows: func_list = [ 'getitem', 'select', 'numpy.array', 'numpy.ndarray', 'numpy.argmax', 'numpy.argmin', 'numpy.max', 'numpy.min', 'numpy.sum', 'numpy.add', 'numpy.subtract', 'numpy.multiply', 'numpy.divide', 'numpy.<', 'numpy.<=', 'numpy.>', 'numpy.>=', 'numpy.==', 'numpy.!=', 'numpy.mean', 'numpy.median', 'numpy.std', 'numpy.var', 'numpy.abs', 'numpy.sqrt', 'numpy.square', 'numpy.log', 'numpy.exp', 'numpy.power', 'numpy.sort', 'numpy.delete', 'numpy.all', 'numpy.any', 'numpy.diff', 'numpy.corrcoef', 'numpy.cov', 'np.array', 'np.ndarray', 'np.argmax', 'np.argmin', 'np.max', 'np.min', 'np.sum', 'np.add', 'np.subtract', 'np.multiply', 'np.divide', 'np.<', 'np.<=', 'np.>', 'np.>=', 'np.==', 'np.!=', 'np.mean', 'np.median', 'np.std', 'np.var', 'np.abs', 'np.sqrt', 'np.square', 'np.log', 'np.exp', 'np.power', 'np.sort', 'np.delete', 'np.all', 'np.any', 'np.diff', 'np.corrcoef', 'np.cov', 'max', 'min', 'sum', 'len', 'str', 'int', 'float', 'abs', 'round', '<', '<=', '>', '>=', '==', '!=' ]

During testing, we first use ChartAst to generate executable json, and then execute this json through the backend and call the corresponding function to get the calculated answer. In this way, the model can avoid most calculation problems; at the same time, compared to Directly train the answer end-to-end. The training data required by this method will be much smaller than the end-to-end case.