Testing on custom questions/queries

ryanzhumich / editsql

MIT License

203 stars 67 forks source link

Testing on custom questions/queries #20

Open Param-Raval opened 4 years ago

Param-Raval commented 4 years ago

I was able to train/test on the given Spider and "spider_data_removefrom" sets but I want to test the model on a given question-query. I tried editing the queries in the dev.json file but the predictions remain the same as the queries in the original dev.json. Can you help me out?

gouldju1 commented 4 years ago

Akshaysharma29 commented 4 years ago

Hi, @Param-Raval have you solved this issue as I am facing the same issue(on Sparc dataset).

Param-Raval commented 4 years ago

Yes, I was able to get a workaround. There are certain folders that need to be deleted in order for the model to register the changes. Delete these and run the training script again:

logs_sparc_editsql
processed_data_sparc_removefrom
processed_data_sparc_removefrom_test

After the preprocessing steps, these will appear again but with the changes you made in the json file.

Akshaysharma29 commented 4 years ago

Hi @Param-Raval thanks for the response but the issue is due to OOV as described in the below issue https://github.com/ryanzhumich/editsql/issues/21 Hence Solving OOV issue has solved this issue.

imanuelroz commented 2 years ago

@Akshaysharma29 Hi I tryied the adaptation proposed by Param but as I add new data I am facing the problem that you mentioned with OOV, did you manage to solve it?