search

salesforce / TabularSemanticParsing

Translating natural language questions to a structured query language
https://arxiv.org/abs/2012.12627
BSD 3-Clause "New" or "Revised" License
222 stars 51 forks source link

Unable to reproduce the results from pretrained checkpoint #29

Closed saparina closed 9 months ago

saparina commented 3 years ago
After fixing the bugs mentioned in #14 (downgrade src/semantic_parser/learn_framework.py L206-225 and moz_sp/__init__.py restore_clause_order function), I got the following results from your pretrained checkpoints: E-SM EXE
70.0 67.9

Do you know the reason for this difference? Could you please make the code compatible with your checkpoints (fix #14)?

xyzCS commented 3 years ago

After fixing the bugs mentioned in #14 (downgrade src/semantic_parser/learn_framework.py L206-225 and moz_sp/__init__.py restore_clause_order function), I got the following results from your pretrained checkpoints:

E-SM EXE 70.0 67.9 Do you know the reason for this difference? Could you please make the code compatible with your checkpoints (fix #14)?

why my accuracy j

After fixing the bugs mentioned in #14 (downgrade src/semantic_parser/learn_framework.py L206-225 and moz_sp/__init__.py restore_clause_order function), I got the following results from your pretrained checkpoints:

E-SM EXE 70.0 67.9 Do you know the reason for this difference? Could you please make the code compatible with your checkpoints (fix #14)?

The experimental result I got is only 0.52. Did you just modify the error code and then use their checkpoint to get this result?

CannonWWW commented 9 months ago

After fixing the bugs mentioned in #14 (downgrade src/semantic_parser/learn_framework.py L206-225 and moz_sp/__init__.py restore_clause_order function), I got the following results from your pretrained checkpoints:

E-SM EXE 70.0 67.9 Do you know the reason for this difference? Could you please make the code compatible with your checkpoints (fix #14)?

could you please tell how to fix the bug? I would greatly appreciate your urgent assistance with this matter.

saparina commented 9 months ago

@CannonWWW Sorry, I don't remember the details but I think I just followed this advice: https://github.com/salesforce/TabularSemanticParsing/issues/14#issuecomment-790586474