Compare the dump data of vidyut with dump data of ashtadhyayi.com

[drdhaval2785]

70 and #71 lead me to raise this query.

In majority of derivations, vidyut is doing great. In some minority of cases like this, there may be wrong forms.

A systematic way of analysis would be to compare the dump of both vidyut and ashtadhyayi.com to analyze the differences. We can iron out the differences. There may have been some error in implementing some sUtra or in understanding grammar commentaries at some place.

Kind attention @akprasad please.

[akprasad]

@drdhaval2785 , +cc @neeleshb

Setup

I've written a simple analysis script, which you can find here.

• This script compares a recent copy of dhatuforms.txt with the output of vidyut-prakriya when running make create_test_files.
• I compared only kartari-tinantas.
• I excluded forms ending with d because vidyut-prakriya creates forms like अभविष्यद्, अभूद्, etc. which cause noise in the comparison.

Summary

• 1525 dhatus have an exact match in all kartari padas.
• 666 have at least one mismatch.
• 38 dhatus are generated by Vidyut but don't have forms on ashtadhyayi.com This includes dhatus like श्रा, .etc.

Here's the full diff:

compare.log

In each mismatch, the first row contains forms in Vidyut that are missing from ashtadhyayi.com, and the second row contains the opposite.

Specific differences

I'm still looking through the results, but here are some patterns that I believe are in Vidyut's favor:

• correct abhyasa in lit forms (1.82, 1.160, 1.166, 1.171, 1.182, 1.209, ...)
• use of णत्व in चोरयाणि etc. (1.160, 1.472, 1.473, 1.477, 1.478, 1.479, ...)
• use of अनिट्-थल् (1.1135, 1.1136, 1.1139, ...)

Vidyut doesn't have any systemic errors that I can see, though I'm sure many of the differences are minor errors that need fixing, such as खवान vs. खौनीहि. I am also limited in my own grammatical knowledge, so I cannot fully access what I'm seeing without more research.

I'll keep this issue open to track the differences. @drdhaval2785 and @neeleshb, please look through compare.log at your leisure and help me understand how some of these differences should be resolved.

[akprasad]

Also +cc @vipranarayan14

[vipranarayan14]

As per my experience (through manually checking and working with both), Vidyut's data is more accurate compared to ashtadhyayi.com's data. But Vidyut is also missing few alternate/special forms and has a few incorrect forms. The most accurate and complete data of verb forms I have come across is that of SanskritAbhyas.in. I think, it would also be useful if we can get the data from them and compare with that also. The main challenge would be matching the dhatus.

Anyway, I will go through the log (slowly) and report if anything needs to be corrected in Vidyut.

Thank you @akprasad for ccing me.

[akprasad]

After syncing with @neeleshb over email, I've learned that the forms on ashtadhyayi.com come from multiple different sources, including SanskritVerb. Rather than compare against ashtadhyayi.com directly, I think it will be more productive to continue comparing against forms from the Siddhanta Kaumudi and other traditional literature.