Switch data source?

[vvasuki]

Evaluate if ingesting data from https://github.com/ashtadhyayi-com/data/blob/master/sutraani/1/1/1.1.11.txt is superior to using https://github.com/sanskrit/ashtadhyayi - seems that they have 1 or 2 more active contributors than the latter.

[vvasuki]

Beware though, many of the changes seem negative - https://github.com/ashtadhyayi-com/data/commit/313573ea2a44358cf791b14062d3c733ec56174d#r40924799

[chandrasekharanr]

@vvasuki , just want to understand this clearly. I have been using https://ashtadhyayi.github.io/ interface to make changes, and wanted to keep it the master source. Are you saying we switch the master to Neelesh mahodaya's ashtadhyayi-com repo?

I am all for maintaining a single master. Before that,

(0) Can you please add Neelesh as a participant in this thread? I am not able to do it.. We can continue the discussion here. (1) Let us merge the changes from the repo https://github.com/sanskrit/ashtadhyayi to Neelesh's repo, so that they don't redo the same changes that have been incorporated.

[vvasuki]

@vvasuki , just want to understand this clearly. I have been using https://ashtadhyayi.github.io/ interface to make changes, and wanted to keep it the master source. Are you saying we switch the master to Neelesh mahodaya's ashtadhyayi-com repo?

We can change https://ashtadhyayi.github.io/ interface so that it gets some of the data from Neelesh mahodaya's ashtadhyayi-com repo, and so that the edit buttons take you there for such data. Question is if that will really bring much improvement. It's ok if you decide to shelf it for now and revisit this after x months when you have more data.

I am all for maintaining a single master. Before that,

(0) Can you please add Neelesh as a participant in this thread? I am not able to do it.. We can continue the discussion here.

@chandrasekharanr - tagged @neeleshb .

(1) Let us merge the changes from the repo https://github.com/sanskrit/ashtadhyayi to Neelesh's repo, so that they don't redo the same changes that have been incorporated.

if he thinks it's worth it, @neeleshb can generate a cl, but it may require much manual review.

[neeleshb]

1) Noted down recent changes from https://app.wercker.com/sanskrit-coders/ashtadhyayi/runs, I can manually copy them to ashtadhyayi.com repo sometimes this week.

2) I don't have any strong opinion on whether there should be 1 master copy. I'm happy to add both of you as contributors at ashtadhyayi.com if you decide to merge the two. However, I'd like to clarify that the data format of ashadhyayi.com repository is highly influenced by the structure of the website (hence in json) and hence it is not (or won't be) be as human readable as the ashtadhyayi.github.io repository.

3) The repository has picked up some traction recently, primarily because multiple groups are using the website for their online sessions. It may be worthwhile to wait and watch what happens, if we are not sure about merging.

4) The edits on ashtadhyayi.com will continue due to the above reason, and in case we decide to have 2 separate repos, I will try to do regular pulls from your side (manually to start with). This will work as long there are sporadic edits on either (or both) sides, but if we see significant traction on both sides, it may be better to merge.

[chandrasekharanr]

2. I don't have any strong opinion on whether there should be 1 master copy. I'm happy to add both of you as contributors at ashtadhyayi.com if you decide to merge the two. However, I'd like to clarify that the data format of ashadhyayi.com repository is highly influenced by the structure of the website (hence in json) and hence it is not (or won't be) be as human readable as the ashtadhyayi.github.io repository.

The reason we created this repo was to clean up the vrttis (there are a lot of errors) by some crowd sourcing, and there were essentially a couple of active websites that had it. avg-sanskrit is now pretty much calmed down. ashtadhyayi.com is the only active (undergoing changes) now. So, when I said master copy, it is just to preserve the effort of whatever has been done, and not to repeat the work.

3. The repository has picked up some traction recently, primarily because multiple groups are using the website for their online sessions. It may be worthwhile to wait and watch what happens, if we are not sure about merging.

Then, I would probably get onto the ashtadhyayi.com boat to make further edits. I am not very regular in making the edits, but I do whenever I find errors in the text I stumble upon while reading.

4. The edits on ashtadhyayi.com will continue due to the above reason, and in case we decide to have 2 separate repos, I will try to do regular pulls from your side (manually to start with). This will work as long there are sporadic edits on either (or both) sides, but if we see significant traction on both sides, it may be better to merge.

[vvasuki]

However, I'd like to clarify that the data format of ashadhyayi.com repository is highly influenced by the structure of the website (hence in json) and hence it is not (or won't be) be as human readable as the ashtadhyayi.github.io repository.

@neeleshb Why not switch to toml? Should be a half an hour one-time thing. Will be more readable (no "\r\n"s - see https://learnxinyminutes.com/docs/toml/ ) as well as retaining structure.

Noted down recent changes from https://app.wercker.com/sanskrit-coders/ashtadhyayi/runs, I can manually copy them to ashtadhyayi.com repo sometimes this week.

Once @neeleshb does the above, perhaps @chandrasekharanr can switch to contributing there (as far as sk, lsk, kaashikaa are concerned), and I can point the ashtadhyayi.github.io to that repo.

[neeleshb]

@vvasuki , not switching to anything else because the goal is not to create a human readable repository but rather have the data in such a way that it can be consumed by the current design of the website without any change. Earlier this data was in mysql, but I then moved it to github to make it publicly editable.

In fact the reason why I am not very sure if we want to treat ashtadhyayi.com repo as a "master repo" is that I don't want the backend format or the data structure to be driven by anything else other than what is needed for the website. Also, I can easily visualize this repo changing in the future to suit the additional needs of the website, like internal cross-references or new custom markups, which may not be easily readable. (The "pc" we discussed earlier is a sample example of this, though I am planning to move away from that for now).

Re: Including changes done on https://app.wercker.com/sanskrit-coders/ashtadhyayi/runs - Yes, will do it in this week.

[vvasuki]

In fact the reason why I am not very sure if we want to treat ashtadhyayi.com repo as a "master repo" is that I don't want the backend format or the data structure to be driven by anything else other than what is needed for the website.

@chandrasekharanr (Thanks, for clarifying @neeleshb) . Given the above, perhaps it is better to just retain the old repo (henceforth the "master" repo) and periodically programmatically import from the ashtadhyayi.com dump. You can make corrections in the ashtadhyayi.com repo since they seem to have more correction activity (and we don't want to waste our time). Can you also update our python scripts (https://github.com/ashtadhyayi/data_curation/tree/master/ashtadhyayi_data ) which you will periodically run on your computer update the master repo?

[neeleshb]

Sounds good to me. Will add both of you there.

[vvasuki]

@chandrasekharanr For convenience, I've added links to nIlesh's site and edit url (see below, should be live shortly); as well as vAsu's vRtti as per separate email thread. Closing this. Will track raise other issues separately.