Making systematic changes using TextPipe

tfbf / Bible-Punjabi-Pavitr-Bible-1945

Bible-Punjabi-Pavitr-Bible-1945

Other

5 stars 9 forks source link

Making systematic changes using TextPipe #95

Closed DavidHaslam closed 7 years ago

DavidHaslam commented 7 years ago

I'm creating this issue as a placeholder to summarise what my bespoke TextPipe filters currently address:

Double quotation marks - see issue #15 (only 3 remain to be fixed)
Multiple whitespace - see issue #20 (solved)
Danda and double danda - see issue #24 (no more vertical lines)
Paragraph markers - see issue #26 (each on its own line)
Paragraphs and double danda - see issue #54 (except for 125 mid-verse ॥ )
Punctuation and spacing - see issue #64 (still some issues to discuss)
Remove BOM from Proverbs - see issue #77 (fixed)
Exclamation mark in Prov 20:14 - see issue #91 (fixed)
Matters relating to "Selah" - see issue #92 (not yet addressed, but was affected by issue #54 )

The above list is in ascending order of issues, which is not the same as the order in my filters.

Not addressed by my TextPipe filter is:

Unicode Normalization - see issue #44 (this can be done when we make the SWORD module)

DavidHaslam commented 7 years ago

TextPipe is a very powerful method to fix things systematically. I've been a TextPipe user since 2001.

The advantage of using TextPipe is that even if I have to download a fresh set of USFM files after someone else updates them in the meantime, I can repeat the operations methodically.

DavidHaslam commented 7 years ago

Activity update:

I have deleted the forked repo that I made in December, because it was too many commits behind.
I forked the tfbf repostory again so I could start working afresh on punjabi_bible_1945
I created a branch from master called process
I copied the 66 usfm files (output from my TextPipe filter) to the usfm folder in process
I committed this local change using the GitHub Desktop client.
I sync'd the process branch using GitHub Desktop client.
I did a pull request from the process branch.

This branch has no conflicts with the base branch. It should be possible to merge all these changes.

DavidHaslam commented 7 years ago

My pull request #96 has since been merged. Thanks, Joshy.

DavidHaslam commented 7 years ago

Several of the above listed issues have been closed that were either fixed or substantially resolved by merging the pull request.

New issue #97 has been opened to address a smaller residual problem.

DavidHaslam commented 7 years ago

I have just tweaked my TextPipe filter that counts words in the Punjabi Bible so that it treats hyphenated words correctly.

merged.words.count.txt

And here is a list of the hyphenated words found therein.

merged.hyphens.words.count.rdl.txt

DavidHaslam commented 7 years ago

As this issue was largely descriptive, I think it may be closed now.

Things that it mentions are covered individually in other issues.

DavidHaslam commented 7 years ago

FIO. A link to my EMDC2012 workshop presentation on using TextPipe.

NB. The material is a bit dated now, especially the Go Bible context of that time.