Add data

[rhnvrm]

The bot has run out of data to tweet. If any one wants to volunteer and contribute to adding data feel free to ping either me or @kumbhakaran on this thread and we can help you get started and things to take care of while adding data.

[rhnvrm]

Steps to add a file:

• Copy data from CAD archives into a new file
• Run the following regex replace for fixing the spacing for line headers: %s/^3\./\n3./g (3 will need to be changed to appropriate headline)
• Run the following regex to trim spaces from lines at the beginning: %s/^ *//g
• Run the following regex to remove double spaces: %s/\s\s\+/ /g (8e339a3)
• Manually review any issues that are visible. (document them here)
• Append the data in this file to data.txt (in case the chapter is ending, truncate the file and move it to data-2.txt and so on and add the work done above to data.txt)
• In case there are duplicate rows while copying, we can run awk -i inplace '!seen[$0]++' data.txt

I used the above steps to add the following commits:

57924f480e2aa8b5549e6965e287175bc9592373, cd3a347fb596a52d074c62d2f6e445ddc2b0ee06, e2aeba5237a23b477f75f311e24ad53f262207a7, c4145b5b013b1a513eaa24467d4ae8921350dbcf, 5df311f7e206efc66aeec9ee25af22689e63b4cb, b268bed19c6ad3fe4cb97e6a7bffbe8e3372ff48, 693705cd5d1bcf721c3f4028571c42a487cadcf9

[rhnvrm]

curl https://www.constitutionofindia.net/constitution_assembly_debates/volume/7/1948-11-26 | htmlq '.ckeditor-content .row' -p  | htmlq -r .tooltiptext -r .summary-block -r .support-text -r .social-block -r .abt-events -t | sed -e 's/^ *//g' |sed -e 's/\s\s\+/ /g' | sed '/^$/d' | awk -i inplace '!seen[$0]++' | sed 's/^7\./\n7./g' >> data.txt

[rhnvrm]

Added Volume 8 - 3944ede7f1009dbade98a046626dd441a57df541

This commit also automates generating the file with a bash script.