unnecessary copying of orig/xxx.txt file

[drdhaval2785]

I tried installing the xampp arrangement in ubuntu based distro. I saw that the scripts do copy files to orig/xxx.txt and pywork/hwextra/xxx_hwextra.txt files.

Question -

1. Why should we copy these two files?
2. Can't we use ../../csl-orig/v00/csl-data/XXXScan/2020/orig/xxx.txt and ../../csl-orig/v00/csl-data/XXXScan/2020/orig/hwextra/xxx_hwextra.txt directly i.e. from csl-orig repository itself?

[drdhaval2785]

Hi @funderburkjim , any thoughts on this front?

[funderburkjim]

The reason is so that the constructed 'xxx' directory (with its orig, pywork, and web subdirectories) will be self-contained.

Why does it matter that xxx be self-contained? Here are some reasons in favor:

• Historically, this is the way I organized things. I'm used to reasoning about code in this context.
• It makes it easier to experiment. For instance, suppose we want to add some additional alternate headwords for xxx. We can do that within xxx/pywork/hwextra/xxx_hwextra.txt and regenerate the xml etc within xxx. No need to have separate versions of redo_xml.sh for the experiment. No need to make a branch in csl-orig.
• Similar cases regarding experimenting with xxx.txt could also be handled in complete isolation from other code.
• Also, consider the xxx/downloads directory which is created by 'sh generate_dict.sh xxx' in csl-pywork/v02. We have not thus far discussed the use cases of xxx/downloads. But if we assume that we want to have xxx/downloads, then it makes sense to have another copy of xxx.txt in xxx/orig.
• Incidentally, I don't think xxx_hwextra should be in csl-orig. If you look at csl-pywork, you'll see that xxx_hwextra in csl-orig is never used. Rather, xxx_hwextra.txt is one of the 'distinctfiles'.

[funderburkjim]

Two suggestions for reorg of csl-orig

It is definitely nice to have all the digitizations in one place, and to have them tracked by Git. This simplifies the introduction of many kinds of small changes, in comparison to the updateByLine approach which made some small changes cumbersome.

But in working with csl-orig, I find the location of xxx.txt awkward.

The current location of xxx.txt is csl-orig/v00/csl-data/XXXScan/2020/orig/xxx.txt

I would prefer to get rid of most of the nesting. Maybe csl-orig/xxx/xxx.txt

This could be done by a program. If such a change were made, then there would need to be only one other change , in csl-pywork/v02/generate_orig.sh.

Given my comments regarding xxx_hwextra.txt, I would also at this time just remove the copies of these that occur in csl-orig.

If you concur, I can do the change or you can do it if you prefer. Let's agree on the changes to be made before either of us does anything.

[drdhaval2785]

I agree with the reorganaization suggestion regarding xxx/xxx.txt. You please do it. I will not be able to spare time in near future.

I disagree, and disagree strongly regarding hwextra suggestion. Its location should be data and not code. In other words, it should logically belong to csl-orig.

[funderburkjim]

Re: hwextra

I'll investigate how to make csl-orig/xxx/xxx_hwextra.txt the source of this data.

[gasyoun]

Rather, xxx_hwextra.txt is one of the 'distinctfiles'.

Agree.

I would prefer to get rid of most of the nesting.

Makes sense.

[funderburkjim]

The reorganization of csl-orig is now completed. New file names are:

• csl-orig/v02/xxx/xxx.txt and csl-orig/v02/xxx/xxx_hwextra.txt. Some more details of the reorg are in csl-orig/reorg.

The dictionary generation programs here in csl-pywork have been altered to be consistent with the csl-orig reorganization.

Further, @drdhaval2785 's request that the hwextra data originate in the csl-orig data has also been implemented in csl-pywork. To avoid confusion, the copies of the hwextra data in csl-pywork has been deleted from this repository (e.g., csl-pywork/v02/distinctfiles/xxx/pywork/hwextra directories have been removed.)

Local installations should be re-synced.

[gasyoun]

To avoid confusion, the copies of the hwextra data in csl-pywork has been deleted from this repository

Sure.

So it's generally done? Would love to see a plan draft. I'm lost as where Cologne is heading next.

[drdhaval2785]

As hwextra had been made part of csl-orig, this issue is closable now. Closing.