t-vim `.vimout` file name clashing

[rijenkii]

Two following use cases do not work correctly:

\typeTEXTfile{folder1/file.txt}
\typeTEXTfile{folder2/file.txt}

\typeTEXTfile[start=1, stop=10]{file.txt}
\typeTEXTfile[start=11, stop=20]{file.txt}

In both cases only a single .vimout file is created.

Similar issue: #57.

[adityam]

I realized that this will clash when writing the patch for #57, but I did not fix it because I don't know what is a good automated solution. Currently, the name of the file is \jobname-\basename-\ext.vimout where \basename is the name of the file and \ext is the extension of the file.

If two files with the same name are in different directories, then the name of the .vimout file will clash. There are two ways to fix this:

• Instead of just using the \basename, use the full path: so, the name of the .vimout file will be path-to-PWD-folder1-file-txt.vimout. ConTeXt does follow this approach while "sanitizing" automatically downloaded files in cache, this can lead to files with very long names.
• Use an md5 or sha has of the full qualified filename. This will also lead to long filenames.

I don't like either solution, so I offer a work-around for now: Ask the user to specify the name of the vimout file: for example:

\typeTEXTfile[output=\jobname-folder1-file.vimout]{folder1/file.txt}
\typeTEXTfile[output=\jobname-folder2-file.vimout]{folder2/file.txt}

I know that this is not ideal, but hopefully this will suffice.

[rijenkii]

This will actually suffice, thank you!

I do like the idea of having hash instead of filename though, maybe add it behind some flag? Though, I don't think that ConTeXt has any built-in hash functions..

[adityam]

There is md5 ... (SHA might also be there but it is an overkill for something like this):

\startluacode
  local filename = "folder1/file.txt"
  local hash = md5.HEX(filename)
  print(">>>>", filename, hash)
\stopluacode

So, we can use something like this:

\usemodule[vim]

\definevimtyping[TEX]
                [
                  syntax=context,
                  output={\jobname-\cldcontext{md5.HEX("\externalfilterinputfile")}.vimout},
                ]

\traceexternalfilters

\starttext
\typeTEXfile[start=2,stop=4]{a/test.tex}
\typeTEXfile[start=5,stop=7]{b/test.tex}

Given the two recent bugs (#57 and this one) on handling of external files, I can even make this default for processing external files.

[rijenkii]

If there is fear about the end user not understanding which output files are for which input file and long filenames, you also make filenames look like file-txt-ah3nv8an.vimout, that is <filename>-<ext>-<8 chars of md5>.vimout. But, that's for you to decide.

[adityam]

Can you test t-filter.mkxl from the hash branch. It passes all my tests.

[rijenkii]

This indeed fixes the problem with folders, but in the OP I have also mentioned a start/end clash: same path, different parts of the file.

[adityam]

There is already a cacheoption key. I can do a hash of #3\externalfilterparameters{cacheoption}. Then, to get different start-stop pairs to work, we will need to specify

\defineexternalfilter[...][cacheoption={\externalfilterparameter{start}\externalfilterparameter{stop}}

These could be the default options in t-vim. Let me test this idea

[adityam]

See the latest commit on the hash branch. That resolves the second bug as well

[rijenkii]

Yeah, that does seem to work correctly with my use cases.