Copybooks not found

[albertolovato]

I tried to parse a directory of Cobol files, using the method new CobolParserRunnerImpl().analyzeFiles(inputFiles, copyFiles, format, null) If I understood correctly, inputFiles are Cobol sources, and copyFiles are copybooks.

I got the following error Copy file "copybooks/abc.cpy" not found in copy files [src/test/resources/copybooks/abc.cpy]. The same happens if I put "abc.cpy" in the same directory of the Cobol source. Am I missing something?

Thank you

[uwol]

Your assumptions are correct. InputFiles are Cobol sources, and copyFiles are copybooks. An illustrating test case for COPY statements is given in class CopyDisplayTest with the call of method analyzeFiles in CobolParseTestRunnerImpl.

Probably the problem is, that in your COBOL source file you have a statement COPY copybooks/abc.cpy instead of COPY abc. Therefore, the copy book lookup does not work, leading to the mentioned error message.

The failing lookup of COPY copybooks/abc.cpy is an error, which I will fix asap in the next days.

Thanks for issuing! Currently, we are building a transformation platform on top of the parser, and thankful for any reported issue.

[albertolovato]

Thank you for the prompt response.

By the way, we have several issues to report. If you prefer a single e-mail over multiple issue reports, please let me know.

Thank you very much

[uwol]

Hi, I would prefer multiple separate issues in GitHub to keep things organized and transparent. Please feel free to report. Thanks!

[Reinhard-Prehofer]

I am still facing above mentioned copy-dir-errors ?

Copy file BSATZ not found in copy files [src\test\resources\cpy].
Copy file 'ADACBOP.CPY' not found in copy files [src\test\resources\cpy].
Copy file 'ADACBCL' not found in copy files [src\test\resources\cpy].

Are you sure we are using a snapshot version where the errors has been fixed? I have been playing around with quotes and apostrophes and fully qualified copybooknames - no changes to the error message at all. IBM_host-copy books tend to implicity use the .CPY extension and the copy book statement in Cobol uses the cobybook-name without the .cpy extension. I see that issue being 18 days old and the last change on the git-sources being 27 days old ??? Thus to my interpretation the git-version we are using cannot include the copy-book-correction

the code snippet would be (also had been playing around with forward/backward slashes etc ...):

    static String SRC_DIR = "src/test/resources/src";
    static String CPY_DIR = "D:\\ws\\CobolLogger\\src\\test\\resources\\cpy";

    public static void main(String[] args)  {

        // generate ASG from plain COBOL code
        String FileName = "CobolCopy1.CBL";
        String cobolFileName = SRC_DIR + "/" + FileName;
        File inputFile = new java.io.File(cobolFileName);
        CobolPreprocessor.CobolSourceFormatEnum format = CobolPreprocessor.CobolSourceFormatEnum.FIXED;
        List<File> copyDirs = new ArrayList<File>();
        copyDirs.add(new File(CPY_DIR));
        try {
            Program program = new CobolParserRunnerImpl().analyzeFile(inputFile, copyDirs, format, CobolDialect.OSVS); 

[Reinhard-Prehofer]

I had a short look at your recent sample about COPY-Books. Now I start to understand and want to point out the difference to common implementations in Cobol

+       final File copyFile1 = new File(
+               "src/test/resources/io/proleap/cobol/preprocessor/copy/variable/CopyReplace1.cpy");
+       final File copyFile2 = new File(
+               "src/test/resources/io/proleap/cobol/preprocessor/copy/variable/CopyReplace2.cpy");
+       final ArrayList<File> copyFiles = Lists.newArrayList(copyFile1, copyFile2);

So the "copyFiles" in your implementation really is a LIST of individual Files !? and NOT DIRECTORIES ... In Cobol usually a predefined Environment variable points to a list of DIRECTORIES where all the individual copy-files are located and there is also some more logic behind how to handle extensions that have not been explicitly named. (like exact name first, then try with .cpy, then .cbl or .cob, and for other extensions you have to be precise and say copy 'mycopybook.abc') Typical environment variables are:

• COBCPY - MicroFocus
• COB_COPY_DIR - GnuCobol

The difference is in "real" projects with several hundred different copy-books, in one project we even have 2.500 copybooks in approx 5 different directories. its rather easy to defines these say 5 directories with export COBCPY=/copydirA:/copydirB;... but it almost needs a preprocessor to define the 2.500 individual copybookfiles and insert them into a huge Listfile or create individual listfiles per cobol program.

And ... not to forget NESTED COPY-Books !?

Honestly speaking, I would reconsider the design with the COPYFILES to rather closely follow the COPYDIRS approach of defining directories only where the COPYbooks are searched for. regards - Reinhard

[uwol]

I committed a fix (https://github.com/uwol/cobol85parser/commit/e8e315b5e5dfd8d7973d8bb4ed846fc27ed782fb), which checks, whether a COPY statement references a copy book by a COBOL word (e. g. unit test) or a literal path (e.g. unit test).

The assumption is, that path references to copy books in COPY statements are expressed always as literals, i.e. Strings.

[Reinhard-Prehofer]

I added a few comment in the issue on github.I will check later with some samples.The problem I See is the different concept zu cobol - not using a list If directories WHERE the COPY-books are inside but listing Individual files ...I expected resolvement Problems with Nester copybooks and when using different extension -  but it will be easier to provide some samples on that issue.We also do have customers with different copybook content for development and testing and Produktion (mostly because of logging and tracing ).Till soon, keeping you informed,Best regards - r Von meinem BlackBerry gesendet – dem sichersten Mobilgerät Von: notifications@github.comGesendet: 26. Juli 2017 2:11 nachm.An: cobol85parser@noreply.github.comAntworten: reply@reply.github.comCc: reinhard.prehofer@splendit.at; comment@noreply.github.comBetreff: Re: [uwol/cobol85parser] Copybooks not found (#8) I committed a fix (e8e315b), which checks, whether a COPY statement references a copy book by a COBOL word (e. g. https://github.com/uwol/cobol85parser/blob/e8e315b5e5dfd8d7973d8bb4ed846fc27ed782fb/src/test/resources/io/proleap/cobol/preprocessor/copy/copyreplace/variable/CopyReplace.cbl#L5) or a literal path (https://github.com/uwol/cobol85parser/blob/e8e315b5e5dfd8d7973d8bb4ed846fc27ed782fb/src/test/resources/io/proleap/cobol/preprocessor/copy/path/variable/CopySubDir.cbl#L5). The assumption is, that path references to copy books in COPY statements are expressed always as literals, i.e. Strings.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.

[uwol]

@Reinhard-Prehofer I am taking a look regarding the example given by you, could be that 'ADACBCL' breaks my fix, I'll check that.

Correct, the copyfiles are a list of individual copy files. Currently, the caller of the parser has to call List<File> files = Lists.newArrayList(directory.listFiles()) before, to read all copy books from a given directory. But it wouldn't make a problem for me, to add a method, which accepts directories.

Thanks for the hint regarding dialects. I think, I will take that into account in my follow-up fix.

[Reinhard-Prehofer]

@uwol A short feedback. If the name of a copybook is quoted, this name is literally searched for. thus copy 'ADACBCL' expects a file ADACBCL to be available, while copy ADACBCL expects a file ADACBCL.CPY to be available.

As I pointed out earlier, there are some rules concerning expected extensions and also defined order the Cobol Compiler is looking for copybooks with that extension. (I try to look up the relevant links from ibm and microsoft in particular)

[uwol]

Ok, thanks for all your input. Version 2.3.0-SNAPSHOT now accepts both files as well as directories in the copy book list. This issue will be part of 2.3.0.

To cope with all the described scenarios, I'm going to add tests and some mechanism for the extension lookup order. Additionally, I think it would be helpful to simply read the COBCPY ENV var for building up the list automatically.

[uwol]

Ok, as of 59b2c015ba8fccfb4909c0bb89163ba9a7c320b2 the preprocessor and parser accepts an optional parameter object, which contains copy book extensions, copy book files and copy book directories. The original copy books parameter has been removed. The commit is deployed in MAVEN repo as 2.3.0-SNAPSHOT.

So to wrap things up:

• The copy books parameter has been removed.
• If no parameter object is provided, the parser searches for copy books in the directory of the COBOL file (case-insensitive) -> gentle fall back for simple cases.
• If an parameter object is provided, the parser searches for the copy book first in the list of copy book files, and then in the list of copy book directories (case-insensitive).
• In case of a cobol-word-based COPY statement (e. g. COPY SomeCopyBook) and if the extensions list in the parameter object is set: The files will be filtered for those extensions.
• In case of a literal-based COPY statement (e. g. COPY 'SomeCopyBook.cbl'): The parser searches for a copy book with that exact name (case-insensitive).
• In case of a literal-based COPY statement with a relative path (e. g. COPY 'copybooks/SomeCopyBook.cbl'): The parser searches for a copy book with that path (case-insensitive).

I'll keep testing this in the next days before build a version 2.3.0. Open for comments!