search

odpi / egeria

Egeria core
https://egeria-project.org
Apache License 2.0
806 stars 260 forks source link

various modules - unit tests output DEBUG during build #2058

Closed planetf1 closed 4 years ago

planetf1 commented 4 years ago

Our test code should generally not be outputting DEBUG messages into the build log.

Currently we see:

14:15:32,136 [INFO] --- maven-surefire-plugin:3.0.0-M3:test (default-test) @ csv-file-connector --- 14:15:32,141 [INFO] 14:15:32,141 [INFO] ------------------------------------------------------- 14:15:32,142 [INFO] T E S T S 14:15:32,142 [INFO] ------------------------------------------------------- 14:15:32,504 [INFO] Running org.odpi.openmetadata.adapters.connectors.csvfile.CSVFileStoreConnectorTest 14:15:32,506 [INFO] Running org.odpi.openmetadata.adapters.connectors.csvfile.CSVFileStoreProviderTest 14:15:32.821 [main] DEBUG org.odpi.openmetadata.frameworks.connectors.ConnectorProviderBase - Connector class name set: org.odpi.openmetadata.adapte rs.connectors.csvfile.CSVFileStoreConnector 14:15:32.834 [main] DEBUG org.odpi.openmetadata.frameworks.connectors.ConnectorBase - New Connector Requested 14:15:32.843 [main] DEBUG org.odpi.openmetadata.frameworks.connectors.ConnectorBase - New Connector initialized: 1ec5d1b6-bd11-4273-b62a-5164b339d9c c, CSVFileStore.Connection.Test,CSVFileStore.Connection.Test 14:15:32.897 [main] DEBUG org.odpi.openmetadata.adapters.connectors.csvfile.ffdc.CSVFileConnectorErrorCode - <== CSVFileConnectorErrorCode.getMessag e([target/test-classes/ComplexColumnsWithColumnNames.csv, 10001]) 14:15:32.899 [main] DEBUG org.odpi.openmetadata.adapters.connectors.csvfile.ffdc.CSVFileConnectorErrorCode - ==> CSVFileConnectorErrorCode.getMessag e([target/test-classes/ComplexColumnsWithColumnNames.csv, 10001]): File target/test-classes/ComplexColumnsWithColumnNames.csv does not have 10001 ro ws 14:15:32.899 [main] DEBUG org.odpi.openmetadata.frameworks.connectors.ffdc.ConnectorCheckedException - 400, org.odpi.openmetadata.adapters.connector s.csvfile.CSVFileStoreConnector, readRecord 14:15:32.899 [main] DEBUG org.odpi.openmetadata.adapters.connectors.basicfile.BasicFileStoreConnector - Closing File 14:15:32.899 [main] DEBUG org.odpi.openmetadata.adapters.connectors.csvfile.CSVFileStoreConnector - Closing Structured File Store 14:15:32.902 [main] DEBUG org.odpi.openmetadata.frameworks.connectors.ConnectorBase - New Connector Requested 14:15:32.903 [main] DEBUG org.odpi.openmetadata.frameworks.connectors.ConnectorBase - New Connector initialized: 0ec3dfcb-2082-4aea-85e4-04291ad57f4 b, CSVFileStore.Connection.Test,CSVFileStore.Connection.Test 14:15:32.904 [main] DEBUG org.odpi.openmetadata.adapters.connectors.basicfile.ffdc.BasicFileConnectorErrorCode - <== BasicFileConnectorErrorCode.get Message([target/test-classes/, CSVFileStore.Connection.Test]) 14:15:32.904 [main] DEBUG org.odpi.openmetadata.adapters.connectors.basicfile.ffdc.BasicFileConnectorErrorCode - ==> BasicFileConnectorErrorCode.get Message([target/test-classes/, CSVFileStore.Connection.Test]): The file target/test-classes/ given in Connection object CSVFileStore.Connection.Test is a directory 14:15:32.904 [main] DEBUG org.odpi.openmetadata.frameworks.connectors.ffdc.ConnectorCheckedException - 400, org.odpi.openmetadata.adapters.connector s.csvfile.CSVFileStoreConnector, getFileName ... etc

It would probably be a good idea to redirect the debug output elsewhere/turn off debug

planetf1 commented 4 years ago

This is also seen with other modules

1772 lines in total

Need to look at surefire logging configuration.

planetf1 commented 4 years ago

Taking the first debug found:

16:13:16,618 [INFO] Running org.odpi.openmetadata.repositoryservices.connectors.stores.metadatacollectionstore.properties.typedefs.AttributeTypeDefC
ategoryTest
16:13:16.626 [main] DEBUG org.odpi.openmetadata.frameworks.connectors.ConnectorBase - New Connector Requested

this is in org/odpi/openmetadata/frameworks/connectors/ConnectorBase.java:69 the log operation is using slf4j

This should be executed in test scope.

Our top level pom has an explicit, universal dependency for

        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <scope>test</scope>
        </dependency>

So in test scopes we should be introducing logback-classic, which by default will log at debug.

For applications (like the CTS) we can introduce a logback.xml to control the logging configuration - destination, level etc. It's also possible to add a logback-test.xml for test scope - however we don't have one module, we have hundreds, and what will always be in the class path...

Instead an alternative is to use slf4f-simple which a) by default logs at INFO b) can be configured through system properties, which are easy to inject via the surefire plugin configuration ie: Add the dependency

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>${slf4j.version}</version>
</dependency>

Instead of logback .. and then 

<plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.18.1</version>
            <configuration>
                <systemPropertyVariables>
                    <org.slf4j.simpleLogger.defaultLogLevel>DEBUG</org.slf4j.simpleLogger.defaultLogLevel>
                    <org.slf4j.simpleLogger.showDateTime>true</org.slf4j.simpleLogger.showDateTime>
                </systemPropertyVariables>
            </configuration>
        </plugin>