[Continuous build] Log improvment : adding lint summary on build failure

[mbarre]

Hello Sualeh,

First of all, thanks for the "dispatch" feature you've add last week, continuous DB quality check is now running on our projects on github/travis, it's really cool.

Here's the proof : ;)

Detailed logs below :

We also use Slack (https://slack.com/) as instant messaging and its famous notifications. Each time, a build is performed on Travis the result is sent to Slack and displayed in the project channel.

As you can see, we customize the icon for the build based on Schemacraler ;)

Enhancement proposal

Now, we realize that some enhancements could be done regarding the logs and especially the build result. In fact when the build fails, the only readable message we have is Too many lints were found, but we didn't get details (tables, column, lint name...) about what causes the build to fail. Even a summary like the result of the command mvn test could be helpful.

Mar 08, 2016 5:46:11 AM io.github.mbarre.schemacrawler.tool.linter.LinterColumnSize lint
INFO: Checking public.pieceafournir.commentaire...
Mar 08, 2016 5:46:11 AM io.github.mbarre.schemacrawler.tool.linter.LinterColumnSize lint
INFO: Checking public.pieceafournir.libelle...
Mar 08, 2016 5:46:11 AM io.github.mbarre.schemacrawler.tool.linter.LinterColumnSize lint
INFO: Checking public.pieceafournir.justification...
Mar 08, 2016 5:46:11 AM io.github.mbarre.schemacrawler.tool.linter.LinterColumnSize lint
INFO: Checking public.rendez_vous.commentaire...
Mar 08, 2016 5:46:11 AM schemacrawler.tools.iosource.FileOutputResource openNewOutputWriter
INFO: Opened output writer to file, /home/travis/build/DSI-Ville-Noumea/pdc-liquibase/pdc_lints.html
Mar 08, 2016 5:46:11 AM schemacrawler.tools.iosource.OutputWriter close
INFO: Closing output writer
Mar 08, 2016 5:46:12 AM schemacrawler.tools.lint.executable.LintDispatcher lambda$dispatch$2
SEVERE: Too many schema lints were found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18.469 s
[INFO] Finished at: 2016-03-08T05:46:12+00:00
[INFO] Final Memory: 6M/84M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project pdc-liquibase: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

travis_time:end:2c0057b8:start=1457415950401687923,finish=1457415972324604875,duration=21922916952

The command "mvn exec:exec" failed and exited with 1 during .

Your build has been stopped.

So we think about something like that :

Results :

Failed lints: 
io.github.mbarre.schemacrawler.tool.linter.LinterColumnSize : table1.column1, table3.column3 ...
io.github.mbarre.schemacrawler.tool.linter.LinterTableWithNoPrimaryKey : table2

Lints run: 17, Failures: 2

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.800 s
[INFO] Finished at: 2016-03-09T11:24:43+11:00
[INFO] Final Memory: 25M/295M
[INFO] ------------------------------------------------------------------------

Let us know what you think about it ?

[adriens]

Nice ticket... we told you we we would do it ;-D Now having so many ideas around lint reporting ... i'm dreaming of something like coveralls.io for databases, based on schemacrawler lints ! That would be awesome.

[schemacrawler]

I love the way you guys are using SchemaCrawler! Basically, you want a lightweight text report to be printed along with the dispatch. I could do that, but I would have to print all of the failed lints, no matter how critical they were. This may be a long list. Will this be ok?

[adriens]

Eh eh eh, was pretty sure you would love it !... schemacrawler is a great tool with a huge potential and it's very exciting to show it ! Let us think about how synthetic le log outpout should be : if its' too verbose/big, Travis will flag the build as failed. Hence it has to be pretty synthetic, indeed for now we should define how it could be aggregated.

[adriens]

We've also though about uploading a maven site (the html lint output) to a gh-pages of the project, making it possible to read it afterwards... this is an another approach. Still, getting some infos on the build is very useful when on Trvais

[schemacrawler]

Output will be verbose, and will look something like the following. I can print the output before the failure, so Travis will not flag the build as failed.

[PUBLIC.FOR_LINT."Global Counts"] should have remarks
[PUBLIC.FOR_LINT."Global Counts"] should have remarks: "Global Count"
[PUBLIC.FOR_LINT.EXTRA_PK] useless surrogate key: PK_EXTRA_PK
[PUBLIC.FOR_LINT.EXTRA_PK] empty table

[adriens]

We're talking about it with @mbarre now, she'll update the ticket n the next minutes. For now, what is sure it's that whenever the build is successful or not, the sythesis like the one you propose is required, ... and a simple count could very well do the trick ! : it gives a idea on the database quality, it'a bit like a dashboard, which is great/ If people want more details, they have to read the real lint ouput.

[mbarre]

Here an example of what we could display :

Following schema lints were found : 

Severity CRITICAL
  No primary key : 1 / ${dispatch-threshold}

Severity HIGH
  Empty table : 2
  All column nullable : 3

Severity MEDIUM
  Useless surrogate key: 10

Severity LOW
  Column without remark : 3
  Custom lint : 3

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.105 s
[INFO] Finished at: 2016-03-09T03:50:45+00:00
[INFO] Final Memory: 9M/134M
[INFO] ------------------------------------------------------------------------

[mbarre]

May be we could add a new attribute (similar to description and summary) to the BaseLinter java class that will be displayed with the count in the build summary

[adriens]

@sualeh : you mean that you did it ?!!

[schemacrawler]

I think so - please look at the changes - but I want to clean up some more stuff before I release.

[mbarre]

:+1:

[adriens]

feeling excited !!!