search

AquaticInformatics / aquarius-field-data-framework

A plugin framework for creating field data parsers for AQUARIUS Time-Series
Apache License 2.0
4 stars 15 forks source link

FieldVisitHotFolderService - Not all log messages are being captured #80

Closed DougSchmidt-AI closed 5 years ago

DougSchmidt-AI commented 5 years ago

During a recent demo, one log message from the Survey123 plugin was not making it to the file-specific log file. It only showing in the main FieldVisitHotFolderService.log.

From the FieldVisitHotFolderService.log

2019-05-15 14:38:06,260 [5] INFO  - Moving '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip' to '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Processing\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip'
2019-05-15 14:38:06,268 [5] INFO  - Loading data file '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Processing\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip'
2019-05-15 14:38:06,275 [5] INFO  - Partial headers detected: IDWR field data survey configuration is invalid: 1 missing column: 'Discharge Measured in CFS'
2019-05-15 14:38:06,297 [5] ERROR - '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Processing\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip' was not parsed by any plugin.
2019-05-15 14:38:06,299 [5] INFO  - Moving '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Processing\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip' to '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Failed\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2) (1).zip'

The Partial headers detected: IDWR field data survey configuration is invalid: 1 missing column: 'Discharge Measured in CFS' message was missing from the equivalent file-specific log in the Failed folder.

That's where customers will look to figure out what is going wrong, so that message needs to end up there.

The file-specific log is here:

2019-05-15 14:38:06.260-07:00 INFO  - Moving '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip' to '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Processing\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip'
2019-05-15 14:38:06.268-07:00 INFO  - Loading data file '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Processing\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip'
2019-05-15 14:38:06.297-07:00 ERROR - '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Processing\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip' was not parsed by any plugin.
2019-05-15 14:38:06.299-07:00 INFO  - Moving '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Processing\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2).zip' to '\\nasvan\Data\Data\PublicShares\DougTest\Rooty\Failed\S123_6e3a6b943e7b437898ef8db04bf11dc9_CSV (2) (1).zip'
DougSchmidt-AI commented 5 years ago

Fixed by #83 in v18.4.15