KeyError when parsing server response upon validation error

[jacob-lee]

I received the following KeyError, after attempting to submit.

Traceback (most recent call last):
  File "/home/jacoblee/miniconda3/envs/ndar/bin/vtcmd", line 8, in <module>
    sys.exit(main())
  File "/home/jacoblee/miniconda3/envs/ndar/lib/python3.8/site-packages/NDATools/clientscripts/vtcmd.py", line 284, in main
    validation_results = validate_files(args.files, w, bp, threads=args.workerThreads, config=config)
  File "/home/jacoblee/miniconda3/envs/ndar/lib/python3.8/site-packages/NDATools/clientscripts/vtcmd.py", line 181, in validate_files
    validation.output()
  File "/home/jacoblee/miniconda3/envs/ndar/lib/python3.8/site-packages/NDATools/Validation.py", line 136, in output
    column = v['columnName']
KeyError: 'columnName'

The key, columnName is hard-coded in the following bit of code:

               file_name = self.uuid_dict[response['id']]['file']
                if response['errors'] == {}:
                    writer.writerow(
                        {'FILE': file_name, 'ID': response['id'], 'STATUS': response['status'],
                         'EXPIRATION_DATE': response['expiration_date'], 'ERRORS': 'None', 'COLUMN': 'None',
                         'MESSAGE':'None','RECORD': 'None'})
                else:
                    for error, value in response['errors'].items():
                        for v in value:
                            column = v['columnName']
                            message = v['message']
                            try:
                                record = v['recordNumber']
                            except KeyError:
                                record = ' '
                            writer.writerow(
                                {'FILE': file_name, 'ID': response['id'], 'STATUS': response['status'],
                                 'EXPIRATION_DATE': response['expiration_date'], 'ERRORS': error, 'COLUMN': column,
                                 'MESSAGE': message, 'RECORD': record})
            csvfile.close()

It looks like response['errors'] dict did not have the expected keys.

I modified the code to print out the error and error messages:

for error, value in response['errors'].items():
    print(f'{error}: {value}')

and it printed:

 data_structure: [{'message': 'File not recognized as a data file, and was not associated or referenced from a data file.'}]

The cause of the error is probably something on my own end, but it does look like the code is not correctly anticipating the structure of the response.