search

spdx / tools-golang

Collection of Go packages to work with SPDX files
Other
116 stars 54 forks source link

SPDX tag-value parsing fails for CRLF terminated file #198

Open surendrapathak opened 1 year ago

surendrapathak commented 1 year ago

The attached file fails to parse with the following error message (please remove .txt before processing)

Error: error processing file

This is likely due to its CRLF line endings. After saving with a sane line ending, parsing succeeds.

It parses successfully with online validator and spdx/tools-python

cbor-sbom.spdx.txt

neilnaveen commented 1 year ago

I was trying to run this, and I encountered this error

go run example_load.go cbor-sbom.spdx.txt

Error while parsing cbor-sbom.spdx.txt: unsupported SPDX version: 'Please refer to the AUTHORS file for contributor/creation details'%

cc @kzantow

kzantow commented 1 year ago

A quick look makes me think bufio.NewScanner (which is used here) is not handling \r, according to a Stack Overflow question

It's possible the input could be sanitized here or the location in the first link could add a split function as the SO answer example.

neilnaveen commented 1 year ago

A quick look makes me think bufio.NewScanner (which is used here) is not handling \r, according to a Stack Overflow question

It's possible the input could be sanitized here or the location in the first link could add a split function as the SO answer example.

Thanks for the suggestion, I tried to make the change but when I tried to test it, I ran into this error 

go run example_load.go cbor-sbom.spdx.txt

Error while parsing cbor-sbom.spdx.txt: unsupported SPDX version: 'Please refer to the AUTHORS file for contributor/creation details'%

which I had previously ran into, this error is not related to the CRLF. How do I test this change?

cc: @kzantow

kzantow commented 1 year ago

@neilnaveen one way is just do delete a bunch of lines out of that file, just making sure to save it with some pertinent \r\n line ending somewhere. Alternately, you could find a working SPDX TV file and then convert it to DOS line endings

neilnaveen commented 1 year ago

how and where do I load a valid spdx TV file so that I can test this?

kzantow commented 1 year ago

@neilnaveen you can look in the tests, for one: https://github.com/spdx/tools-golang/tree/main/examples/sample-docs/tv

surendrapathak commented 1 year ago

Also, to compare the problematic/good sbom, you can try :

  1. Problem ones: https://github.com/shamblett/cbor/releases/tag/5.1.2 (part of zip)
  2. Good ones for centos:latest built with syft 0.73.0 : https://sbomlc.s3.amazonaws.com/syft-0.73.0_centos-latest.spdx.tv?AWSAccessKeyId=AKIA2ZBFUJ4NNQGYD5OF&Signature=kvkkSvDCo%2FXt6XiO4nwlDnNCyl0%3D&Expires=1709195015