search

spdx / tools

SPDX Tools
Apache License 2.0
123 stars 68 forks source link

rdf to spreadsheet conversion taking longer time #253

Open vcmgc27 opened 3 years ago

vcmgc27 commented 3 years ago

Hello,

Trying to convert an *.rdf file to spreadsheet format using spdx-tools-2.2.2-jar-with-dependencies.jar

But it is displaying below error:

Command used is this:

java -jar spdx-tools-2.2.2-jar-with-dependencies.jar RdfToSpreadsheet /opt/spdx/output/file1.rdf /opt/spdx/output/file1.xls

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging. 11:43:47.338 [main] ERROR org.spdx.rdfparser.license.ListedLicenses - I/O error opening Json TOC URL, using local TOC file Connection timed out (Connection timed out) Connection timed out (Connection timed out)

But no output generated. Takes too long time.

Please help in this.

Thanks

goneall commented 3 years ago

The SPDX tools tries to fetch current listed license information from the URL https://spdx.org/licenses/licenses.json

The first error on logging is a known problem and can be ignored.

The second error indicates it is not able to fetch the license information from https://spdx.org/licenses/licenses.json This could be a proxy configuration or some other network issue.

There is a local cache of the licenses it falls back on but that doesn't seem to be working properly.

You can set a system property to force the use of the local cache.

@vcmgc27 try running the following and see if it helps:

java -DSPDXParser.OnlyUseLocalLicenses=true -jar spdx-tools-2.2.2-jar-with-dependencies.jar RdfToSpreadsheet /opt/spdx/output/file1.rdf /opt/spdx/output/file1.xls

BTW - the code for the local cache was recently updated in version 2.2.2, so there may have been a defect introduced. I'll see if I can duplicate the problem.

goneall commented 3 years ago

@vcmgc27 Were you able to get it to work with the UseLocalLicenses=true?

vcmgc27 commented 3 years ago

Sorry for the delay in reply. Will check and revert back on this.

vcmgc27 commented 3 years ago

Hi, Tried the above command. Received below output: ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging. Warning: The following verifications failed for the resultant SPDX RDF file: LGPL-2.1+ is deprecated. in Linux Kernel LGPL content GPL-2.0 is deprecated. in Linux Kernel GPL-2.0 is deprecated. in Linux Kernel GPL-2.0+ is deprecated. in Linux Kernel GFDL-1.1 is deprecated. in GNU Free Documentation License 1.1 Content

goneall commented 3 years ago

@vcmgc27 This would be a successful run.

The log4j error is a know problem issue #26

The warnings are due to a change in the SPDX license list where those license ID's have been replaced by LGPL-2.1-or-later, GPL-2.0-only, GPL-2.1-or-later and GFDL-1.1-only respectively.

You should have a successfully created excel file.

You can use this as a work-around until we identify/fix the original problem.

I'll try running the tools on my local system without internet access to see if it uses the cache automatically as designed.

goneall commented 3 years ago

@vcmgc27 I just tested the code the internet disabled and the code worked as expected falling back to the license cache.

I did find a possibly related issue (see PR #257), but it doesn't explain the errors you are seeing.

I suspect you may have a unique network configuration with a proxy server causing the problem.

For reference, access to the http license list is managed in the ListedLicenses.java file - specifically the getLicenseModel method

vcmgc27 commented 3 years ago

Thanks for the update, will try and let you know. Also, can you let me know the default location where this license file get stored.

goneall commented 3 years ago

The license files are stored in a resources/stdlicenses in the JAR file source.

You can also specify a different location using the SPDXParser.LocalLicensesDir option e.g.:

java -DSPDXParser.OnlyUseLocalLicenses=true -DSPDXParser.LocalLicensesDir=/licenses/current -jar spdx-tools-2.2.2-jar-with-dependencies.jar

where /licenses/current is a directory containing the licenses - see the files in resources/stdlicenses for the format.

goneall commented 3 years ago

@vcmgc27 Any updated on this issue? I'm working on a new release today and just checking if we need to have any additional fixes.