search

hsiehsh168168 / warc-tools

Automatically exported from code.google.com/p/warc-tools
0 stars 0 forks source link

test warc files does not validate at all #97

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?
1. for I in $(ls wdata/testwfile/*.warc); do ./warcvalidator -f $I; done

What is the expected output? What do you see instead?
Some validated warc-files - some files not validated as valide warc files

This is the output I get:
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
warcvalidator: app/warcvalidator.c:124: main: Assertion `w' failed.
Aborted
warcvalidator: app/warcvalidator.c:124: main: Assertion `w' failed.
Aborted
warcvalidator: app/warcvalidator.c:124: main: Assertion `w' failed.
Aborted
warcvalidator: app/warcvalidator.c:124: main: Assertion `w' failed.
Aborted
warcvalidator: app/warcvalidator.c:124: main: Assertion `w' failed.
Aborted
warcvalidator: app/warcvalidator.c:124: main: Assertion `w' failed.
Aborted
warcvalidator: app/warcvalidator.c:124: main: Assertion `w' failed.
Aborted
warcvalidator: app/warcvalidator.c:124: main: Assertion `w' failed.
Aborted
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"
> debug: lib/private/wheader.c :2048:"Incorrect Warc Version"

Original issue reported on code.google.com by bjarne.a...@gmail.com on 2 Sep 2008 at 8:41

GoogleCodeExporter commented 8 years ago 
Bjarne, 
Your example isnt' correct at 100%. In "wdata/testwfile" dit, some WARC are 
valid, some other not and we 
deliberatly did that.
Anyway, we made warc-tools more compatible between versions 0.17 and 0.18. So, 
it'll accept both version 
because they are technically the same.

Thanks
Y.

Original comment by voidptr...@gmail.com on 14 Sep 2008 at 12:16

GoogleCodeExporter commented 8 years ago 
I just wondered why all files failed in my distribution. I was expecting some 
files
to fail and some files to validate - none of that happened.

Original comment by bjarne.a...@gmail.com on 14 Sep 2008 at 5:30

GoogleCodeExporter commented 8 years ago 
Hi Bjarne,
Please, try to understand the commands before using them.

You're doing that:
for I in $(ls wdata/testwfile/*.warc); do ./warcvalidator -f $I; done

With this, you can't distinguish any result as you're in "quiet" mode.

Please, use the verbose mode and put an "echo" to debug a bit like this :

for I in $(ls wdata/testwfile/*.warc); do echo "---- $I"; ./warcvalidator -v -f 
$I; done

Now you can see valid WARC.

Hope this help.

cheers
Y.

Original comment by voidptr...@gmail.com on 14 Sep 2008 at 9:33