benibela / xidel

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
http://www.videlibri.de/xidel.html
GNU General Public License v3.0
681 stars 42 forks source link

[Bug] file:move() returns ERangeError: Range check error #56

Closed Reino17 closed 4 years ago

Reino17 commented 4 years ago
xidel --version
Xidel 0.9.9
(20200930.7580.9cac3c17e8ed)

http://www.videlibri.de/xidel.html
by Benito van der Zander <benito@benibela.de>

xidel -se "'a.txt' ! (file:exists(.),file:last-modified(.))"
true
2020-10-24T14:08:12-02:00

xidel -se "file:move('a.txt','b.txt')"
An unhandled exception occurred at $00445B64:
ERangeError: Range check error
  $00445B64
  $0049E814
  $0048DEC4
  $0048DD80
  $00436E47
  $00438ACF
  $00436276
  $00435D9F
  $0043F9AD

A second bug is that file:last-modified() here returns the wrong timezone-offset. It should've been +02:00.

benibela commented 4 years ago

How? I do not get that:

Z:\tmp>echo a > a.txt                                                                          

Z:\tmp>xidel-0.9.9.20200930.7580.9cac3c17e8ed.win32\xidel.exe  -se "file:move('a.txt','b.txt')"

Z:\tmp>
Reino17 commented 4 years ago

I have very little experience with Pascal, but after having done some reading on "range checking" I've tried to compile xidel with -Cr...

C:\FPC-3.0.4\bin\i386-win32>fpc.exe -h | find "Range"
      -Cr        Range checking

...and I thought I would then get a compilation-error on the matter. I did not. I guess I misunderstand what the compiler option is for. xidel still compiled without errors and calling the binary with file:move() again I get ERangeError: Range check error.

Is there something else I can do to pinpoint the cause of this bug (because you can't, I guess)?

benibela commented 4 years ago

...and I thought I would then get a compilation-error on the matter. I did not. I guess I misunderstand what the compiler option is for.

-Cr enables the range checking. You never get an range check error without -Cr

Is there something else I can do to pinpoint the cause of this bug (because you can't, I guess)?

Compile it with -gl, so it can print a useful backtrace

Reino17 commented 4 years ago
An unhandled exception occurred at $00445154:
ERangeError: Range check error
  $00445154  FILEEXISTSASTRUEFILE,  line 181 of D:/Storage/Media/Binaries/xidel/videlibri-code_hg/components/pascal/data/xquery_module_file.pas
  $00497F83  TXQTERMNAMEDFUNCTION__EVALUATE,  line 2790 of D:/Storage/Media/Binaries/xidel/videlibri-code_hg/components/pascal/data/xquery_terms.inc
  $004867EE  TXQUERY__EVALUATE,  line 7285 of D:/Storage/Media/Binaries/xidel/videlibri-code_hg/components/pascal/data/xquery.pas
  $004866A7  TXQUERY__EVALUATE,  line 7267 of D:/Storage/Media/Binaries/xidel/videlibri-code_hg/components/pascal/data/xquery.pas
  $0043599E  TPROCESSINGCONTEXT__EVALUATEQUERY,  line 2079 of xidelbase.pas
  $00437714  TEXTRACTION__PROCESS,  line 2387 of xidelbase.pas
  $00434B90  SUBPROCESS,  line 1864 of xidelbase.pas
  $00434644  TPROCESSINGCONTEXT__PROCESS,  line 1965 of xidelbase.pas
  $0043EAFA  PERFORM,  line 3651 of xidelbase.pas

This you mean?

benibela commented 4 years ago

fixed? https://github.com/benibela/internettools/commit/f396528ae985abc756462c00118b5c8717ad83c3

benibela commented 4 years ago

And now it also does not use FILEEXISTSASTRUEFILE anymore for moving https://github.com/benibela/internettools/commit/84155681854ffe197d5f36481bda8a5a6aced0d4

Reino17 commented 4 years ago

fixed? benibela/internettools@f396528

Yes. Thank you!
For testing purposes this was the first time I used file:move(). I noticed the oldest xidel binary I have lying around on my pc, which is v0.9.8, shows the same error. So I think there's a good chance more Windows users were vulnerable.

xidel -se "file:last-modified('a.txt')"
2020-10-24T14:08:12+02:00

Fixed as well. Thanks.