Allow searching for file content offset

[amerlyq]

Given two raw binary files ./dump.bin and ./chunk how to search one in another ? We can open first file r2 -n dump.bin. But how to specify for / to use content of ./chunk? If there is no way to specify, could it be possible to add new option for searching ?

[amerlyq]

Lightning fast implementation :) Testing results for this subset:

bin-copy dump.bin 0x8000 0x50 > chunk
printf '\x55' > chunk1b
r2 -n dump.bin

Requested function works as expected. Thanks.

> /f chunk
Searching 80 bytes in [0x0-0x13a000]
hits: 1
0x00008000 hit0_0 005e70...

> /f chunk1b
Searching 1 bytes in [0x0-0x13a000]
hits: 400
...

Specifying only offset (till file end) don't work.

> /f chunk 20
Invalid offset or size

> /f chunk 0x20
Invalid offset or size

Size don't affect anything, only offset.

> /f chunk 20 20
Searching 60 bytes in [0x0-0x13a000]
hits: 1
0x00008014 hit1_0 005e6f...

> /f chunk 0x20 0x20
Searching 48 bytes in [0x0-0x13a000]
hits: 1
0x00008020 hit5_0 005e6f...

However, size have peculiar parsing rules...

> /f chunk 20 0x20
Searching 60 bytes in [0x0-0x13a000]
hits: 1
0x00008014 hit4_0 005e6f...

> /f chunk 0x20 20
Invalid offset or size

And can't read files with st_size == 0

> /f /proc/self/mem 0x40000 20
Invalid offset or size

[radare]

Can you add tests for this? Ill fix those issues today.

Thanks for testing!

On 25 Mar 2017, at 03:33, Dmytro Kolomoiets notifications@github.com wrote:

Lightning fast implementation :) Testing results for this subset:

bin-copy dump.bin 0x4000 0x50 > chunk printf '\x55' > chunk1b r2 -n dump.bin Requested function works as expected. Thanks.

/f chunk Searching 80 bytes in [0x0-0x13a000] hits: 1 0x00008000 hit0_0 005e70...

/f chunk1b Searching 1 bytes in [0x0-0x13a000] hits: 400 ... Specifying only offset (till file end) don't work.

/f chunk 20 Invalid offset or size

/f chunk 0x20 Invalid offset or size Size don't affect anything, only offset.

/f chunk 20 20 Searching 60 bytes in [0x0-0x13a000] hits: 1 0x00008014 hit1_0 005e6f...

/f chunk 0x20 0x20 Searching 48 bytes in [0x0-0x13a000] hits: 1 0x00008020 hit5_0 005e6f... However, size have peculiar parsing rules...

/f chunk 20 0x20 Searching 60 bytes in [0x0-0x13a000] hits: 1 0x00008014 hit4_0 005e6f...

/f chunk 0x20 20 Invalid offset or size And can't read files with st_size == 0

/f /proc/self/mem 0x40000 20 Invalid offset or size — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[radare]

Proc pid mem is not a regular file. You cant read it with a slurp or use it as a keyword because it can be only mmapped and seeked

On 25 Mar 2017, at 03:33, Dmytro Kolomoiets notifications@github.com wrote:

Lightning fast implementation :) Testing results for this subset:

bin-copy dump.bin 0x4000 0x50 > chunk printf '\x55' > chunk1b r2 -n dump.bin Requested function works as expected. Thanks.

/f chunk Searching 80 bytes in [0x0-0x13a000] hits: 1 0x00008000 hit0_0 005e70...

/f chunk1b Searching 1 bytes in [0x0-0x13a000] hits: 400 ... Specifying only offset (till file end) don't work.

/f chunk 20 Invalid offset or size

/f chunk 0x20 Invalid offset or size Size don't affect anything, only offset.

/f chunk 20 20 Searching 60 bytes in [0x0-0x13a000] hits: 1 0x00008014 hit1_0 005e6f...

/f chunk 0x20 0x20 Searching 48 bytes in [0x0-0x13a000] hits: 1 0x00008020 hit5_0 005e6f... However, size have peculiar parsing rules...

/f chunk 20 0x20 Searching 60 bytes in [0x0-0x13a000] hits: 1 0x00008014 hit4_0 005e6f...

/f chunk 0x20 20 Invalid offset or size And can't read files with st_size == 0

/f /proc/self/mem 0x40000 20 Invalid offset or size — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[Maijin]

https://github.com/radare/radare2-regressions

[amerlyq]

Ok, I will submit tests there. But I need some help. I can't easily understand how to cook ./chunk file to use with /f chunk. Or how to replace it with some pipe/temp/etc in current testsuite in portable way. At least point me to tests where that was done as example.

I can replace ./dump.bin with cooked FILE='malloc://1024', and therefore I would prefer to cook needle as well instead of pushing it into repo.

P.S. I know pid mem isn't regular, but didn't think there were such limitations in slurp itself. Maybe I hoped for too much of reasonable conditional behavior :)

[radare]

Take any file from bins that are already commited

On 25 Mar 2017, at 14:55, Dmytro Kolomoiets notifications@github.com wrote:

Ok, I will submit tests there. But I need some help. I can't easily understand how to cook ./chunk file to use with /f chunk. Or how to replace it with some pipe/temp/etc in current testsuite in portable way. At least point me to tests where that was done as example.

I can replace ./dump.bin with cooked FILE='malloc://1024', and therefore I would prefer to cook needle as well instead of pushing it into repo.

P.S. I know pid mem isn't regular, but didn't think there were such limitations in slurp itself. Maybe I hoped for too much of reasonable conditional behavior :)

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[amerlyq]

Alright, I will do that. But is there a way to cook two files simultaneously at all? Like mmap second file and wx into its fd ?

On Sat, Mar 25, 2017, 17:04 radare notifications@github.com wrote:

Take any file from bins that are already commited

On 25 Mar 2017, at 14:55, Dmytro Kolomoiets notifications@github.com wrote:

Ok, I will submit tests there. But I need some help. I can't easily understand how to cook ./chunk file to use with /f chunk. Or how to replace it with some pipe/temp/etc in current testsuite in portable way. At least point me to tests where that was done as example.

I can replace ./dump.bin with cooked FILE='malloc://1024', and therefore I would prefer to cook needle as well instead of pushing it into repo.

P.S. I know pid mem isn't regular, but didn't think there were such limitations in slurp itself. Maybe I hoped for too much of reasonable conditional behavior :)

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/radare/radare2/issues/7121#issuecomment-289217256, or mute the thread https://github.com/notifications/unsubscribe-auth/AFdvMErqYYTOHZ5z0TL6is4wZvTXwE_Aks5rpSzpgaJpZM4Mom_0 .

[amerlyq]

Beware, tests may need adjusting for expected values, as currently most of them fail with 'invalid argument'.

By the way, I don't know what's going on with args here, nor with search window size. Therefore no reasonable test yet.

[0x00000000]> /f ../bins/elf/analysis/hello.raw 0x3 0x4
Searching 47 bytes in [0x0-0x40]
hits: 0
[0x00000000]> /f ../bins/elf/analysis/hello.raw 0x5 0x4
Invalid offset or size
[0x00000000]> /f ../bins/elf/analysis/hello.raw 0x5 0x6
Searching 45 bytes in [0x0-0x40]
hits: 0
[0x00000000]> /f ../bins/elf/analysis/hello.raw 0x6 0x3
Invalid offset or size

... (some random session here)

[0x00000000]> /f ../bins/elf/analysis/hello.raw 1 1
Searching 257 bytes in [0x0-0x40]
hits: 0

[radare]

Then mark them as broken

On 25 Mar 2017, at 22:17, Dmytro Kolomoiets notifications@github.com wrote:

Beware, tests may need adjusting for expected values, as currently most of them fail with 'invalid argument'.

By the way, I don't know what's going on with args here, nor with search window size. Therefore no reasonable test yet.

[0x00000000]> /f ../bins/elf/analysis/hello.raw 0x3 0x4 Searching 47 bytes in [0x0-0x40] hits: 0 [0x00000000]> /f ../bins/elf/analysis/hello.raw 0x5 0x4 Invalid offset or size [0x00000000]> /f ../bins/elf/analysis/hello.raw 0x5 0x6 Searching 45 bytes in [0x0-0x40] hits: 0 [0x00000000]> /f ../bins/elf/analysis/hello.raw 0x6 0x3 Invalid offset or size

... (some random session here)

[0x00000000]> /f ../bins/elf/analysis/hello.raw 1 1 Searching 257 bytes in [0x0-0x40] hits: 0 — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[amerlyq]

Ping. What about bugfix to those commits? Thanks to you I could apply some forensic in r2 to my dataset in this two weeks, but being able to use currently broken offset/size would be of much help to automate my scripts :). As currently I clumsy shape/crop chunks of data in external tools before searching. Sigh :(

[amerlyq]

Maybe, someone at least could reopen this issue until bugfixes confirmed?

[radare]

You can commit a block of a file thats already in r2r/bins to write your tests.

[ret2libc]

This issue has been moved from radareorg/radare2 to radareorg/ideas as we are trying to clean our backlog and this issue has probably been created a long while ago. This is an effort to help contributors understand what are the actionable items they can work on, prioritize issues better and help users find active/duplicated issues more easily. If this is not an enhancement/improvement/general idea but a bug, feel free to ask for re-transfer to main repo. Thanks for your understanding and contribution with this issue.

[trufae]

I dint think this issue should be moved to ideas. It must be closed or reviewed and see if we miss any of the tests mentioned