no matches found

[Mis012]

I get "no matches found", even though with diaphora, I got quite a few matches (sadly the compiled version has no symbols and I'm not sure if the format even supports it). Is this to be expected, or am I doing something wrong? I know this is a young project, but the documentation seems to suggest it works well nevertheless. So I understand if the answer is that it's indeed possible it won't work at all, but I don't want to give up my hope without asking :)

[joxeankoret]

Is it a C code base or C++? If it's a C code base, it's a bug. If it's C++ or anything else, it's expected at this stages.

[Mis012]

Mis012@Mis012::[censored]> find -name *.c | wc -l
645
Mis012@Mis012:[censored]> find -name *.cpp | wc -l
2

seems alright

[joxeankoret]

It might be a bug but I have no data whatsoever to verify this point. Can you tell me which source code are you trying to port symbols from? If you prefer to send me an encrypted e-mail, you can get my PGP key from here: http://joxeankoret.com/

[Mis012]

-----BEGIN PGP MESSAGE-----

hQIOA+Z2ZGBn4vGBEAf/Wr69nmqDLGFQgTc9ix53fod/oOJin/07umasB6CqkvmO CAUSb7bUsqIF5OzK8dGdsuWI64xgJnmaPoHAyCT2Z2MWGPN1USKTWrCT0sluKRt1 vumo70I1pqXUrKf189qVrGRWwFzu2x8YkOsdbuz+BUrHErDsMB1IM8atex7OyL6h lRMsD1RrHMF2NgYVPa1ZWpgMcd7lClzVwcSSrD4RdjA2RAxeIGLlKEENkOwWcFvk X2RCsQmuuM2L6wcEyos6S6LRXWL3kjtWc6YqfQV+pAuAVn8AvS0YWfcU8WJ6yLec TbXptu6DW1/b8VYI9xWMS2x1q0XwBY1I8bT3cfUKngf8DFl0twEow5kATjoyw/md ArHRD/2+ZCAteORunGb+2V2KN2xf27erhqUhGaeIsUur85rC6WNL7S7Ei+eJufvL KHGA8QsQ/aZjozOTv3zRCwgPpLhQHdfF5/c59PEczmp/0DsRMaROFePhh0IHBW/8 GSrtoFCyPKMRqrvvHyfrrp7N3EkfUEKEWtwCSLdcb9xbrdZ+eHW/pRGG4v4p5h58 m468574iOxtIQ/+Rxqn1nOZ98xpaBsT7ITMKPkg9L4BEfWecFdBt5jUIqlU/RizO uytDG4C+JeIpR25PaGMlgXI5lq5yLiuQrUv4kW96w6DAmSpo1mkoCs95scQoUbpA oNLAAQFnTeSXnYZkD9TdTdx1lvcL0lCN7HkqBfoy7gmfO6xnRWU6KbKb8ElA3s1D xR9XaSa+YwfhSBlriO1tVj+SvDIF3Xc1RGZWkVAs9gKft+D7riMrywZV3XnyHYZp jNq9MZORGGohV6crgeEdjki/xNCKr6vgubwIj33aDFbhtGtNbj7X1rLwPSybvQk2 xiEE1UXA/b7AqT6qITju3Tx7BQgBPVxD4FWq14M2B3Jw24Z8DFK++LFMAeX/NyXi WbItgiI= =h0MU -----END PGP MESSAGE-----

[joxeankoret]

Got it. Just need a binary.

[Mis012]

secret.gpg.not.txt mumble fsck upload filters mumble

[joxeankoret]

It works perfectly here. For example: there is a perfect match to some kind of malloc function to idaapi.get_imagebase() + 0x7dea414. In any case, try executing the following command inside the sources directory:

$ find . -name "*.h" -exec dirname {} ';' | sort -u | awk '{ print "-I"$1 }' | xargs echo

Then, open your 'sbd.project' file and replace the cflags value in the [PROJECT] section with the output of the previous command and re-do the export.

[Mis012]

still nothing :( I used this, could that be a problem? https://github.com/Surge1223/ida-mbn-sbl-loader/blob/master/ida-mbn-sbl-loader.py

[joxeankoret]

Wait, what IDA version are you using?

[Mis012]

6.8

[joxeankoret]

Wow... it's very oldie. Let me try with that version.

[joxeankoret]

Try using the normal IDA's loader instead of that Python loader, it works for me.

[Mis012]

ah, I was actually using that by accident neither works

[joxeankoret]

Can you share the log of what IDA says? Like this example:

[Sun Oct 28 12:15:23 2018] Decision tree based system available
[Sun Oct 28 12:15:23 2018] Finding best matches...
[Sun Oct 28 12:15:23 2018] Minimum score for calculations: 0.000000
[Sun Oct 28 12:15:23 2018] Minimum score to show results : 0.000000
[Sun Oct 28 12:15:23 2018] Finding callgraph matches...
[Sun Oct 28 12:15:23 2018] Iteration 1, discovered a total of 52 row(s)...
[Sun Oct 28 12:15:24 2018] Iteration 2, discovered a total of 28 row(s)...
[Sun Oct 28 12:15:24 2018] Iteration 3, discovered a total of 34 row(s)...

[joxeankoret]

I can assure you it works:

[Mis012]

actually, if I remove the -src.sqlite, I get an error:

[Sun Oct 28 15:52:41 2018] Exporting current database...
[Sun Oct 28 15:52:41 2018] Exporting database [...]\-src.sqlite
[Sun Oct 28 15:52:42 2018] ERROR: 'utf8' codec can't decode byte 0xb0 in position 0: invalid start byte
'utf8' codec can't decode byte 0xb0 in position 0: invalid start byte
Traceback (most recent call last):
  File "C:\Program Files (x86)\IDA 6.8\python\idaapi.py", line 601, in IDAPython_ExecScript
    execfile(script, g)
  File "[...]/pigaios/sourceimp_ida.py", line 509, in <module>
    main()
  File "[...]/pigaios/sourceimp_ida.py", line 492, in main
    importer = CIDABinaryToSourceImporter()
  File "[...]/pigaios/sourceimp_ida.py", line 365, in __init__
    CBinaryToSourceImporter.__init__(self, GetIdbPath())
  File "[...]/pigaios\sourceimp_core.py", line 103, in __init__
    self.open_or_create_database()
  File "[...]/pigaios/sourceimp_ida.py", line 396, in open_or_create_database
    exporter.export(self.db_filename)
  File "[...]/pigaios\sourcexp_ida.py", line 457, in export
    self.do_export(f)
  File "[...]/pigaios\sourcexp_ida.py", line 414, in do_export
    len(constants), json.dumps(list(constants)), loops, len(switches),
  File "C:\Python27\Lib\json\__init__.py", line 243, in dumps
    return _default_encoder.encode(obj)
  File "C:\Python27\Lib\json\encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "C:\Python27\Lib\json\encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xb0 in position 0: invalid start byte

[joxeankoret]

This is odd, it worked before and now it doesn't? Please try adding the following lines to the beginning of sourceimp_ida.py:

import sys
reload(sys)
sys.setdefaultencoding("utf-8")

[Mis012]

I think I most likely missed the warning, and it didn't show again

[joxeankoret]

Can it also be that it failed at export time before for a reason and it was trying to match against an empty IDA's exported database? That makes sense to me.

[Mis012]

probably

same error now

[joxeankoret]

Linux or Mac? Can you please try removing the -src.sqlite database and re-running it from IDA like the following:

$ LANG=en_US.UTF8 /path/to/your/idaq

[Mis012]

ehm wine on ehm ... Linux

[joxeankoret]

Woah... try with that environment variable anyway, please.

[Mis012]

Mis012@Mis012:[censored]> echo $LANG
en_US.UTF-8

[joxeankoret]

Try with the following patch, please:

diff --git a/sourcexp_ida.py b/sourcexp_ida.py
index ff5797d..a933b22 100644
--- a/sourcexp_ida.py
+++ b/sourcexp_ida.py
@@ -256,7 +256,7 @@ class CBinaryToSourceExporter:
           if str_constant is not None:
             if len(str_constant) > 1:
               #print("0x%x: %s" % (ea, repr(str_constant)))
-              constants.add(str(str_constant))
+              constants.add(unicode(str_constant, errors='ignore'))

     return constants, externals, globals_uses

[joxeankoret]

Wait, I know the reason, just give me a second...

[joxeankoret]

And now you should be able to export cleanly with this commit: https://github.com/joxeankoret/pigaios/commit/1a36ce7606e4bd136a9637948c24074b126af462

PS: Please remember to remove the XXX-src.sqlite database before (I will fix it too soon).

[Mis012]

works now, thanks :) I'll report when it's done exporting

[Mis012]

ran for a while... ...then

[Sun Oct 28 17:01:47 2018] Exporting current database...
[Sun Oct 28 17:01:47 2018] Exporting database C:\users\Mis012\Desktop\workbench\sm-a300fu\sbl1-src.sqlite
[Sun Oct 28 17:07:22 2018] Finding best matches...
[Sun Oct 28 17:08:24 2018] Minimum score for calculations: 0.000000
[Sun Oct 28 17:08:24 2018] Minimum score to show results : 0.000000
[Sun Oct 28 17:08:24 2018] ERROR: 'utf8' codec can't decode byte 0x98 in position 0: invalid start byte

[Mis012]

https://github.com/joxeankoret/pigaios/blob/master/sourceimp_core.py#L166

[joxeankoret]

Please pull the latest changes (https://github.com/joxeankoret/pigaios/commit/5106bee86fe65d9f0c96f901b4c46028df2a31d8) and re-export the symbols from the source code.

[Mis012]

no difference :(

[joxeankoret]

It's very odd. Please contact me privately so I can share with you some files to test it out.

[joxeankoret]

Fixed (in supported IDA versions) with the latest commits.