search

sepinf-inc / IPED

IPED Digital Forensic Tool. It is an open source software that can be used to process and analyze digital evidence, often seized at crime scenes by law enforcement or in a corporate investigation by private examiners.
Other
884 stars 209 forks source link

ALeapp task #2095

Open patrickdalla opened 4 months ago

patrickdalla commented 4 months ago

Hi @lfcnassif , I decided to create this pull request, but marked it as draft, as I am leaving on vacation

lfcnassif commented 4 months ago

Thanks @patrickdalla! I think it is good to create the PR in draft mode, since this is a big implementation and others can see the progress and comment on.

patrickdalla commented 1 month ago

I've made some more tests and pontual enhancements and corrections. I think I should some third developer evaluation/opinion before continuing, @lfcnassif. I fact, if the design is good and no errors found, I think this can be merged as is.

lfcnassif commented 1 month ago

Thank you very much @patrickdalla! This is a very important feature, I'll try to test it after other ready PRs scheduled for 4.2 in the queue.

One question: if an UFDR is processed with this PR, will we get duplicated results coming from PA decoding and from ALeapp decoding? If yes, I think it should be avoided or be configurable. Maybe with a similar approach used for WhatsApp today, or maybe with the approach that would be implemented for #2012.

patrickdalla commented 1 month ago

Aleapp processes data from FFS. It expects data files in its original path, not in the way they are in UFDR.

Em seg., 20 de mai. de 2024, 21:01, Luis Filipe Nassif < @.***> escreveu:

Thank you very much @patrickdalla https://github.com/patrickdalla! This is a very important feature, I'll try to test it after other ready PRs scheduled for 4.2 in the queue.

One question: if an UFDR is processed with this PR, will we get duplicated results coming from PA decoding and from ALeapp decoding? If yes, I think it should be avoided or be configurable. Maybe with a similar approach used for WhatsApp today, or maybe with the approach that would be implemented on

2012 https://github.com/sepinf-inc/IPED/issues/2012.

— Reply to this email directly, view it on GitHub https://github.com/sepinf-inc/IPED/pull/2095#issuecomment-2121504987, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG247S7KCJ75V56GGLYFXADZDKMGZAVCNFSM6AAAAABDVRWLH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGUYDIOJYG4 . You are receiving this because you were mentioned.Message ID: @.***>

patrickdalla commented 1 month ago

Anyway, there is already a config file to inform which Aleapp parser not to run.

Em seg., 20 de mai. de 2024, 22:22, Patrick Bernardina < @.***> escreveu:

Aleapp processes data from FFS. It expects data files in its original path, not in the way they are in UFDR.

Em seg., 20 de mai. de 2024, 21:01, Luis Filipe Nassif < @.***> escreveu:

Thank you very much @patrickdalla https://github.com/patrickdalla! This is a very important feature, I'll try to test it after other ready PRs scheduled for 4.2 in the queue.

One question: if an UFDR is processed with this PR, will we get duplicated results coming from PA decoding and from ALeapp decoding? If yes, I think it should be avoided or be configurable. Maybe with a similar approach used for WhatsApp today, or maybe with the approach that would be implemented on #2012 https://github.com/sepinf-inc/IPED/issues/2012.

— Reply to this email directly, view it on GitHub https://github.com/sepinf-inc/IPED/pull/2095#issuecomment-2121504987, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG247S7KCJ75V56GGLYFXADZDKMGZAVCNFSM6AAAAABDVRWLH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGUYDIOJYG4 . You are receiving this because you were mentioned.Message ID: @.***>

patrickdalla commented 1 month ago

Also, thinking twice, the task searches for items from already processed items and its paths. As IPED uses UFDR xml info to restore original path in cel FS, maybe, with some simple modification, Aleapp parser can find these items too. I will check it.

Em seg., 20 de mai. de 2024, 22:22, Patrick Bernardina < @.***> escreveu:

Aleapp processes data from FFS. It expects data files in its original path, not in the way they are in UFDR.

Em seg., 20 de mai. de 2024, 21:01, Luis Filipe Nassif < @.***> escreveu:

Thank you very much @patrickdalla https://github.com/patrickdalla! This is a very important feature, I'll try to test it after other ready PRs scheduled for 4.2 in the queue.

One question: if an UFDR is processed with this PR, will we get duplicated results coming from PA decoding and from ALeapp decoding? If yes, I think it should be avoided or be configurable. Maybe with a similar approach used for WhatsApp today, or maybe with the approach that would be implemented on #2012 https://github.com/sepinf-inc/IPED/issues/2012.

— Reply to this email directly, view it on GitHub https://github.com/sepinf-inc/IPED/pull/2095#issuecomment-2121504987, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG247S7KCJ75V56GGLYFXADZDKMGZAVCNFSM6AAAAABDVRWLH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGUYDIOJYG4 . You are receiving this because you were mentioned.Message ID: @.***>

lfcnassif commented 1 month ago

Anyway, there is already a config file to inform which Aleapp parser not to run.

Great!

Also, thinking twice, the task searches for items from already processed items and its paths. As IPED uses UFDR xml info to restore original path in cel FS, maybe, with some simple modification, Aleapp parser can find these items too. I will check it.

I think it can be useful, for example, for an application supported by ALeapp but not supported by PA, or if PA decoding brings incomplete or eventually wrong results. For this last example, disabling PA results importing per application may be needed, but that is related to #2012.

PS: Running ALeapp into AB backups (when #2079 is merged) should be very useful too.

patrickdalla commented 1 month ago

Yes, it worked with the modifications of last commit, ALeapp plugins found and processed items from UFDR.

prosch88 commented 3 weeks ago

I'm eager to see this merged into main. Is this bringing ileapp too? Or is this planned for another PR?

lfcnassif commented 3 weeks ago

I'm eager to see this merged into main. Is this bringing ileapp too? Or is this planned for another PR?

AFAIK this is just about ALeapp integration, iLeapp should be done later. Would you like to help testing? There is a snapshot with this support below, you should be logged in github to see the download link: https://github.com/sepinf-inc/IPED/actions/runs/9180006157

lfcnassif commented 3 weeks ago

Hi @patrickdalla, an user/developer is trying to test this, but got an error "no module named geopy". What is the updated python dependency list needed to run this PR?

lfcnassif commented 3 weeks ago

Just found this list in Teams, posting here for those willing to help testing, let me know if it is outdated:

bcrypt==3.2.0
beautifulsoup4==4.8.2
bencoding
blackboxprotobuf
fitdecode==0.10.0
folium==0.14.0
geopy==2.3.0
packaging==20.1
pillow
polyline==2.0.0
protobuf==3.10.0
PyCryptodome
PySimpleGUI
pytz
simplekml
wheel
xlsxwriter==3.1.1
xmltodict
python-magic
libmagic
python-magic-bin
filetype

Just put those into a requirements.txt file and run from iped embedded python: pip install -r requirements.txt