search

sepinf-inc / IPED

IPED Digital Forensic Tool. It is an open source software that can be used to process and analyze digital evidence, often seized at crime scenes by law enforcement or in a corporate investigation by private examiners.
Other
947 stars 218 forks source link

IPED 3.18.12 transforms absolute path in relative, causing error #891

Closed arisjr closed 2 years ago

arisjr commented 2 years ago

Dear developers,

I've set ledDie=/mnt/led/die/rfdie.dat, an absolute path, and IPED interpret as relative path, turning ledDie path into /root/IPED/iped/mnt/led/die/rfdie.dat.

I'm using IPED 3.18.12 on linux Ubuntu 20.04. Follows the run log with the error.

Changing the flag to the default value, using relative path to the embedded ledDie, works.

Regards, Aristeu

2021-12-24 09:00:36 [INFO] [gpinf.indexer.IndexFiles] Indexador e Processador de Evidências Digitais 3.18.12
2021-12-24 09:00:36 [INFO] [indexer.util.CustomLoader] Adding to classpath: /root/IPED/optional_jars/java-dbx-1.1-p5.jar
2021-12-24 09:00:36 [INFO] [indexer.util.CustomLoader] Adding to classpath: /root/IPED/optional_jars/Readme.txt
2021-12-24 09:00:36 [INFO] [indexer.util.CustomLoader] Adding to classpath: /root/IPED/optional_jars/stanford-corenlp-3.8.0.jar
2021-12-24 09:00:36 [INFO] [indexer.util.CustomLoader] Adding to classpath: /root/IPED/optional_jars/telegram-decoder-impl-1.0.5-SNAPSHOT.jar
2021-12-24 09:00:36 [INFO] [indexer.util.CustomLoader] Adding to classpath: /usr/share/java/sleuthkit-4.6.5.jar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/IPED/iped-3.18.12/lib/log4j-slf4j-impl-2.17.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/IPED/iped/lib/log4j-slf4j-impl-2.17.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2021-12-24 09:00:36 [INFO] [gpinf.indexer.Configuration] Loading configuration from /root/IPED/iped/profiles/pt-BR/pedo
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] Operating System: Linux
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] Java Version: 1.8.0_312
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] Architecture: amd64
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] Current Directory: /operacoes/operacao_teste/alvo1/HD1
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] CPU Cores: 16
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] numThreads: 8
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] Memory (Heap) Available: 5726 MB
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] Arguments: -Djava.awt.headless=true -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Xmx6G /root/IPED/iped/iped.jar -d imagem.dd -o SARD --portable --nologfile --nogui -profile pedo
2021-12-24 09:00:36 [INFO] [indexer.process.Statistics] Config Options: enableRegexSearch=true | addUnallocated=true | ignoreHardLinks=true | enableCarving=true | minRawStringSize=4 | sortPDFChars=false | outputOnSSD=false | maxBackups=10 | enableLedWkff=true | autoManageCols=true | indexCorruptedFiles=true | robustImageReading=false | enableProjectVicHashLookup=true | extraCharsToIndex= | pdfToImgResolution=250 | enableImageSimilarity=true | ignoreDuplicates=false | enableOCR=false | enableVideoThumbs=true | pageSegMode=1 | enableAudioTranscription=false | addFileSlacks=true | indexFileContents=true | maxFileSize2OCR=100000000 | indexTempOnSSD=true | entropyTest=true | filterNonLatinChars=false | tskJarPath=/usr/share/java/sleuthkit-4.6.5.jar | processFileSignatures=true | projectVicHashSetPath=/mnt/ProjectVic/latest.json | externalConvMaxMem=512M | numThreads=8 | timeOutPerMB=2 | enableKnownMetCarving=true | enableHTMLReport=true | enableExternalParsing=false | indexUnallocated=true | locale=pt-BR | enableDocThumbs=false | enableGraphGeneration=true | enableNamedEntityRecogniton=false | hash=md5; sha-1; sha-256; edonkey | exportFileProps=true | enableFileParsing=true | backupInterval=3600 | forceMerge=false | enableKff=true | mplayerPath=/usr/bin/mplayer | embutirLibreOffice=true | enableImageThumbs=true | enableLanguageDetect=true | indexTemp=/mnt/ipedtmp | expandContainers=true | optional_jars=../optional_jars/ | openImagesCacheWarmUpThreads=256 | excludeKffIgnorable=false | enableKFFCarving=true | numExternalParsers=auto | preOpenImagesOnSleuth=false | maxPDFTextSize2OCR=100 | OCRLanguage=por | timeOut=180 | phoneParsersToUse=internal | ledDie=/mnt/led/die/rfdie.dat | externalParsingMaxMem=512M | kffDb=/mnt/kff/kff.db | unallocatedFragSize=10737418240 | minFileSize2OCR=10000 | convertCharsToLowerCase=true | ledWkffPath=/mnt/led/wkff | confdir=/root/IPED/iped/profiles/pt-BR/pedo/conf | commitIntervalSeconds=1800 | textSplitSize=10485760 | searchThreads=1 | externalPdfToImgConv=true | processImagesInPDFs=true | convertCharsToAscii=true | photoDNAHashDatabase=/mnt/PhotoDNA/PhotoDNAChildPornHashes.txt | enableLedDie=true | pdfToImgLib=icepdf | indexUnknownFiles=true | minItemSizeToFragment=104857600 | openImagesCacheWarmUpEnabled=false | regripperFolder=../regripper/ | openWithDoubleClick=ask_if_exe | numImageReaders=auto | enablePhotoDNA=false | useNIOFSDirectory=false |
2021-12-24 09:00:39 [INFO] [indexer.process.Manager] Evidence 1: '/operacoes/operacao_teste/alvo1/HD1/imagem.dd'
2021-12-24 09:00:39 [MSG] [indexer.process.ProgressConsole] Configurando índice...
2021-12-24 09:00:39 [INFO] [indexer.process.Manager] Creating index...
2021-12-24 09:00:39 [INFO] [indexer.util.ConfiguredFSDirectory] Using MMapDirectory to open index...
2021-12-24 09:00:39 [INFO] [indexer.process.Worker] Starting Tika
2021-12-24 09:00:39 [WARN] [tika.config.InitializableProblemHandler$3] J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.

2021-12-24 09:00:39 [INFO] [browsers.parsers.EdgeWebCacheParser] Libesedb library version: 20210424
2021-12-24 09:00:40 [WARN] [indexer.parsers.PDFOCRTextParser] Plugin JPEG2000 not found, JPX images will not be decoded from PDFs. You can download it from https://mvnrepository.com/artifact/com.github.jai-imageio/jai-imageio-jpeg2000/1.3.0 and put it in optional_jars folder. Warn: that plugin is worse to decode JPX outside of PDFs!
2021-12-24 09:00:40 [INFO] [indexer.process.Worker] Starting SkipCommitedTask
2021-12-24 09:00:40 [MSG] [indexer.process.ProgressConsole] Inicializando SkipCommitedTask
2021-12-24 09:00:40 [INFO] [indexer.process.Worker] Starting IgnoreHardLinkTask
2021-12-24 09:00:40 [MSG] [indexer.process.ProgressConsole] Inicializando IgnoreHardLinkTask
2021-12-24 09:00:40 [INFO] [indexer.process.Worker] Starting TempFileTask
2021-12-24 09:00:40 [MSG] [indexer.process.ProgressConsole] Inicializando TempFileTask
2021-12-24 09:00:40 [INFO] [indexer.process.Worker] Starting SignatureTask
2021-12-24 09:00:40 [MSG] [indexer.process.ProgressConsole] Inicializando SignatureTask
2021-12-24 09:00:40 [INFO] [indexer.process.Worker] Starting SetTypeTask
2021-12-24 09:00:40 [MSG] [indexer.process.ProgressConsole] Inicializando SetTypeTask
2021-12-24 09:00:40 [INFO] [indexer.process.Worker] Starting SetCategoryTask
2021-12-24 09:00:40 [MSG] [indexer.process.ProgressConsole] Inicializando SetCategoryTask
2021-12-24 09:00:40 [INFO] [indexer.process.Worker] Starting RefineCategoryTask
2021-12-24 09:00:40 [MSG] [indexer.process.ProgressConsole] Inicializando RefineCategoryTask
2021-12-24 09:00:40 [INFO] [indexer.process.Worker] Starting HashTask
2021-12-24 09:00:40 [MSG] [indexer.process.ProgressConsole] Inicializando HashTask
2021-12-24 09:00:41 [INFO] [indexer.process.Worker] Starting KFFTask
2021-12-24 09:00:41 [MSG] [indexer.process.ProgressConsole] Inicializando KFFTask
2021-12-24 09:00:41 [INFO] [indexer.process.Worker] Starting LedKFFTask
2021-12-24 09:00:41 [MSG] [indexer.process.ProgressConsole] Inicializando LedKFFTask
2021-12-24 09:00:41 [ERROR] [process.task.LedKFFTask] Invalid LED database path: /mnt/led/wkff
2021-12-24 09:00:41 [INFO] [indexer.process.Worker] Starting ProjectVICHashLookup
2021-12-24 09:00:41 [MSG] [indexer.process.ProgressConsole] Inicializando ProjectVICHashLookup
2021-12-24 09:00:41 [INFO] [process.task.ProjectVICHashLookup] Loading ProjectVic json /mnt/ProjectVic/latest.json
2021-12-24 09:02:05 [INFO] [process.task.ProjectVICHashLookup] Compressing ProjectVic hashes.
2021-12-24 09:02:05 [INFO] [process.task.ProjectVICHashLookup] Sorting ProjectVic hashes.
2021-12-24 09:02:19 [INFO] [process.task.ProjectVICHashLookup] Sorting ProjectVic hashes.
2021-12-24 09:02:55 [INFO] [process.task.ProjectVICHashLookup] Number of ProjectVic md5 hashes loaded: 12820210
2021-12-24 09:02:55 [INFO] [process.task.ProjectVICHashLookup] Number of ProjectVic sha1 hashes loaded: 11564135
2021-12-24 09:02:55 [INFO] [process.task.ProjectVICHashLookup] Number of ProjectVic photoDNA hashes loaded: 0
2021-12-24 09:02:55 [INFO] [process.task.ProjectVICHashLookup] Number of ProjectVic series loaded: 8080
2021-12-24 09:02:55 [INFO] [indexer.process.Worker] Starting DuplicateTask
2021-12-24 09:02:55 [MSG] [indexer.process.ProgressConsole] Inicializando DuplicateTask
2021-12-24 09:02:55 [INFO] [indexer.process.Worker] Starting AudioTranscriptTask
2021-12-24 09:02:55 [MSG] [indexer.process.ProgressConsole] Inicializando AudioTranscriptTask
2021-12-24 09:02:55 [INFO] [indexer.process.Worker] Starting VideoThumbTask
2021-12-24 09:02:55 [MSG] [indexer.process.ProgressConsole] Inicializando VideoThumbTask
2021-12-24 09:02:55 [INFO] [process.task.VideoThumbTask] Task enabled.
2021-12-24 09:02:55 [INFO] [process.task.VideoThumbTask] MPLAYER version: MPlayer 1.3.0 (Debian), built with gcc-9 (C) 2000-2016 MPlayer Team
2021-12-24 09:02:55 [INFO] [indexer.process.Worker] Starting ParsingTask
2021-12-24 09:02:55 [MSG] [indexer.process.ProgressConsole] Inicializando ParsingTask
2021-12-24 09:02:55 [INFO] [indexer.process.Worker] Starting RegexTask
2021-12-24 09:02:55 [MSG] [indexer.process.ProgressConsole] Inicializando RegexTask
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.CPFRegexValidatorService found for CPF
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.CNPJRegexValidatorService found for CNPJ
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.swift.SwiftCodeRegexValidatorService found for SWIFT
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.CNHValidatorService found for CNH
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.PisPasepRegexValidatorService found for PISPASEP
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.BoletoRegexValidatorService found for BOLETO
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.TituloEleitorRegexValidatorService found for TITULO_ELEITOR
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.CreditCardRegexValidatorService found for CREDIT_CARD
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.ScriptValidatorService found for CRIPTO_POSSIBLE_SEED_PHRASE_EN
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.ScriptValidatorService found for CRIPTO_POSSIBLE_SEED_PHRASE_PT
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.ScriptValidatorService found for EXAMPLE
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.BitcoinAddressValidatorService found for CRIPTOCOIN_BITCOIN_ADDRESS
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.BitcoinAddressValidatorService found for CRIPTOCOIN_BITCOIN_BIP38_ENC_PRIV_K
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.BitcoinAddressValidatorService found for CRIPTOCOIN_BITCOIN_WIF_PRIV_K_UNC_PUB_K
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.BitcoinAddressValidatorService found for CRIPTOCOIN_BITCOIN_WIF_PRIV_K_COMP_PUB_K
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.BitcoinAddressValidatorService found for CRIPTOCOIN_BITCOIN_BIP32_HD_XPRV_KEY
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.BitcoinAddressValidatorService found for CRIPTOCOIN_BITCOIN_BIP32_HD_XPUB_KEY
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.BCHAddressValidatorService found for CRIPTOCOIN_BITCOIN_CASH_CASHADDR
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.EthereumAddressValidatorService found for CRIPTOCOIN_ETHEREUM
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.RippleAddressValidatorService found for CRIPTOCOIN_RIPPLE
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.TelefoneRegexValidatorService found for PHONE
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.TelefoneRegexValidatorService found for TELEFONE
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.PlacaRegexValidatorService found for CAR
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.PlacaRegexValidatorService found for PLACA
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.IPLRegexValidatorService found for IPL
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.FacebookContactValidatorService found for FACEBOOK
2021-12-24 09:03:16 [INFO] [task.regex.RegexValidator] Validator br.gov.pf.iped.regex.RIFRegexValidatorService found for RIF
2021-12-24 09:03:16 [INFO] [indexer.process.Worker] Starting LanguageDetectTask
2021-12-24 09:03:16 [MSG] [indexer.process.ProgressConsole] Inicializando LanguageDetectTask
2021-12-24 09:03:17 [INFO] [indexer.process.Worker] Starting NamedEntityTask
2021-12-24 09:03:17 [MSG] [indexer.process.ProgressConsole] Inicializando NamedEntityTask
2021-12-24 09:03:17 [INFO] [indexer.process.Worker] Starting ExportFileTask
2021-12-24 09:03:17 [MSG] [indexer.process.ProgressConsole] Inicializando ExportFileTask
2021-12-24 09:03:17 [INFO] [indexer.process.Worker] Starting MakePreviewTask
2021-12-24 09:03:17 [MSG] [indexer.process.ProgressConsole] Inicializando MakePreviewTask
2021-12-24 09:03:17 [INFO] [indexer.process.Worker] Starting DocThumbTask
2021-12-24 09:03:17 [MSG] [indexer.process.ProgressConsole] Inicializando DocThumbTask
2021-12-24 09:03:17 [INFO] [process.task.DocThumbTask] Task disabled
2021-12-24 09:03:17 [INFO] [indexer.process.Worker] Starting ImageThumbTask
2021-12-24 09:03:17 [MSG] [indexer.process.ProgressConsole] Inicializando ImageThumbTask
2021-12-24 09:03:17 [INFO] [process.task.ImageThumbTask] Thumb Size: 160
2021-12-24 09:03:17 [INFO] [process.task.ImageThumbTask] Extract Thumb: true
2021-12-24 09:03:17 [INFO] [indexer.process.Worker] Starting DIETask
2021-12-24 09:03:17 [MSG] [indexer.process.ProgressConsole] Inicializando DIETask
2021-12-24 09:03:17 [ERROR] [gpinf.indexer.IndexFiles] Processing Error: Invalid DIE database file: /root/IPED/iped/mnt/led/die/rfdie.dat

ERROR!!!
Check the log at /root/IPED/iped-3.18.12/log/IPED-2021-12-24-09-00-35.log
wladimirleite commented 2 years ago

I ran a quick test here using an absolute path and it worked fine. Looking at the code (belq), I saw that it checks if the model file exists in the given path. If it is not there, only then it tries to add the "base" path as a suffix.

File dieDat = new File(diePath.trim()); // Here diePath is the one that came from the configuration file
if (!dieDat.exists()) // If not found
    dieDat = new File(new File(Configuration.getInstance().appRoot), diePath.trim()); // then add the "base" path

Can you double check if the absolute path you used is correct (and accessible to IPED's process)?

wladimirleite commented 2 years ago

By the way, I used 3.18.12 here too, but on Windows 10, although it doesn't seem to be an OS specific issue.

lfcnassif commented 2 years ago

I'm also not able to reproduce this with Ubuntu-20.04, openjdk-1.8.0_292 and iped-3.18.12. I tried to execute from iped-3.18.12 root folder and from an unrelated folder, setting ledDie = /media/DRIVE_F/LED/V1.27.00/die/rfdie.dat

@arisjr if you think this isn't a permission issue like @tc-wleite said and can provide a detailed step by step to reproduce this, please reopen.

arisjr commented 2 years ago

Sorry, guys, I was out on vacation... Just saw now, the path was wrong indeed. The correct path was /mnt/led/pedo/die/rfdie.dat.

Thanks and regards, Aristeu