Closed Softdev1 closed 9 months ago
\ We are working on creating a fine tuned model on open source code LLMs to detect vulnerabilities in smart contracts. but we are looking to study if there is a possibility we can enhance the LLM output with static analysis tools which are already good at detecting some vulnerabilities.
The idea is LLMs are good at finding logical errors from code comments and how code is written, but there is no guarantee that it always provides correct results. whereas static analysis tools always provide deterministic errors in code, but miss out on logical errors. So we are looking for benefits we can achieve if we combine the both.
We can utilize these to generate datasets - we did this with slither analisis dataset and enhanced the output with LLMs
Use it in application level along with the LLM - here we can first let the static analysis run. from the output, we can in theory parse the line numbers and gather the code along with issue and ask LLM to fix the issue. this should scale to codebases since we are not inputing the whole codebase to LLM instead the issues found by static tools - also should be faster
\ Mythx is not free, unlike the other analysed tools, and it has three different analysis modes: quick, standard and deep. In quick mode, it detected integer overflows and underflows, bad randomness, out of bounds array access and unprotected ether withdrawal but failed to detect transaction order dependency and outdated compiler version. In summary, Mythx was able to detect the 3 more obvious and dangerous vulnerabilities of the contracts that could lead to exploitation. MythX's output is colour-coded by severity and includes SWC ID to link the vulnerabilities detected to the SWC Registry. \
Mythril detected integer overflows and underflows, bad randomness, out of bounds array access and unprotected ether withdrawal, but failed to detect transaction order dependency, outdated compiler version and also failed to warn about solidity coding best practices: deprecated functions, state variables default visibility and floating pragma like MythX. In summary, Mythril was able to detect the 3 more obvious and dangerous vulnerabilities of the contracts that could lead to exploitation.In Mythril output, findings are very detailed, includes SWC ID to link the vulnerabilities detected to the SWC Registry and even includes example transactions and transaction sequences that can trigger the vulnerabilities found. \
Slither does not have detectors for integer overflow and underflow and therefore did not detect these vulnerabilities, but it was able to detect out of bounds array access (as SWC-109), missing input validations, bad randomness and timestamp dependence. In summary, Slither was able to detect the 2 of the 3 more obvious and dangerous vulnerabilities of the contracts that could lead to exploitation. Slither's output is colour-coded by severity, with high-severity detectors printed first,followed by the medium-severity and low-severity outputs and includes references for more information about the vulnerability. In addition, it includes some solidity coding best practices warnings that are useful but do not represent vulnerabilities. We can also use https://github.com/pessimistic-io/slitherin for more detectors on top.
Securify2 This did not work for me, it uses docker image to run but i am getting errors on build command.
Description
We need to utilize the capabilities of static analysis tools along with LLMs to be more deterministic about iq code detection. explore the possibility of integrating such deterministic approches to detection along with LLMs for iq code
Goal
Research on deterministic vulnerability analysis along with LLMs