search

Fraunhofer-AISEC / cpg

A library to extract Code Property Graphs from C/C++, Java, Go, Python, Ruby and every other language through LLVM-IR.
https://fraunhofer-aisec.github.io/cpg/
Apache License 2.0
288 stars 62 forks source link

Python: Todo list #394

Open maximiliankaul opened 3 years ago

maximiliankaul commented 3 years ago

The experimental python branch has been merged to master. It currently supports some python language features but is far from complete. This issue is supposed to track open todos for the python implementation.

### Tasks
- [x] #402
- [x] Publish to pip (to be discussed) -> not relevant with the Kotlin rewrite
- [x] Split implementation into multiple files
- [x] #1653
- [x] #1640
- [ ]  #1604
- [ ]  #1654
- [ ] Implement support for multiple return values (see for example: https://github.com/Fraunhofer-AISEC/cpg/blob/06995d44afb21212c72fe18320324d76d562575a/cpg-language-python/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/python/ExpressionHandler.kt#L175 or related https://github.com/Fraunhofer-AISEC/cpg/blob/06995d44afb21212c72fe18320324d76d562575a/cpg-language-python/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/python/ExpressionHandler.kt#L246)
- [ ] Field initializes
- [ ] #1649
- [ ] slices (DFG)
- [ ] constant propagation ((multi) value evaluator)
- [ ] #1651
- [ ] #1652
- [ ] handle `fromat_spec` for `FormattedValues` https://github.com/Fraunhofer-AISEC/cpg/blob/06995d44afb21212c72fe18320324d76d562575a/cpg-language-python/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/python/ExpressionHandler.kt#L89
- [ ] Consistency: use `private fun` or `internal fun` consistently.
- [ ] #1677
- [ ] #1702
- [ ] Add declaration pass: traverse EOG instead of AST
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1843
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1841
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1835

Several nodes are unhandled in the Python frontend. This Issue is here to track whether we can and have implemented the parser node and what the current Ideas as on how we map these nodes to the current graph(existing nodes or a new node):

### Tasks ([StatementHandler](https://github.com/Fraunhofer-AISEC/cpg/blob/ab6b5c80666cae6b79f73ada522643fe5b6df1eb/cpg-language-python/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/python/StatementHandler.kt#L69-L77))
- [ ] #1709
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1712
- [ ] Match
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1713
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1710
- [ ] #1704
- [ ] TryStar
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1711
- [x] AsyncWith
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1718
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1714
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1715
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1716
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1717
- [ ] Await
- [ ] https://github.com/Fraunhofer-AISEC/cpg/issues/1719
- [ ] YieldFrom
KuechA commented 4 weeks ago

Some notes concerning the above tasks:

Add declaration pass: traverse EOG instead of AST

This doesn't work because the Pass dependencies don't allow it.

constant propagation ((multi) value evaluator)

What is expected here? Basic concepts are available (i.e., +, -, etc.) If we need some specific information in a special scenario, we should probably extract this to another evaluator

Field initializes

What should we do here? This needs a more precise description

Slices (DFG)

Task description too imprecise

Match

Basic support/implementation in #1801 . Many of the cases are missing.