Doing awesome software analysis

[sils]

Here's something cool and awesome: we can do stuff like liveness analysis, very busy expression analysis etc. to automatically propose really really smart corrections.

That would give us a huge advantage, this is some research stuff that I'm currently learning for an exam and I think we could implement that possibly with a decent accuracy using coalang.

CC @abhsag24 and @SanketDG : to make this happen we would need to be able to extract statements, get all variables used in an expression, understand assignments and we would probably need to understand ifs and whiles. That's a lot but I feel like we could potentially still make this happen. Maybe we could partner up with BNFC and make like a simplified mini grammar for languages we support... not fully sure. We should also take a look on how CPD does it.

[gitmate-bot]

Thanks for reporting this issue! A coalaian will look at it soon.

[Makman2]

then we should definitely evolve some coAST prototype. Though it may not match the full purpose of coAST yet, we should have it there :)

[sils]

step by step

[SanketDG]

I was not aware of the terms you mentioned, but after knowing about them, I am still not sure I understand what they mean in practical code. Some examples could help.

(PS: I just got on a new laptop, so I won't be active much for the next couple of days 👍 )

[sils]

Essentially it's about some analysis that is taught in theoretical computer science but nobody uses them in practice because there's no proper implementation. You usually need to know where a variable is defined and where it is used. From that you can already derive lots of stuff! E.g.:

• If a computation inside a loop could be moved out (performance)
• Unused code of course
• Find different variables that are assigned to the same expressions
• Suggest code simplification (e.g. an expression that is calculated in all branches of an if can be calculated before)
• Find code clones (we already have the algorithm! :P)
• Loads of more stuff!

[jayvdb]

pyflakes already has the core of this for python. It builds more metadata than it uses, because it only reports real bugs. It could report/fix much more. I'd be very happy to help with integration if you are intetested. Note that this problem domain is complex in python (and similar) due to inner funcs being accessed outside their outer func and del being valid anywhere. But that is an edge case, and could be ignored by adding a flag to the name to indicate that it can be reached by a del and thus its lifetime is not predicted.

[sils]

@jayvdb might be interesting to see how they're doing it, the algorithms are quite straightforward though. I was hoping we could make something that would work for more than just python, in a way that would allow us to supply a language definition of some kind to add support for new languages.

Our vision is that we stop rewriting such algorithms for every language.

[abhay-raizada]

why not start work directly on coAST? a generic AST builder provided specifications for the language we can have various depenency Bears like the AnnotationBear, like say VariablesBear which gets all the variables given their declarations are specified in the coalang? and we can use all such Bears to build an AST?

[sils]

because coAST is not something you can built easily. We have to approach it step by step IMO. This goes pretty close and it is a doable step. We can look further when we're there.

[jayvdb]

Do you have a task, repo or wiki about coAST? Im guessing https://github.com/PyCQA/astroid/issues/329#issuecomment-211251704 is relevant.

[sils]

There's a pretty much unmaintained repo with some ideas lying around at https://github.com/coala-analyzer/CoAST

[IpshitaC]

Could I take this up as a part of RGSoC @Udayan12167 ?

[Makman2]

@IpshitaC coAST is a bit out of scope for a gsoc project, especially if you aren't that familiar with coala's bear system :)

[IpshitaC]

Oh alright! Any suggestions as to what would be suitable for an RGSoC project on improving generic bear quality?

[Makman2]

@IpshitaC have you looked at our gsoc projects site? Maybe you find something that interests you regarding this topic :)

[Makman2]

Or you get a new idea by reading them ;)