Requirements on python files? -- failing to graph

[sk3499]

Hello, I'm having issues with pyan. Specifically, when I try to graph my python files, I get blank graph. I tested pyan with a simple test.py file and it works fine, but my large program shows nothing. Is there specific programming style it expects? I'm not sure if this is a style issue, or an issue with I have an associated library it calls ? or something else. The only thing I can think of is I have a weird mix of global variables and calls to classes in my library file.

When I put both the library and my main file on the command line calling pyan , it graphs only the library file. Does that seem right?

I can't post the code it is private.

I have python 3.8 on windows 64 bit.

[Technologicat]

Hi,

No, there's no specific style Pyan expects. It actually converts the source code to an AST using ast.parse and analyzes that, so if Python itself can understand the program, then Pyan should be able to understand most of it.

Python is a large language, though, and some features are not supported. See the README for a list, under the heading "The analyzer does not currently support". It should be complete, but hasn't been updated lately, so features new in 3.7 or 3.8 may be missing from it.

If you put all the Python files on the command line, Pyan should analyze and graph all of them, including any references between them. It sounds like Pyan is not reading your main program correctly for some reason.

Could you try modvis.py -cag --dot (here) on your files to see if it can generate a graph of how your modules depend on each other? If it works while Pyan itself produces a blank graph, then it could be a problem in Pyan's import analyzer.

Or try Pyan with the --very-verbose command line flag and look at the output, it might help see what's going on?

(For debugging, you can omit the --dot flag for Pyan. The visual graph generation is a separate postprocessing step, and this sounds like something is going wrong already in the analysis.)

[sk3499]

For modvis , it outputs:

digraph G {
    graph [rankdir=TB, clusterrank="local"];
    subgraph cluster_G {

        graph [style="filled,rounded",fillcolor="#80808018", label=""];
    }
    }

---

For verbose mode, it outputs:

========== pass 1, file 'MYFILE.py' ========== Scopes now: {'MYFILE': } Module Resolving base classes All base classes (non-recursive, local level only): {} Resolving method resolution order (MRO) for all analyzed classes Method resolution order (MRO) for all analyzed classes: {} ========== pass 2, file 'MYFILE.py' ========== Scopes now: {'MYFILE': } Module <class 'pyan.writers.DotWriter'> running Start subgraph Finish subgraph

[Technologicat]

Thanks for testing. So, both tools produce an empty graph. It's probably not the import analyzer, then.

(I forgot to mention, modvis.py also supports --very-verbose, which should tell about the import statements it sees. Likewise, omitting --dot is possible, modvis.py too generates the graph in a separate postprocessing step. In your case, there should probably be an import in MYFILE.py that loads your library.)

Pyan is behaving as if it didn't see any of the code in MYFILE.py. There should be quite many log messages printed, at least one for each function or class encountered. The lack of any scopes other than the module top level also suggests it's not seeing any class/function definitions for some reason.

Which Pyan version are you running? Is it from PyPI, or directly from this GitHub repo? There are a few small fixes that haven't yet been packaged (nor pushed to davidfraser's stable repo), so one possible thing to try is the latest Pyan from git, unless you tried it already.

Other than that... I'm guessing here, but maybe language version compatibility? Do you have a virtualenv (or Anaconda, or equivalent) where you could easily run Pyan in Python 3.6?

(Your own program doesn't need to run in 3.6. Pyan running in 3.6 can analyze the code as long as it doesn't use any new syntax that only exists in Python 3.7 or later. Off the top of my head, I can't recall any new syntax in 3.7 or 3.8 - there was an internal AST representation change in 3.8 (the compiler now produces ast.Constant nodes instead of ast.Num and ast.Str for constants), but the surface syntax, i.e. what we actually write in .py files, should be the same from 3.6 to 3.8.)

If, like me, you happen to be new to the virtualenv stuff... coming from a science background, here are the commands for Anaconda, see points 3-6. Last I looked, the Python ecosystem provided a bewildering variety of virtualenv options, each slightly different, and I'm not sure if the dust has settled yet, so to speak. If you are already familiar with another virtualenv system, just use that - it shouldn't matter.

Testing this just needs any virtualenv with Python 3.6. Then get the latest Pyan from this GitHub repo, python3 setup.py install it in the virtualenv, and try analyzing your code with it.

[Technologicat]

Ah, one more thing: character encoding of your .py files? Pyan currently assumes utf-8.

[sk3499]

very verbose didnt change anything on modvis.

I'm using the IDLE editor on windows to write and run the script, so whatever is the default there.

I have no idea why, but I put the most recent version of my code in, and tried again and it seems to work now. Very strange.

[Technologicat]

I have no idea of Windows defaults, that's why I asked. :)

(If you ask me, nowadays everyone should be using utf-8. Unicode is the single huge codepage for the 21st century, while utf-8 is its sanest encoding. But its rates of adoption have historically varied between different OSs and software.)

Glad that you got it working!