scottrogowski / code2flow

Pretty good call graphs for dynamic languages
MIT License
3.98k stars 295 forks source link

return compile SyntaxError: invalid character #69

Open reetss opened 2 years ago

reetss commented 2 years ago

Hi

I have a problem with generating graphs. I have a different .py files an some give me an error:

code

output

Anyone has an idea?

scottrogowski commented 2 years ago

That's interesting. It looks like you have an issue decoding a file with a special character. Code2flow theoretically supports this. It looks like you might also be on Windows? My guess is that your file is encoded in something besides utf-8 and there is some issue decoding. Maybe try this and let me know what it's encoded with? https://stackoverflow.com/questions/3710374/get-encoding-of-a-file-in-windows

reetss commented 2 years ago

Hi @scottrogowski
thanks for the fast reply! It looks like they were all utf-8 image

scottrogowski commented 2 years ago

Interesting. Any chance you can send over the source of one of the bad files so I can reproduce?

On Mon, Sep 19, 2022, 11:30 reetss @.***> wrote:

Hi @scottrogowski https://github.com/scottrogowski thanks for the fast reply! It looks like they were all utf-8 [image: image] https://user-images.githubusercontent.com/101587483/191077730-658ca0b9-d667-4d66-ac27-027978c9a337.png

— Reply to this email directly, view it on GitHub https://github.com/scottrogowski/code2flow/issues/69#issuecomment-1251326132, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADR6O723QWKFPCAAKKCT4TV7CPJVANCNFSM6AAAAAAQQDBF2I . You are receiving this because you were mentioned.Message ID: @.***>

reetss commented 2 years ago

It's within the stumpty library from TD Ameritrade: https://github.com/TDAmeritrade/stumpy/tree/main/stumpy It works for some of them. But I am getting the same error for most of the files.

scottrogowski commented 2 years ago

Sounds good. Might take a few days to get back to you. Thanks for reporting the issue

On Mon, Sep 19, 2022 at 2:33 PM reetss @.***> wrote:

It's within the stumpty library from TD Ameritrade: https://github.com/TDAmeritrade/stumpy/tree/main/stumpy It works for some of them. But I am getting the same error for most of the files.

— Reply to this email directly, view it on GitHub https://github.com/scottrogowski/code2flow/issues/69#issuecomment-1251519629, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADR6OYSTBFR2Y3H4XJVIYLV7DE3BANCNFSM6AAAAAAQQDBF2I . You are receiving this because you were mentioned.Message ID: @.***>

reetss commented 2 years ago

Thank you, for looking after it!

scottrogowski commented 2 years ago

Sorry to take a few weeks. Seems to work fine for me with that library including in the way you're calling it via API. Given where it is failing, my last thought is that maybe you are running it on python 2 instead of python 3? You can check that with python --version. Otherwise, I would need access to a Windows machine to debug which I don't have :/.

code2flow.code2flow('/Users/scoot/Downloads/stumpy-main/stumpy/gpu_stump.py', 'out.json')
Code2Flow: Found 1 files from sources argument.
Code2Flow: Implicitly detected language as 'py'.
Code2Flow: Processing 1 source file(s).
Code2Flow:   /Users/scoot/Downloads/stumpy-main/stumpy/gpu_stump.py
Code2Flow: Found groups ['File: gpu_stump'].
Code2Flow: Found nodes ['(global)', '_compute_and_update_PI_kernel', '_gpu_stump', 'gpu_stump'].
Code2Flow: Found calls ['QT_first_fnames.append()', 'QT_fnames.append()', 'UNKNOWN_VAR.clear()', 'ValueError()', '_gpu_stump()', 'abs()', 'core._check_P()', 'core._get_QT()', 'core.are_arrays_equal()', 'core.array_to_temp_file()', 'core.check_window_size()', 'core.preprocess()', 'cuda.current_context()', 'cuda.grid()', 'cuda.gridsize()', 'cuda.to_device()', 'device_indices.copy_to_host()', 'device_profile.copy_to_host()', 'enumerate()', 'int()', 'isinstance()', 'len()', 'logger.warning()', 'logging.getLogger()', 'math.ceil()', 'math.fabs()', 'math.isinf()', 'max()', 'min()', 'mp.Pool()', 'mp.set_start_method()', 'np.ceil()', 'np.empty()', 'np.full()', 'np.load()', 'np.sqrt()', 'np.where()', 'os.remove()', 'pool.apply_async()', 'pool.close()', 'pool.join()', 'range()', 'result.get()'].
Code2Flow: Found variables ['M_T-><Call owner_token=np token=load>', 'M_T_fname-><Call owner_token=core token=array_to_temp_file>', 'QT-><Call owner_token=np token=load>', 'QT_first-><Call owner_token=np token=load>', 'QT_first_fname-><Call owner_token=core token=array_to_temp_file>', 'QT_fname-><Call owner_token=core token=array_to_temp_file>', 'T_A-><Call owner_token=np token=load>', 'T_A_fname-><Call owner_token=core token=array_to_temp_file>', 'T_B-><Call owner_token=np token=load>', 'T_B_fname-><Call owner_token=core token=array_to_temp_file>', 'blocks_per_grid-><Call owner_token=math token=ceil>', 'config->UNKNOWN_MODULE', 'core->UNKNOWN_MODULE', 'cuda->UNKNOWN_MODULE', 'device_M_T-><Call owner_token=cuda token=to_device>', 'device_QT_even-><Call owner_token=cuda token=to_device>', 'device_QT_first-><Call owner_token=cuda token=to_device>', 'device_QT_odd-><Call owner_token=cuda token=to_device>', 'device_T_A-><Call owner_token=cuda token=to_device>', 'device_T_B-><Call owner_token=cuda token=to_device>', 'device_indices-><Call owner_token=cuda token=to_device>', 'device_profile-><Call owner_token=cuda token=to_device>', 'device_Σ_T-><Call owner_token=cuda token=to_device>', 'device_μ_Q-><Call owner_token=cuda token=to_device>', 'device_σ_Q-><Call owner_token=cuda token=to_device>', 'excl_zone-><Call owner_token=None token=int>', 'gpu_aamp->UNKNOWN_MODULE', 'indices-><Call owner_token=device_indices token=copy_to_host>', 'indices-><Call owner_token=np token=full>', 'indices_fname-><Call owner_token=core token=array_to_temp_file>', 'logger-><Call owner_token=logging token=getLogger>', 'logging->UNKNOWN_MODULE', 'math->UNKNOWN_MODULE', 'mp->UNKNOWN_MODULE', 'np->UNKNOWN_MODULE', 'os->UNKNOWN_MODULE', 'out-><Call owner_token=np token=empty>', 'p_norm-><Call owner_token=None token=abs>', 'pool-><Call owner_token=mp token=Pool>', 'profile-><Call owner_token=device_profile token=copy_to_host>', 'profile-><Call owner_token=np token=full>', 'profile-><Call owner_token=np token=sqrt>', 'profile_fname-><Call owner_token=core token=array_to_temp_file>', 'start-><Call owner_token=cuda token=grid>', 'stop-><Call owner_token=None token=min>', 'stride-><Call owner_token=cuda token=gridsize>', 'zone_start-><Call owner_token=None token=max>', 'zone_stop-><Call owner_token=None token=min>', 'Σ_T-><Call owner_token=np token=load>', 'Σ_T_fname-><Call owner_token=core token=array_to_temp_file>', 'μ_Q-><Call owner_token=np token=load>', 'μ_Q_fname-><Call owner_token=core token=array_to_temp_file>', 'σ_Q-><Call owner_token=np token=load>', 'σ_Q_fname-><Call owner_token=core token=array_to_temp_file>'].
Code2Flow: Generating output file...
Code2Flow: Wrote output file 'out.json' with 2 nodes and 1 edges.
Code2Flow: Code2flow finished processing in 0.02 seconds.