Resolve "Upload examples of GSP" - [merged]

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 02:16

Merges 12-upload-examples-of-gsp -> master

Closes #12

There are 4 examples in the example folder. They use a function in the examples/utils.py file to make the graph in networkx of the Santiago de Chile metro network. The 4 examples start with the name metro_ followed by a descriptive name.

metro_simulation2.py is too complex and I am pretty sure it would needs a thorough revision. It's fun and the results are interesting but we may want to consider leaving it out this round and just include the other metro_simulation example which just simulates a signal spreading over the network.

The dependencies have been updated to include some extra toobloxes in the dev mode.

gitignore file also has been updated to not include results from the examples.

Added edges.txt to the examples folder.

Let's see how it goes.. :robot:

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 02:16

requested review from @aweinstein

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

Commented on examples/metro_graph_signal.py line 3

Give the exact link to the viajes_bajada.xlsb file. I see many links on that page but can't find one.

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

Commented on examples/metro_graph_signal.py line 19

Add try...except clause to catch the exception thrown when the file is missing. Print a message with the location of the file.

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

Commented on examples/metro_regression.py line 24

Fix docstring format.

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

Commented on examples/metro_regression.py line 23

This function seems quite general. Maybe it is better to add it to PyGSP2 as one of the visualization functions. Along the same lines, isn't already a function to visualize a graph signal over a NetworkX graph?

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

Commented on examples/metro_regression.py line 55

Add try...except.

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

Commented on examples/metro_graph_signal.py line 52

The data file and graph preprocessing repeats among the examples. Create a single function for this that is common to all the examples and place it in utils.py.

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

Commented on examples/metro_regression.py line 66

Rewrite using W.sum(axis=1) and np.diag. I am surprised that NetworkX doesn't have a function to extract the degree matrix. If getting the NetworkX graph degree matrix and its inverse is common, consider adding it to the main code.

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

Commented on examples/metro_simulation2.py line 63

Use a temporary directory (using https://docs.python.org/3/library/tempfile.html) instead of this directory.

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 06:28

To be continued...

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:06

Commented on examples/metro_graph_signal.py line 3

I had changed the filename in my local directory. Now it's the default filename that comes when downloading "Tablas de Subidas y Bajadas" this will dowload 2 xlsb files, one of subidas another of bajadas. We use only the bajadas.

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:06

Commented on examples/metro_graph_signal.py line 3

changed this line in version 2 of the diff

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:06

Commented on examples/metro_graph_signal.py line 19

changed this line in version 2 of the diff

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:06

added 1 commit

• b6679185 - change docstring and filename

Compare with previous version

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:30

Commented on examples/metro_graph_signal.py line 19

Added the try, except clause. In the except I added sys.exit(-1) to terminate the command. If I run the file on a terminal this will just throw the messages I added to be printed when entering the exception:

Data file was not found in:
 C:\Users\J-RC\OneDrive - Universidad de Chile\Documentos\Projects\FONDECYT_alejandro\desarrollo_toolbox\pygsp
Download it from:
https://www.dtpm.cl/descargas/modelos_y_matrices/Tablas%20de%20subidas%20y%20bajadas%20nov23.zip

But, oddly, if I run the individual cell in my Ipython kernel it throws and AttributeError exception. This only happens due to the last sys.exit(-1) line. Its independent of the argument that goes into the sys.exit function. Adding AttributeError to the list of exceptions doesn't solve the problem.

Here is the error

FileNotFoundError                         Traceback (most recent call last)
File c:\Users\J-RC\OneDrive - Universidad de Chile\Documentos\Projects\FONDECYT_alejandro\desarrollo_toolbox\pygsp\examples\metro_graph_signal.py:14
     13 try:
---> 14     commutes = pd.read_excel('2023.11 Matriz_baj_SS_MH.xlsb',
     15                              sheet_name='bajadas_prom_laboral')
     16 except FileNotFoundError:

File c:\ProgramData\anaconda3\envs\mne\Lib\site-packages\pandas\io\excel\_base.py:495, in read_excel(io, sheet_name, header, names, index_col, usecols, dtype, engine, converters, true_values, false_values, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, parse_dates, date_parser, date_format, thousands, decimal, comment, skipfooter, storage_options, dtype_backend, engine_kwargs)
    494     should_close = True
--> 495     io = ExcelFile(
    496         io,
    497         storage_options=storage_options,
    498         engine=engine,
    499         engine_kwargs=engine_kwargs,
    500     )
    501 elif engine and engine != io.engine:

File c:\ProgramData\anaconda3\envs\mne\Lib\site-packages\pandas\io\excel\_base.py:1550, in ExcelFile.__init__(self, path_or_buffer, engine, storage_options, engine_kwargs)
   1549 else:
-> 1550     ext = inspect_excel_format(
   1551         content_or_path=path_or_buffer, storage_options=storage_options
   1552     )
   1553     if ext is None:
...
-> 1150         mod = inspect.getmodule(cf.tb_frame)
   1151         if mod is not None:
   1152             mod_name = mod.__name__

AttributeError: 'tuple' object has no attribute 'tb_frame'

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:33

added 1 commit

• f9eb6d3f - add try - except clause

Compare with previous version

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:33

Commented on examples/metro_graph_signal.py line 19

Apparently no negative exit codes should be used.

https://docs.python.org/2/library/sys.html#sys.exit

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:46

Commented on examples/metro_regression.py line 23

I have not been able to find a graph signal plotting function from NetworkX. Pygsp does have one included, it is not as pretty though and not as flexible, I don't see options for transparency and colormap. For this first version I would put it in utils since idealy, the plotting function from pygsp should behave in the way we wanted to.

Pygsp Graph.plot function:

undefined

Made up for the example function: undefined

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 11:57

Commented on examples/metro_regression.py line 24

Fixed

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 13:33

Commented on examples/metro_regression.py line 24

changed this line in version 4 of the diff

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 13:33

Commented on examples/metro_regression.py line 23

changed this line in version 4 of the diff

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 13:33

Commented on examples/metro_regression.py line 55

changed this line in version 4 of the diff

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 13:33

Commented on examples/metro_regression.py line 66

changed this line in version 4 of the diff

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 13:33

added 1 commit

• d6bfd0a5 - added header specification. Added functions to utils. Fixed docstring.

Compare with previous version

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 15:48

added 1 commit

• 420d81b8 - Delete make-anmation option

Compare with previous version

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 15:53

added 1 commit

• 99b693b3 - Deleted one condition for break. Now simulations can be performed with other initial conditions

Compare with previous version

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 16:11

added 1 commit

• 9c00e56b - Improve the file docstring.

Compare with previous version

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 16:27

Commented on examples/metro_regression.py line 29

Add pyxlsb as a dependency.

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 16:29

Commented on examples/metro_graph_signal.py line 15

We also need to tell people to download 'metroCoords.geojson' (or add it to the repository).

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 16:38

added 1 commit

• 1dce856a - Fix import of utils.

Compare with previous version

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 16:47

added 2 commits

• 503204d6 - Change imports
• 5eab73a3 - Change directory to where the file is when a database is loaded.

Compare with previous version

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 18:13

added 1 commit

• 712d9b4d - Update the script docstring.

Compare with previous version

[lucaslanek]

In GitLab by @aweinstein on Jun 12, 2024, 18:23

Commented on examples/metro_regression.py line 23

Can we add an issue then to improve the graph signal function?

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 18:28

Commented on examples/metro_regression.py line 23

Done

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 18:36

Commented on examples/metro_regression.py line 29

Added dependency

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 18:37

added 1 commit

• 76bdaa92 - Add pyxlsb as extra dependency

Compare with previous version

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 18:51

added 1 commit

• 7c1727a6 - add geojson requirement in docstring of the examples

Compare with previous version

[lucaslanek]

In GitLab by @julioRodino on Jun 12, 2024, 18:51

Commented on examples/metro_graph_signal.py line 15

Done

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 05:29

Commented on examples/metro_graph_signal.py line 15

I think we need to provide a link to download the file, not the instructions to create it. It is easy to make a mistake when creating this file since you need to have the city of Santiago exactly on the map. I suggest uploading the file to Zenodo. Then add this link to the docstring.

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 05:35

Commented on examples/utils.py line 141

Fix the docstring.

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 05:36

Commented on examples/utils.py line 99

Fix the docstring.

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 05:37

Commented on examples/utils.py line 34

Fix the docstring.

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 06:05

Commented on examples/utils.py line 114

The label doesn't appear in the figure. Also, I think the label should be the title of the figure.

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 06:50

added 1 commit

• 696e5377 - Simplify and fix colorbar.

Compare with previous version

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 07:02

Commented on examples/utils.py line 114

changed this line in version 14 of the diff

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 07:02

added 1 commit

• b29452c5 - Add parameter title to plot function.

Compare with previous version

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 08:55

Commented on examples/metro_graph_signal.py line 15

I just created the repository in zenodo: https://zenodo.org/records/11637462

I'll fix this during the day.

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 11:33

Commented on examples/utils.py line 34

changed this line in version 15 of the diff

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 11:33

added 1 commit

• 9f4dfaaf - Use Santiago Metro datafiles from Zenodo.

Compare with previous version

[lucaslanek]

In GitLab by @aweinstein on Jun 13, 2024, 11:35

Commented on examples/metro_graph_signal.py line 15

Fixed.