gmingas
commented
4 years ago We had a very useful meeting yesterday with Andrew Elliott and Mihai Cucuringu as a follow up from the FCA workshop.
We discussed graph/network data in general and also more specifically about synthesising those. Key points:
- Applications: There is a lot of interest in synthesising network data especially in sectors like finance and telecommunications. Synthesis is useful for anomaly detection, fraud detection, money flow analysis, testing algorithms, sharing data with partners, etc.
- Synthesis algorithms: CTGAN is one option but mainly if your edges are relatively independent. If you have locally dense networks it starts to fail. Other options include RNNs, other GAN-based models, Exponentially Random graph Models (ERGM). Data can be represented in non-tabular formats.
- Data formats: The most common are tabular edge lists (e.g. in .csv). Matrices are also used. Formats like GML, GraphML (XML-based) are also widely used. Python networkx is popular and supports many formats
- Utility metrics: There are standard metrics of similarity between graphs that we can use. A combination of many of them should give a reasonably good metric or in some applications particular ones are of more importance:
- Nbr. of nodes, nbr of edges and ratio between the two
- Density (what % of possible connections exist)
- Degree of distribution (distribution of nbr. of connections for each node)
- Clustering measures (nbr. of triangles)
- Connected components (size of larger connection)
- Assortativity
- Edge direction
- Self loops
- Weight distributions
- Privacy: Not clear what methods can be used here but there is some literature on differential privacy. Andrew could send a list of papers.
- Network analysis crash course: link
- Other links: FNA platform
- NetworkComparison library
We can now process graph data with CTGAN but the synthesis model is not very advanced (see PR).
We want to:
See Derek's notebooks linked from #105.