Closed RandomString123 closed 4 years ago
I am not sure what the problem is exactly, could you provide a minimum working example?
In principle graph networks always take graphs as inputs and output full graphs with the same structure, and a configurable number of features per-edge, per-node, and per-global. So in principle all you need is to match the output sizes to have the same size as the inputs, and build a loss to minimize the difference between the features of the output graphs (produced by feeding the ablated input graphs to the graph net) and the un-ablated training graphs.
In the next comment is a file that is a toy example I created. Imagine you wanted to represent all of the possible functions x + y = z as a a graph where x = {1,2,3}, y = {4,5,6}. Then over-fit a model so that if it sees the graph 1 + 5 = ? it will return the completed graph 1 + 5 = 6.
From your above message and my thoughts over the day I think there are three problems here and they all probably stem from my unfamiliarity with tf and graph networks.
1) I have too many free variables and they are unbounded so the algorithm just randomly bounces around. 2) I am probably using the wrong model. 3) My loss function is probably sub-optimal.
I did some more testing over the weekend and found a bug in my loss model. Put a better version below. It still does much worse than I would expect for the reasons above. The linear model still doesn't do very well I think i should be using another when ablating out a nodes attributes and trying to predict its value based on the rest of the graph.
Hi, there seems to be several things that could be fixed in that code (I tried fixing them an the model trains in 100 iterations):
edge_model_fn
, node_model_fn
and global_model_fn
are snt.Linear
, so the model will have very little capacity (Although in this case it is enough to solve the problem if you connect the graph as I indicated above). You should probably use snt.nets.MLP
insteadedge_model_fn
, to have an output size of 1, this means the model only has a single edge feature, that has to output to either 1 or 2, to output the correct edge values, so it cannot really use the edges to send information between nodes. In this case the easiest solution would be to remove the edges from the loss, and increase OUTPUT_EDGE_SIZE. If you actually want to be able to make edge predictions of size one too, the usual alternative is to still have a larger OUTPUT_EDGE_SIZE to produce an output embedding for each edge, and then use an additional linear later only on output_graph.edges
with an output size of 1. In practice the best option is to use a final modules.GraphIndependent
to decode nodes, edges and globals independently.Hope this helps!
Those suggestions got me a lot closer to a working solution. Thanks, it is a good starting point for toying around with the networks and seeing how they operate.
I am new to graph networks and I have reviewed, and am using, this project as a springboard to learn more. I understand the examples provided with this project and thought I could extend those examples to perform what most graph network papers describe as a basic task, predicting {Node, Edge, Global} attributes of a graph based on graph structures and relationships learned from training graphs.
However, every model I seem to apply does not produce the expected target graph it produces some intermediate graph that I can not figure out how to decode back into my original graph with the missing attributes filled in. I think it would help if there was an example project that performs {Node, Edge, Global} attribute prediction of a graph based training performed on similar graphs? Even if this example just trains on random graphs / data. I am more interested in structuring a network to: