Can we create a graph model for all of our contextual data and then use the same graph to train all our downstream models?

[dbanka]

In the examples mentioned in the demo, we saw that graph construction was done while training the specific use case like classification. Is there a way, we can construct graph on whole of our contextual data and then use that graph for our downstream applications like classification, knowledge graph completion etc.?

[AlanSwift]

Do you mean that the graph construction component should be disentangled from the pipeline such that it can be used for other applications? Eg, after we trained the classification app with the dependency graph, we can use the same graph to train another app without constructing the graphs again?

[dbanka]

Yes exactly.

[AlanSwift]

OK. Currently, the processed graph data can be reused. You can just copy it to another project and load it to train another app. Do you encounter some problems?

[sandeepKondaveeti]

For our use case, Constructed Graph using the approach provided in Text_Classification application; I am able to construct the graph. Our Idea is as mentioned in the earlier comments, to try to use the obtained graph for further applications like KG_Completion in this scenario. I have modified wrangle_KG.py accordingly to create the entities and relation files. (e1rel_to_e2_full.json, e1rel_to_e2_ranking_dev.json, e1rel_to_e2_ranking_test.json, e1rel_to_e2_train.json), and modified the below code snippet for the same

Code Block:
def main(args, model_path):
    input_keys = ['e1', 'rel', 'rel_eval', 'e2', 'e2_multi1', 'e2_multi2']
    p = Pipeline(args.data, keys=input_keys)
    p.load_vocabs()
    vocab = p.state['vocab']

    train_batcher = StreamBatcher(args.data, 'train', args.batch_size, randomize=True, keys=input_keys)
    dev_rank_batcher = StreamBatcher(args.data, 'dev_ranking', args.test_batch_size, randomize=False, keys=input_keys)
    test_rank_batcher = StreamBatcher(args.data, 'test_ranking', args.test_batch_size, randomize=False, keys=input_keys)
    data = []
    rows = []
    columns = []
    num_entities = vocab['e1'].num_token
    num_relations = vocab['rel'].num_token
    if args.model is None:
        model = ConvE(args, vocab['e1'].num_token, vocab['rel'].num_token)
    elif args.model == 'conve':
        model = ConvE(args, vocab['e1'].num_token, vocab['rel'].num_token)
    elif args.model == 'distmult':
        model = Distmult(args, vocab['e1'].num_token, vocab['rel'].num_token)
    elif args.model == 'complex':
        model = Complex(args, vocab['e1'].num_token, vocab['rel'].num_token)
    elif args.model == 'ggnn_distmult':
        model = GGNNDistMult(args, vocab['e1'].num_token, vocab['rel'].num_token)
    elif args.model == 'gcn_distmult':
        model = GCNDistMult(args, vocab['e1'].num_token, vocab['rel'].num_token)
    elif args.model == 'gcn_complex':
        model = GCNComplex(args, vocab['e1'].num_token, vocab['rel'].num_token)
    else:
        raise Exception("Unknown model!")

    if args.model in ['ggnn_distmult', 'gcn_distmult', 'gcn_complex']:
#         graph_path = 'examples/pytorch/kg_completion/data/{}/processed/KG_graph.pt'.format(args.data)
        graph_path = 'examples/pytorch/text_classification/data/trec/processed/constituency_graph/data.pt'
        KG_graph = torch.load(graph_path)
#         print("KG_graph", KG_graph)
#         KG_graph = KG_graph.to('cuda')
    else:
        KG_graph = None train_batcher.at_batch_prepared_observers.insert(1,TargetIdx2MultiTarget(num_entities, 'e2_multi1', 'e2_multi1_binary'))
    eta = ETAHook('train', print_every_x_batches=args.log_interval)
    train_batcher.subscribe_to_events(eta)
    train_batcher.subscribe_to_start_of_epoch_event(eta)
    train_batcher.subscribe_to_events(LossHook('train', print_every_x_batches=args.log_interval))
    model.cuda()
    if args.resume:
        model_params = torch.load(model_path)
        print(model)
        total_param_size = []
        params = [(key, value.size(), value.numel()) for key, value in model_params.items()]
        for key, size, count in params:
            total_param_size.append(count)
            print(key, size, count)
        print(np.array(total_param_size).sum())
        model.load_state_dict(model_params)
        model.eval()
        ranking_and_hits(model, test_rank_batcher, vocab, 'test_evaluation')
        ranking_and_hits(model, dev_rank_batcher, vocab, 'dev_evaluation')
    else:
        model.init()
    total_param_size = []
    params = [value.numel() for value in model.parameters()]
    print(params)
    print(np.sum(params))
    best_mrr = 0
    opt = torch.optim.Adam(model.parameters(), lr=args.lr, weight_decay=args.l2)
    for epoch in range(args.epochs):
        model.train()
        for i, str2var in enumerate(train_batcher):
            opt.zero_grad()
            e1 = str2var['e1']
            rel = str2var['rel']
            e2_multi = str2var['e2_multi1_binary'].float()
            # label smoothing
            e2_multi = ((1.0-args.label_smoothing)*e2_multi) + (1.0/e2_multi.size(1))
            pred = model.forward(e1, rel, KG_graph)
            loss = model.loss(pred, e2_multi)
            loss.backward()
            opt.step()
            train_batcher.state.loss = loss.cpu()
        model.eval()
        with torch.no_grad():
            if epoch % 2 == 0 and epoch > 0:
                print("KG_graph:: ", KG_graph)
                dev_mrr = ranking_and_hits(model, dev_rank_batcher, vocab, 'dev_evaluation', kg_graph=KG_graph)
                if dev_mrr > best_mrr:
                    best_mrr = dev_mrr
                    print('saving best model to {0}'.format(model_path))
                    torch.save(model.state_dict(), model_path)
            if epoch % 2 == 0:
                if epoch > 0:
                    ranking_and_hits(model, test_rank_batcher, vocab, 'test_evaluation', kg_graph=KG_graph)

Currently I am not able to use the graph as I am getting AttributeError: 'StreamBatcher' object has no attribute 'loaders'. I need help in steps/process to re-use the already constructed graph in KG Completion.

[Nicoleqwerty]

@sandeepKondaveeti Hi, the graph used in KG Completion is a single graph4nlp.pytorch.data.dataGraphData object, while the data.pt file built by the graph_construction module is a dict object (in version 0.5) or a graph4nlp.pytorch.data.dataset.Dataset object (in version 0.4). I guess this leads to the AttributeError.

[sandeepKondaveeti]

@Nicoleqwerty , Please let us know if there is a way to covert the dict object to a single graph4nlp.pytorch.data.dataGraphData. We want to use the same graph for multiple applications.

[Nicoleqwerty]

The dict is in the form of {'train': list of DataItems, 'test': list of DataItems, ...}. You can use DataItem.graph to access GraphData.

[sandeepKondaveeti]

@Nicoleqwerty Even we try to access GraphData for all the values in the dict DataItem.graph, we will be having multiple graphs, my query is how can we combine all of the dataitems to single graph.

[sandeepKondaveeti]

@Nicoleqwerty Another query is can we do a vice-versa of constructing the Graph using KG-Completion and use the same graph data for Text-Classification, Since there are multiple dependency files like data.pt, local.pt and vocab.pt for self._build_dataloader() in line 240 of run_text_classifier.py file. To debug the same modified dataset.py file function build_topology line 711 with the graph constructed from KG_Completion, Please let me know if this approach of constructing graph in one application and use the graph in another is possible if yes any other areas I need to modify the build_path.

[Nicoleqwerty]

@sandeepKondaveeti Hi, you can use to_batch() to combine a list of GraphData objects to a single graph.

[Nicoleqwerty]

@sandeepKondaveeti

@Nicoleqwerty Another query is can we do a vice-versa of constructing the Graph using KG-Completion and use the same graph data for Text-Classification, Since there are multiple dependency files like data.pt, local.pt and vocab.pt for self._build_dataloader() in line 240 of run_text_classifier.py file. To debug the same modified dataset.py file function build_topology line 711 with the graph constructed from KG_Completion, Please let me know if this approach of constructing graph in one application and use the graph in another is possible if yes any other areas I need to modify the build_path.

Do you mean that you want to do a KG node classification task? In our examples, the graph construction of KG Completion builds a single KG graph directly. The graph construction of text classification builds a dependency or constituency graph for each text. If you want to merge these text graphs as a graph, you can use to_batch() mentioned above.

[dbanka]

@Nicoleqwerty @AlanSwift What my thought was that we can convert all our data relationships to graph embedding model using graph construction. Then given any use case be it KG completion or classification, we can use the same graph. Lets say I have 10 different types of entities and 20 types of relationships between them. I wanted to put all of the data into graph model and then if I want to train a classification model using data from just 2 of those entity types, I can train a classification model taking the graph embedding from the graph model. Is my approach correct or am I missing something? Because I am unable to understand why should we construct separate graph for each of our use cases as shown in the examples.

[Nicoleqwerty]

@dbanka Hi, you can check the example codes of KG completion and text classification. They are two completely different tasks with different inputs. For KG completion, the input is a single KG and the goal of KG completion is to predict the missing links between entity nodes. For text classification, each input is a (text, label) pair. A text classification dataset has lots of pairs. Unlike KG, there are usually no relation between texts in text classification. So we construct separate graph for each text.

As for whether the constructed graph can be reused, I think take KG completion and KG node classification as a pair of examples is more suitable. Both tasks may take the same KG as input. You can certainly do KG completion or node classification on the same graph.

[dbanka]

@Nicoleqwerty What my point was that can we do Graph construction for all of our data? Then in future if we have to train any classification model or any other model, we can just get the graph embedding of the inputs required for that model and train a deep neural network model for classification using those embedding as input. In doing so, we will have a graph model for all our data and then we can use the graph embedding from that model to train any model we want.

[Nicoleqwerty]

@dbanka The processed graph data can be reused. Since KG completion and text classification are two completely different tasks, you cannot directly apply the graphs built in text classification to KG completion example codes. You can merge the graphs built in text classification using to_batch().

[xiao03]

Close this issue due to inactivity from the user. Please reopen it when necessary.