search

CogComp / zoe

Zero-Shot Open Entity Typing as Type-Compatible Grounding, EMNLP'18.
43 stars 5 forks source link

Questions about title2freebase mapping #32

Closed zhaoxy92 closed 5 years ago

zhaoxy92 commented 5 years ago

Hi,

I noticed that the file title2freebase.pickle is generated from the function def convert_freebase(freebase_file_name, freebase_sup_file_name). My questions are:

  1. It seems freebase_file_name and freebase_sup_file_name have the same format, could you explain what these two files are and where to get them?
  2. Related to the first question, looks like the key-value pairs in freebase_file_name and freebase_sup_file_name are (wikititle, freebase type) pairs. How did you get them?
  3. Another related one, how did you map wiki title to freebase entities and then get corresponding freebase types?

Thank you very much!

Slash0BZ commented 5 years ago

Hi, freebase_file_name and freebase_sup_file_name are results of a hack of engineering: the latter is an update or supplement to an outdated mapping we have. As a result, I think you could just use one file that is a union of both maps.

In terms of how we got them: they are processed from Google dumps. It was separately processed, and I think the aforementioned two files are fairly complete based on this dump; please let me know if you have encountered incompleteness issues.

As for your third question, there is a MID in every data entry, and Google also provided MID to Wikidata IDs (vice versa), where you can find corresponding Wikipedia titles.

Please let me know if you have further questions, thanks.

zhaoxy92 commented 5 years ago

Hi, freebase_file_name and freebase_sup_file_name are results of a hack of engineering: the latter is an update or supplement to an outdated mapping we have. As a result, I think you could just use one file that is a union of both maps.

In terms of how we got them: they are processed from Google dumps. It was separately processed, and I think the aforementioned two files are fairly complete based on this dump; please let me know if you have encountered incompleteness issues.

As for your third question, there is a MID in every data entry, and Google also provided MID to Wikidata IDs (vice versa), where you can find corresponding Wikipedia titles.

Please let me know if you have further questions, thanks.

Thanks. I guess it's more of an engineering question. In the freebase dump we have triplets (subject, predicate, object), but I found MID occurs in both sub and obj. Could you please let me know which ones are counted a valid mid and which ones are counted as a valid type? Thank you!

Slash0BZ commented 5 years ago

I see. From what I remember, subject is the entity, predicate is the relation type. For example, if you are only interested in getting entity freebase types, you should look at RDF tuples where the predicate is type.object.type, while in those cases, object should be human-readable types. In other words, I don't think you need to care about the MIDs in objects.

I am not 100% sure on this, but I can check the data tomorrow. Please let me know if this is consistent with the dump you have.

zhaoxy92 commented 5 years ago

I see. From what I remember, subject is the entity, predicate is the relation type. For example, if you are only interested in getting entity freebase types, you should look at RDF tuples where the predicate is type.object.type, while in those cases, object should be human-readable types. In other words, I don't think you need to care about the MIDs in objects.

I am not 100% sure on this, but I can check the data tomorrow. Please let me know if this is consistent with the dump you have.

Thanks for the clarification! This is very helpful!