search

commonsense / conceptnet-numberbatch

Other
1.29k stars 143 forks source link

number batch code #44

Closed meghmehta closed 7 years ago

meghmehta commented 7 years ago

Hi,

I've run the code over the whole weekend but it still has not completed. You had mentioned it would take about a day to run... do you think there was something that happened incorrectly?

Thanks,

Megh

rspeer commented 7 years ago

What's happening? Has it computed any results in the build-data directory? Is it possible that you're out of RAM and swapping (which will slow things down by orders of magnitude)?

Also, if I can ask, what's your goal in reproducing this particular evaluation? If it's for the sake of the science, you should probably use a later version that actually got published. And if you have custom data you want to incorporate, then you don't have to run all the conditions to reproduce a data table in an unpublished paper.

meghmehta commented 7 years ago

No, I still don't have a build-data directory....I wanted to incorporate custom data and then have a look at the vectors produced...

rspeer commented 7 years ago

Okay, so you could save some pain by modifying the code to just build one matrix, not all the matrices we ever evaluated.

But it sounds like you aren't even able to compute a single matrix. Is it just swapping or what?

In general I'd encourage you to update to the published build process, incorporated in ConceptNet 5.5: https://github.com/commonsense/conceptnet5/wiki/Build-process . But if you have problems with a lack of computing resources you're still going to have them.

meghmehta commented 7 years ago

Okay thanks - so where can I add my own vectors? Also where could I change so that I only build one matrix? Thanks for the help!

meghmehta commented 7 years ago

The code finished running but there is still no build-data folder containing any output....

rspeer commented 7 years ago

You would add your own vectors by changing the conceptnet5 code. There isn't a neatly-packaged process for it. Feel free to contribute it if you come up with one.

If you want me to keep trying to debug your run of this old version of conceptnet-numberbatch, you're going to have to tell me what it did output, not just what it didn't output.