search

SMRFoundation / NodeXLBasic

1 stars 1 forks source link

Memory problem importing data from Facebook Fan Pages with NodeXL Pro #317

Open SMRFoundation opened 6 years ago

SMRFoundation commented 6 years ago

Hello, I’m trying to import data from Fan Pages Renault (Id: 85969866019) and Volkswagen (Id: 157405840954098). I want to download data between 01-09-2015 and 20-01-2016, but Independently of the number of posts, when it comes to prepare the worksheet and create the networks, the program stops working and displays an exception error. The application display the following error for both IDs: • “The network couldn´t be obtained. Details: [OutOfMemoryException]: Exception of type 'System.OutOfMemoryException' was thrown” • “The network couldn´t be obtained. Details: [NullReferenceException]: Object reference not set to an instance The Fan Page with the Id 157405840954098 has 617 posts and 9.840 comments and the other, Id 85969866019, has 325 posts and 1.068 comments. The dimension of the date are totally different, but I have the same errors. My laptop has 8GB RAM installed and when I was monitoring the problem with Windows task manager, Excel was only using 37% when the errors appeared. My Laptop is configured with Window 10, System 64-bits and run Microsoft Office 365 ProPlus Excel 32-bits. I think my problem is related with the fact that the networks I’m trying to download are too big and Excel ran out of memory trying to load it. Is there the need to allocate more memory to import a large set of data? Can you confirm if I’m correct or how can I solve and fix these errors? I want try to convert from Excel 32-bits to Excel 64-bits to allocate up 8Tb, but first I’d like to know if there are any issues with NodeXL Pro and Microsoft Office 365 ProPlus Excel 64-bits Thank you very much.

Attachments

Error-NodeXL Pro.JPG ID_85969866019 & ID_157405840954098.pdf

This work item was migrated from CodePlex

CodePlex work item ID: '65066' Vote count: '1'

SMRFoundation commented 6 years ago

[arber@1/25/2016] Hello,

you are getting SystemOutOfMemory exception because your Excel version is 32-bit. If you upgrade to 64-bit, Excel will be able to allocate more memory, but you may still get this error if the network is too large.

There is no known issue with NodeXL Pro and Microsoft Office 365 ProPlus Excel 64-bits.

The other error (“The network couldn´t be obtained. Details: [NullReferenceException]: Object reference not set to an instance) is not related to the fact that Excel is 32-bit (it looks like some Facebook data that you requested are missing).

If you want I can try downloading those data (I have a 64-bit Excel) and tell you if I succeeded. In order for me to do that, you have to send me a screenshot of the options you selected on the importer.

Regards, Arber

SMRFoundation commented 6 years ago

[MariaFreire@1/26/2016] Dear Arber,

Thank you for your availability and readiness. Indeed, I’d be much obliged if you could try to download those data and therefore reach some conclusion on whether the problem is related with the excel version or anything else. Please find in attachment the screenshots I selected for both IDs. Again, thank you very much and hope to get some news from you soon.

Best regards Manuela Freire

SMRFoundation commented 6 years ago

[UnknownUser@1/26/2016]

SMRFoundation commented 6 years ago

[arber@1/27/2016] Hello,

I tried to download the above networks and I never got the OutOfMemory exception (office 2013 64-bit, 8GB of RAM). However I did get the second exception which states that we have a bug in the Facebook importer and we will fix it.

What I can suggest you is limit the number of comments/likes per post (I tried that and I successfully downloaded the network) and leave the default attributes selection as is (removing the profile picture causes the importer to through an exception).

Regards, Arber