gridcoin-community / Gridcoin-Research

Gridcoin-Research
MIT License
588 stars 173 forks source link

Neural Network shows 0 whitelisted projects and 0 mag for all cpids. #88

Closed AcridSmoke13 closed 7 years ago

AcridSmoke13 commented 8 years ago

Tried deleting contents of NeuralNetwork folder and resyncing. No luck.

Execute syncdpor2 returns a syncing message but the NN doesn't update.

debug2.log shows that there was an error accessing a number of files (all in the neural network folder)

Here's the relevant part:

May 31 2016 15:27, Testnet : False May 31 2016 15:27, Testnet : False May 31 2016 15:27, Finished syncing DPOR cpids. May 31 2016 15:27, EnsureTeamIsSynchronized: 493038 May 31 2016 15:27, Deleting Projects May 31 2016 15:27, EraseNeuralNetwork:No files found matching 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project.dat'. May 31 2016 15:27, EraseNeuralNetwork:No files found matching 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\projects.dat'. May 31 2016 15:27, EraseNeuralNetwork:No files found matching 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\whitelist.dat'. May 31 2016 15:27, Updating magnitude in testnet=False for 1992 cpids. May 31 2016 15:27, EraseNeuralNetwork:No files found matching 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\cpid.dat'. May 31 2016 15:27, UFA Timestamp: 1000000 May 31 2016 15:27, GUFA Timestamp: 1464283407.04738 May 31 2016 15:27, Starting Phase II May 31 2016 15:27, Complete Sync: Updating mags May 31 2016 15:27, Updating Magnitudes Without consensus data May 31 2016 15:27, 1992 CPIDs starting out with clean slate. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #1 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:27, Attempt #0 While storing data row Projects,Project, PK: NeuralNetwork : The process cannot access the file 'C:\Users[REDACTED]\AppData\Roaming\gridcoinresearch\NeuralNetwork\project_projects_NeuralNetwork.dat' because it is being used by another process., Retrying. May 31 2016 15:28, Testnet : False May 31 2016 15:28, 4930809391e0812100407fbafe51ddd3b0b91356037905-25-2016 16:57:52f83970a0f3c0cf27a70c4d11eb4fb91c May 31 2016 15:28, 0 CPIDs remaining with no witnesses. Cleaning up problem. May 31 2016 15:28, UpdNetworkAvgs Start Time May 31 2016 15:28, UpdNetworkAvgs End Time May 31 2016 15:28, Unable to get quote data probably due to SSL being blocked: Conversion from string "559buysupport5772.20964827" to type 'Double' is not valid. May 31 2016 15:28, Unable to get quote data probably due to SSL being blocked: Conversion from string "0.00001025buysupport8.46097691" to type 'Double' is not valid.

grctest commented 8 years ago

I can confirm I've seen the same issue of "The process cannot access the file 'file path' because it is being used by another process." in the past.

gridcoin commented 8 years ago

Thank you for your analysis. CM sent this to my attention today. I will also post a message on CCtalk ASAP, but for now, I am feverishly analyzing it.

gridcoin commented 8 years ago

So, those errors you are receiving are actually handled by the app (we need to make it so they do not log and gracefully get handled, but in reality the code does re-loop and take care of all the file access errors gracefully in the current verson, its just the logs do not look very nice).

But moving on to the critical problem for now, the issue is (and this is a Coincidence - it happened exactly the same time RTM delisted or blacklisted bitcoin utopia). After 6 months, our projects in the chain are erased from the whitelist to save space. No one re-whitelisted them.

I am whitelisting them now, and testing the neural network. I'll keep you posted, but it looks like this is the real problem.

Rob

grctest commented 8 years ago

gridcoin: We can close this issue, unless you're wanting to keep it open to address the projects needing re-whitelisted every 6 months?

Is it every 6 months because after 6 months the registration transaction is outwith the 6 month lookback period? What if we sent these transactions to specific burn addresses which our clients could monitor? Would be far faster than looking through the last 172800 blocks. All we would need to do is integrate the final watch-only commit from earlier in the year and this would be possible. https://github.com/gridcoin/Gridcoin-Research/issues/82#issuecomment-219294915

Hell, we could do the same for beacon advertisements preventing them expiring after 6 months too.

gridcoin commented 7 years ago

Its 6 months to reduce client load time- during load, the client only keeps the last 6 month of block messages (project names, beacons, votes, contracts) in memory.