Open vladimirlogachev opened 2 years ago
Could you please share your observations if there are any? when the memory usage increases? after the integration test contract endpoint activation?
I was just starting all 4 executables 1 by one, waiting for previous to sync, and the plutus-pab-examples
is the last one.
At block 11 752 the amount of swap used was 22 GB At block 15 427 (slot 6 959 416) the amount of swap used was 72 GB
I did not even try to start the integration test. Please tell me your email/telegram/other contact, so that I could give you an access to the server. It doesn't contain anything private at the moment.
I've just tried running plutus-pab-examples
with --rollback-history 1000
, but I have no idea whether 1000
is a sane value or not.
It is still synchronizing more successfully, and memory consumption looks more healthy.
Current block: 26 939. Current slot: 8 253 200
UPD:
It's slow.
Current block: 47 207. Current slot: 20 223 898
(this last line of output did not change for at least 7 hours!)
And it's still possible to reach OOM soon...
Yeah, reducing the rollback history might help a bit but is not a general case. Working on the fix.
I have the same problem but in a more modest docker instance with "only" 11 GB of RAM. I have been increasing the amount of RAM of the instance in successive tries until the maximum allowed in my 16 GB machine and the final result has been the same: the plutus_pab_examples being killed. All four executables: the cardano-node, cardano-wallet, plutus-chain-index and the pab examples running in the same docker container.
If it helps anyone: setting the dev options to the current slot, cleaning the contract db (rm my.db
+ migrate
), and rerunning leads to a PAB service that is stable for a while. Enough for developing with at least.
If it helps anyone: setting the dev options to the current slot, cleaning the contract db (
rm my.db
+migrate
), and rerunning leads to a PAB service that is stable for a while. Enough for developing with at least.
To make it more explicit, unconment the lines here of pab-config.yml to:
developmentOptions:
pabRollbackHistory: null
pabResumeFrom:
#{ "tag" : "PointAtGenesis" }
{ "tag" : "Point"
, "pointBlockId" : " enter here the last block id from the log of the cardano node"
, "pointSlot" : { "getSlot" : the slot }
}
And now it seems to synchronize well. Following Lars lectures: https://youtu.be/X6AyZIZ0vaE?t=769
I would like to know why the PAB app needs to sync It may be important for the knowledge of the cardano blockchain. Is it because of the possibility of deep rollbacks?
up
Summary
We have set up a server just to be able to follow the test-node instructions:
x86_64 12 CPU cores 24 GB RAM 80 GB swap
But the
plutus-pab-examples
executable seems to be killed due to OOM.At block
11 752
the amount of swap used was 22 GB At block15 427
(slot6 959 416
) the amount of swap used was 72 GB And that's far from completion: According to the testnet explorer, at the moment of submitting the issue, the latest slot on testnet is55 601 949
, block3 474 971
If it will help to find the problem, we can provide SSH access to the server.
Steps to reproduce the behavior
https://github.com/input-output-hk/plutus-apps/tree/main/plutus-pab/test-node
Actual Result
The pab examples sync process does not complete due to OOM
If it will help to find the problem, we can provide SSH access to the server.
Expected Result
The pab examples sync process completes
Describe the approach you would take to fix this
No response
System info
OS: Ubuntu Version: 18.04 plutus-apps: v2022-04-06