search

tqtezos / minter

OpenMinter is dApp framework for enabling the creation and collection of non-fungible tokens (NFTs) on Tezos.
MIT License
192 stars 91 forks source link

Excessive resource consumption makes minter difficult to use in sandbox mode #229

Closed murbard closed 3 years ago

murbard commented 3 years ago

Running yarn start:sandbox sends all CPUs close to 100% and starts swapping large amounts of memory to disk. I have quad core Intel i7-8650U CPU @ 1.90GHz with 16 GB of ram. It's making it impossible to use the minter.

lambdahands commented 3 years ago

This is an issue specific to sandbox mode. What's happening is the flextesa node is trying to "keep up" with blocks at a fixed interval (5 seconds I believe) up to the current timestamp. Stopping the system and starting it again at some later point in time, let's say 2 hours, means the node will try to process 1440 blocks as fast as it can, which will eat up large amounts of resources.

As I pointed out in another issue, the "solution" lies with clearing out all sandbox data with bin/reset --confirm (or sudo bin/reset --confirm on Linux) so the sandbox starts from scratch. This obviously isn't desirable if you want to persist your data across long running sessions, but I suggest in that case to configure OpenMinter in testnet mode. Not only will it avoid this issue, but it will use much fewer resources in general since it doesn't rely on a local indexer system.

murbard commented 3 years ago

1.I did the reset, it are all the resources nonetheless.

  1. Why is it creating a block every 5 seconds when inactive, instead of just doing it when there is a tx in the mempool?

  2. Why not use the mockup mode for this?

On Mon, 8 Mar 2021, 15:58 lambdahands, notifications@github.com wrote:

This is an issue specific to sandbox mode. What's happening is the flextesa node is trying to "keep up" with blocks at a fixed interval (5 seconds I believe) up to the current timestamp. Stopping the system and starting it again at some later point in time, let's say 2 hours, means the node will try to process 1440 blocks as fast as it can, which will eat up large amounts of resources.

As I pointed out in another issue, the "solution" lies with clearing out all sandbox data with bin/reset --confirm (or sudo bin/reset --confirm on Linux) so the sandbox starts from scratch. This obviously isn't desirable if you want to persist your data across long running sessions, but I suggest in that case to configure OpenMinter in testnet mode. Not only will it avoid this issue, but it will use much fewer resources in general since it doesn't rely on a local indexer system.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tqtezos/minter/issues/229#issuecomment-792854103, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMETPW254PZ7YBGKI6XIGTTCTX4DANCNFSM4YZLLOVQ .

lambdahands commented 3 years ago

1.I did the reset, it are all the resources nonetheless.

This could be other local services that are consuming a lot of resources on sandbox mode. Like I mentioned before, the sandbox environment spins up quite a few services (12 total), compared to testnet mode which only spins up four.

  1. Why is it creating a block every 5 seconds when inactive, instead of just doing it when there is a tx in the mempool?
  2. Why not use the mockup mode for this?

I believe @smondet would have more insight into how flextesa is configured to process blocks at fixed intervals and ways we may be able to help us out with our use case

murbard commented 3 years ago

Running a simple dapp locally on my machine to test minting NFTs should not involve running 12 different docker containers.

smondet commented 3 years ago

Flextesa by the default runs the baker/endorser/accuser setup so they obey the time-between-blocks protocol parameter.

What are the 12 - 4 = 8 containers?!? That sounds like a lot.

lambdahands commented 3 years ago

The high container count has to do with running BCD services locally. That system requires many dependencies like RabbitMQ, ElasticSearch, and Postgres, to support the indexer itself (tzkt), its API, the GUI, and a metrics service. Those seven services plus the sandbox container make up the eight required to run a local indexer and node instance.

murbard commented 3 years ago

My laptop becomes completely unresponsive when I try to run it. It starts heating up, processes run out of memory etc. I suppose with some work it might be possible to get it to work, but "difficult" isn't an accurate description. It took half an hour for me to be able to go back to the console and kill the process.

lambdahands commented 3 years ago

This issue is fixed via the changes to running OpenMinter in v0.4.0