bitcoin-dev-project / warnet

Monitor and analyze the emergent behaviors of Bitcoin networks
https://warnet.dev
MIT License
63 stars 28 forks source link

Prevent docker compose based warnet from downloading the same docker images so many times #209

Closed m3dwards closed 4 months ago

m3dwards commented 6 months ago

This could be more of an issue for developers of warnet who are running frequent tests rather than users who will likely just pull once.

A suggestion could be to add a flag to request to keep the images when stopping (compose down) a network.

Alternatively, would developer documentation suggesting to manually pull commonly used images solve the problem as I think compose will only remove images that it pulled as part of it's up command.

willcl-ark commented 6 months ago

I think actually the second approach is "nicer" for users -- the image will be erased (in standard docker compose fashion?) after it's done with. Adding docs to pull a few select images seems reasonable.

I pulled each image we have and ran a new network. I didn't see any hash comparisons done or images downloads (but still possible it did something in the bg -- I didn't disable or monitor networking). So it seems to work for me if you pull the images.

The only loophole might be CI... but perhaps if we all stop pulling them, we'll stop hitting our limits?

m3dwards commented 6 months ago

I think the problem on CI will exist anyway as any pulled images will be destroyed when that actions runner is destroyed.

On another project I logged into docker hub as part of the actions runner to increate the limits. For us, it was more tricky than just using a login action as that would only count for images pulled on the command line. Any images pulled from code weren't pulled using the docker hub user so we had to also docker hub login from code to finally get the raised limit we needed.

pinheadmz commented 6 months ago

Discussed offline, I'm having trouble reproducing this. We fixed it! (?)

willcl-ark commented 4 months ago

Going to close for now. We can re-open if it reoccurs.