fix: Fix snapshots in case of bad existing snapshot #70

[pritamghanghas]

• don't fail if the existing snapshot is missing or has errors. proceed with taking a new snapshot.

Issue

• Link: https://github.com/NebraLtd/product-management/issues/70
• Summary: Before taking the snapshot, create_snapshot.sh tries to get the height from the previously uploaded latest-snap.json file. If it's invalid, taking the snapshot just fails. We now allow taking the snapshot even if the previously uploaded latest-snap.json is invalid.

How if we fail to get current snapshot height, initialize it to zero so that the scripts proceeds to take a fresh snapshot.

Screenshots

References

Checklist

• [ ] Tests added
• [ ] Cleaned up commit history (rebase!)
• [ ] Documentation added
• [ ] Thought about variable and method names

[sonarcloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[pritamghanghas]

@marvinmarnold @KevinWassermann94 I have aligned the snapshotter (Miner running in vm) sys.config located at /var/miner_config/sys.config with upstream release sys.config. But leaving the device config.template as is because the issue was only about snaptshotter. Not sure whether rpc_timeout is actually doing anything because i had turned off mux at the same time. It is probably worthless because I could only find its usage in test suite in erlang miner codebase. But doesn't seem to be huring anything.

[marvinmarnold]

Not sure whether rpc_timeout is actually doing anything because i had turned off mux at the same time.

@KevinWassermann94 what do you think about RPC timeouts? Are we seeing this in our fleets and do you think it's related to this mux setting? Other than some hardware specific settings (like RAM), we should generally keep the snapshotter and fleet configs in sync.

[KevinWassermann94]

RPC timeouts are common when loading a new snapshots, but the process is still continuing. However there seems to be an issue with the snapshot loading, which might be caused due to our changed heap size or another settings

[pritamghanghas]

Just to recap, Main issue was files disappearing from bucket and thus jq existing due to 404 not being a valid json response. Rest of the issues were related to gz fixes missing from the script which we had done sometime back. Don't know how that was the case. But I wasted sometime debugging something that was already fixed in master. I am leaving rcp_timeout config just as reminder that such an option might be useful in the future, even though looking at current code only miner tests module seem to be using it. mux setting shouldn't have any impact on snapshotter. but aligned to upstream just to avoid confusion.