UMT first block after hard fork is not stored to GCS and may be missed if archive is not started first

Preliminary Checks

[X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/MinaProtocol/mina/issues
[X] This issue is not a question, feature request, RFC, or anything other than a bug report. Please post those things in GitHub Discussions: https://github.com/MinaProtocol/mina/discussions

Description

After the hard fork, I started the node, waited for it to sync, and then turned up the archive. Unfortunately, this meant I missed the first block (1277-3NK6S443vSpoSQ9va6ZCX4CewAUqmmig7d2eTZwRYe3oqPnWVMEx). Usually, that's not a big deal - a download missing blocks script will pick it up from GCS; however, this block is unique, and the precomputed block is not stored in GCS.

https://discord.com/channels/484437221055922177/1212960015795888188

Steps to Reproduce

At hard fork, start mina daemon and wait to sync before starting mina-archive
...

Expected Result

Missed "genesis"-ish block would be present in GCS.

Actual Result

Block was not put to archive db (because archive was not running at the moment) and was also not put to GCS, so there was no (normal) way to recover.

Stopping node, removing .mina-config, and starting up again, I was able to get the block inserted into the archive db.

Daemon version

Commit eb0a6d00675756972f4f22ecaa10532bae601c59 on branch hardfork-automation-umt

How frequently do you see this issue?

Rarely

What is the impact of this issue on your ability to run a node?

Low

Status

Mina daemon status
-----------------------------------

Global number of accounts:                     201806
Block height:                                  1300
Max observed block height:                     1300
Max observed unvalidated block height:         1300
Local uptime:                                  36m29s
Ledger Merkle root:                            jxLLJAweSiqXNXncqR41fYSQekzJ7SWCG8mVtQHbCYELVqxLhj2
Protocol state hash:                           3NKSQVVNN1zVAjupqktkyxf72zjKQQYkaVfx9rfhW4ZDHUMNW1SR
Chain id:                                      ded2784524f624295e63cede1a4ab9348fbcde7160876c4d55f1481b9d3ef542
Git SHA-1:                                     eb0a6d00675756972f4f22ecaa10532bae601c59
Configuration directory:                       /home/minar/.mina-config
Peers:                                         45
User_commands sent:                            0
SNARK worker:                                  None
SNARK work fee:                                100000000
Sync status:                                   Synced
Catchup status:                                
        To build breadcrumb:           0
        To initial validate:           0
        Finished:                      24
        To download:                   0
        Waiting for parent to finish:  0
        To verify:                     0

Block producers running:                       0
Coinbase receiver:                             Block producer
Best tip consensus time:                       epoch=0, slot=156
Best tip global slot (across all hard-forks):  3256
Consensus time now:                            epoch=0, slot=158
Consensus mechanism:                           proof_of_stake
Consensus configuration:                       
        Delta:                     0
        k:                         290
        Slots per epoch:           7140
        Slot duration:             3m
        Epoch duration:            14d21h
        Chain start timestamp:     2024-02-29 21:00:00.000000Z
        Acceptable network delay:  3m

Addresses and ports:                           
        External IP:    
        Bind IP:        0.0.0.0
        Libp2p PeerID:  
        Libp2p port:    8304
        Client port:    8301

Metrics:                                       
        block_production_delay:             7 (0 0 0 0 0 0 0)
        transaction_pool_diff_received:     7
        transaction_pool_diff_broadcasted:  0
        transactions_added_to_pool:         33
        transaction_pool_size:              7
        snark_pool_diff_received:           111
        snark_pool_diff_broadcasted:        0
        pending_snark_work:                 0
        snark_pool_size:                    133

Additional information

Error from daemon (because archive service hadn't started yet.)

Could not send breadcrumb to archive:
... ["Unix.Unix_error","Connection refused","connect","127.0.0.1:3086"] ...

MinaProtocol / mina