Pool stuck in `AmmStatus::WaitingTrade`, despite being past `poolOpenTime`

On devnet, I have created an OpenBook market with the sdk (sdk-examples repo) utilsCreateMarket.ts script. Then with the OpenBook market ID I was able to successfully run the sdk ammCreatePool.ts script. I was also able to run the ammAddLiquidity.ts script.

Finally I have been trying unsuccessfully to run the swapOnlyAmm.ts script with the Raydium pool ID. It is failing with this output.

[
  "Program ComputeBudget111111111111111111111111111111 invoke [1]",
  "Program ComputeBudget111111111111111111111111111111 success",
  "Program HWy1jotHpo6UqeQxx49dpYYdQB8wj9Qk9MdxwjLvDHB8 invoke [1]",
  "Program log: Error: InvalidStatus",
  "Program HWy1jotHpo6UqeQxx49dpYYdQB8wj9Qk9MdxwjLvDHB8 consumed 12732 of 199850 compute units",
  "Program HWy1jotHpo6UqeQxx49dpYYdQB8wj9Qk9MdxwjLvDHB8 failed: custom program error: 0x16"
]

I have reverse engineered this amm program to find where this error is thrown in process_swap_base_in instruction. https://github.com/raydium-io/raydium-amm/blob/18bed23de5a29b038a93db493fafd5ff4ee5386b/program/src/processor.rs#L2328-L2348

        if !AmmStatus::from_u64(amm.status).swap_permission() {
            msg!(&format!("swap_base_in: status {}", amm.status));
            let clock = Clock::get()?;
            if amm.status == AmmStatus::OrderBookOnly.into_u64()
                && (clock.unix_timestamp as u64) >= amm.state_data.orderbook_to_init_time
            {
                amm.status = AmmStatus::Initialized.into_u64();
                msg!("swap_base_in: OrderBook to Initialized");
            } else {
                return Err(AmmError::InvalidStatus.into());
            }
        } else if amm.status == AmmStatus::WaitingTrade.into_u64() {
            let clock = Clock::get()?;
            if (clock.unix_timestamp as u64) < amm.state_data.pool_open_time {
                return Err(AmmError::InvalidStatus.into());
            } else {
                amm.status = AmmStatus::SwapOnly.into_u64();
                msg!("swap_base_in: WaitingTrade to SwapOnly");
            }
        }

I can see that the status of my pool is 7 which is AmmStatus::WaitingTrade.

So I can see that my call will end up in this else if block

 else if amm.status == AmmStatus::WaitingTrade.into_u64() {
            let clock = Clock::get()?;
            if (clock.unix_timestamp as u64) < amm.state_data.pool_open_time {
                return Err(AmmError::InvalidStatus.into());
            } else {
                amm.status = AmmStatus::SwapOnly.into_u64();
                msg!("swap_base_in: WaitingTrade to SwapOnly");
            }
        }

So here I can see that I should get this error if the cluster clock time is < the pool open time. Here is the issue. The cluster clock time is higher than the pool open time for me. You can see the pool open time in here

{
    id: 'BeDL4VM836pE5C8wVMW92uFjE6jAvdDRCRg7De33z6qi',
    status: <BN: 7>,
    nonce: <BN: fc>,
    maxOrder: <BN: 7>,
    depth: <BN: 3>,
    baseDecimal: <BN: 6>,
    quoteDecimal: <BN: 6>,
    state: <BN: 1>,
    resetFlag: <BN: 0>,
    minSize: <BN: e8d4a51000>,
    volMaxCutRatio: <BN: 1f4>,
    amountWaveRatio: <BN: 4c4b40>,
    baseLotSize: <BN: 3b9aca00>,
    quoteLotSize: <BN: 1>,
    minPriceMultiplier: <BN: 1>,
    maxPriceMultiplier: <BN: 3b9aca00>,
    systemDecimalValue: <BN: 3b9aca00>,
    minSeparateNumerator: <BN: 5>,
    minSeparateDenominator: <BN: 2710>,
    tradeFeeNumerator: <BN: 19>,
    tradeFeeDenominator: <BN: 2710>,
    pnlNumerator: <BN: c>,
    pnlDenominator: <BN: 64>,
    swapFeeNumerator: <BN: 19>,
    swapFeeDenominator: <BN: 2710>,
    baseNeedTakePnl: <BN: 0>,
    quoteNeedTakePnl: <BN: 0>,
    quoteTotalPnl: <BN: 0>,
    baseTotalPnl: <BN: 0>,
    poolOpenTime: <BN: 18ea11243ad>,
    punishPcAmount: <BN: 0>,
    punishCoinAmount: <BN: 0>,
    orderbookToInitTime: <BN: 0>,
    swapBaseInAmount: <BN: 0>,
    swapQuoteOutAmount: <BN: 0>,
    swapBase2QuoteFee: <BN: 0>,
    swapQuoteInAmount: <BN: 0>,
    swapBaseOutAmount: <BN: 0>,
    swapQuote2BaseFee: <BN: 0>,
    baseVault: PublicKey [PublicKey(D82GAMtYBLBh84QHWcUYYcQDkPLq131sPgjfZVyYoPUj)] {
      _bn: <BN: b418bc2c6c093a3ecf81e77d045d7705d0cf634534931be056502caac13f3f80>
    },
    quoteVault: PublicKey [PublicKey(D3CpiAQ56Zjy6NqddPXqvAR5JdvzcDZZUim5RThms9F6)] {
      _bn: <BN: b2dcc5c71802f1a527b08711ad1047a6f8b3b2c89e645a66452bb819b310be61>
    },
    baseMint: PublicKey [PublicKey(3zL6LK3z5oj2iDK8HQ7XDGqYY7XrPHeVFKC6gaEUSuqn)] {
      _bn: <BN: 2c67113ad23394bbaecb45e28eb089f6b8e035bf8116d561aeeddc669cdc00d5>
    },
    quoteMint: PublicKey [PublicKey(AF5pQJq6F5u2gq5durDS6ZgKU3DCoCYKZNzvHHyXPGBp)] {
      _bn: <BN: 8954c2b4e156d4e9b60cb28849a80cc2f517b3e1af595f32990c20cc030788df>
    },
    lpMint: PublicKey [PublicKey(35sZ4xy2wfDZBkfTWCPEPzxfcAWL482gDVaUnryeFVow)] {
      _bn: <BN: 1ef6db811546c7be4cd0ba6fd7ee10b3c1c4c97a5025d4d04043189a11cf06d2>
    },
    openOrders: PublicKey [PublicKey(3GUbTwqAMs9SSJj7xGjP36ZrdvfzxfS9NWeHfBLBGTqs)] {
      _bn: <BN: 21ae497043577bc43ff50052c7d0d24588a80fb612a9776fbe8a7d2ce6254212>
    },
    marketId: PublicKey [PublicKey(ETKaccJ7KvQbm1TdaDW4TzU9ardj14HLXcTX2RVaVB2w)] {
      _bn: <BN: c7e609fec0d4949e03c91797afc00f76f313ea84da3593f426445c64efb29b48>
    },
    marketProgramId: PublicKey [PublicKey(EoTcMgcDRTJVZDMZWBoU6rhYHZfkNTVEAfz3uUJRcYGj)] {
      _bn: <BN: cd0ebc1d9bfaddd6c92de05291d52f7b39929894d00def7441b6834db337be44>
    },
    targetOrders: PublicKey [PublicKey(AagJHUbd4MD6tPUXeNbjnTCtoMPzDa1eZznqKUZhMCT3)] {
      _bn: <BN: 8e59c88ebfeabbaba395298160480ba95ef4ae9895a66fdbbcf0add4414a9bd2>
    },
    withdrawQueue: PublicKey [PublicKey(11111111111111111111111111111111)] {
      _bn: <BN: 0>
    },
    lpVault: PublicKey [PublicKey(11111111111111111111111111111111)] {
      _bn: <BN: 0>
    },
    owner: PublicKey [PublicKey(Adm29NctkKwJGaaiU8CXqdV6WDTwR81JbxV8zoxn745Y)] {
      _bn: <BN: 8f23dd5f0cef66174bc0330afc30a2bc999e235fafeca689e9c80aad792ee793>
    },
    lpReserve: <BN: 16dfb0d0ec00>,
    padding: [ <BN: 0>, <BN: 0>, <BN: 0> ]
  }

Converting to human readable form yields a poolOpenTime of 1712099312557. That time was an hour ago... This is all from my 2nd attempt. I had the same issue before and the poolOpenTime was 18 hours in the past and still threw the same error.

So then I thought maybe the cluster time is way out of sync, but I checked and it's synced with reality.

I've reverse engineered everything and have no idea why I can't swap. It should see that the poolOpenTime is less than the current cluster time and change to AmmStatus::SwapOnly, allowing my swap to go through. Any thoughts appreciated 🤝

raydium-io / raydium-sdk

Pool stuck in `AmmStatus::WaitingTrade`, despite being past `poolOpenTime` #72