botpress / studio

The studio is the main interface you'll use to build and edit your chatbot.
https://botpress.com/docs/quickstart#conversation-studio
38 stars 38 forks source link

fix(studio-be): fix cache invalidation being discarded by flow service #229

Closed laurentlp closed 2 years ago

laurentlp commented 2 years ago

This PR should fix an issue where overriding a bot with the same id (or uploading a bot with an id that already existed) would result in having outdated flow content.

Here is the reason why we should check for matches with and without the prefix /data: botpress/studio/#2

This endpoint (/invalidateFile) is called whenever there is an invalidation in Botpress. Since we remove the data/ part from the key when running using SQLite, the flow-service would discard the cache invalidation resulting in outdated flow data being used.

I also updated the config loader and hint service to make sure that they watch for both keys containing or not the data/ part.

Closes https://github.com/botpress/v12/issues/1581 Closes DEV-2203

Testing

  1. To reproduce the issue, create an on_stage_request hook using this code:
  const STAGE_SEPERATOR = '__'
  const VERSION_SEPERATOR = ' :: v'
  const botId = bot.id.split(STAGE_SEPERATOR)[0]
  const STAGE_KEY = `${botId}-PipelineVersion`
  const botLogger = bp.logger.forBot(bot.id)

  const getPromoteVersionStage = () => {
    if (pipeline.length < 2) return 'NEVER'
    let stage2 = pipeline[1]
    return stage2.id
  }

  const getNextVersion = async () => {
    const promoteVersion = getPromoteVersionStage() === bot.pipeline_status.stage_request.id

    let globalKvs = bp.kvs.global()
    let storeExists = await globalKvs.exists(STAGE_KEY)
    let curVersion = storeExists ? await globalKvs.get(STAGE_KEY) : 0

    if (!promoteVersion) return curVersion

    let nextVersion = curVersion + 1

    await globalKvs.set(STAGE_KEY, nextVersion)

    return nextVersion
  }

  const getStageId = () => [botId, bot.pipeline_status.stage_request.id].join(STAGE_SEPERATOR)

  const getStageName = async () => {
    let botName = bot.name.split(VERSION_SEPERATOR)[0]
    let nextVersion = await getNextVersion()

    return `${botName}${VERSION_SEPERATOR}${nextVersion}`
  }

  const handler = async () => {
    try {
      let stageId = getStageId()
      let stageName = await getStageName()

      bot.id = stageId
      bot.name = stageName
    } catch (err) {
      botLogger.error(JSON.stringify(err, null, 2))
    }
  }

  return handler()
  1. Make sure you add at least one other stage to your Bot Pipeline.
  2. Create an empty bot and add two nodes with content to it.
  3. Promote your bot to the next stage.
  4. Both bots should work properly.
  5. Remove one of the nodes in the original bot.
  6. Promote it again
  7. This time the promoted bot will still display the deleted node.

You can then repeat these steps to see that the issue is not resolved.

laurentlp commented 2 years ago

@allardy I need more details on why we have to handle this when we call /invalidateFile :

if (process.BPFS_STORAGE === 'disk') {
  key = key.replace('::data/', '::')
}

We might want to move this logic inside a util function so that all services listening to invalidation can select the files they want to handle accordingly

linear[bot] commented 2 years ago
DEV-2203 (BUG) Overriding a bot on a stage breaks DialogEngine and fails to update content-elements (botpress/botpress botpress/v12#1581)

**Describe the bug**\ In a workspace with 3 pipeline stages when a bot on some stage is overridden (using `on_request_change` pipeline hook below): 1. Newly added contents are not visible in the flow editor and contents page of the bot\ Screen Shot 2021-12-23 at 00 37 06 2. Chat stops working\ Screen Shot 2021-12-23 at 00 35 48 May be related to [this](https://github.com/botpress/v12/issues/1569) issue. **To Reproduce**\ Steps to reproduce the behavior: 1. Set up a pipeline with 3 stages 2. Create an empty bot\ Screen Shot 2021-12-23 at 00 27 33 3. Add some initial content to it\ Screen Shot 2021-12-23 at 00 29 48 4. Promote the bot to the next stage\ Screen Shot 2021-12-23 at 00 30 13 5. Promoted bot should work for now\ Screen Shot 2021-12-23 at 00 31 49 6. Add some new content to the original bot and promote it again\ Screen Shot 2021-12-23 at 00 32 35 7. Open the studio for the promoted bot. You should see newly added content-elements rendered empty and webchat not working\ Screen Shot 2021-12-23 at 00 46 07\ Screen Shot 2021-12-23 at 00 35 48 **Expected behavior**\ The promoted bot should have a working webchat and render newly added content-elements in the flow editor **Environment (please complete the following information):** * OS: macOS * Browser Chrome * Browser Version Version 96.0.4664.110 (Official Build) (x86_64) * Botpress Version v12.26.7 * NodeJS v12.18.1 **Additional context**\ Add any other context about the problem here. [botpress/borpress botpress/v12#1581](https://github.com/botpress/v12/issues/1581) by [@ daukadolt](https://github.com/daukadolt)