[BUG] NLU loses its training after updates and/or unuse.

botpress / nlu

This repo contains every ML/NLU related code written by Botpress in the NodeJS environment. This includes the Botpress Standalone NLU Server.

23 stars 21 forks source link

[BUG] NLU loses its training after updates and/or unuse. #71

Closed hacheybj closed 1 year ago

hacheybj commented 3 years ago

I have some chatbots with very low usage.

I see two things happening:

When I update Botpress, I have to manually train my chatbots again... Training is lost?
After long periods of inactivity, I have to manually train my chatbot... Training is lost!

franklevasseur commented 3 years ago

Hi,

@hacheybj I must say you are great at finding bugs and this is really annoying 😆

The first point you're mentioning is not a bug, but a feature. When you update Botpress, you also update the NLU version which triggers an invalidation of your models. This mechanism allows us to break nlu models and is the equivalent of doing a migration.

The second point is what bothers me... I believe @sebburon has already observed this issue in the past.

What version of Botpress are you using ? Can the bot still answer your messages with proper NLU predictions or is the model completely gone ?

Anyway thanks for reporting this issue, I'll investigate a bit more and keep an eye open.

François

sebburon commented 3 years ago

Yes, I did have the same issues. I will try and keep information on the version and everything next time I encounter this issue.

hacheybj commented 3 years ago

For point 1.

I have over 20 bots to maintain "live", and I update botpress weekly.

Is there a way to trigger training on all bots within an environment (potentially sequentially and in all languages), rather than enter manually to hit train?
I may even like to have an indicator in the main UI next to the bot name that displays it being non-trained. I have to enter each to see the state they are in.

hacheybj commented 3 years ago

For point 2.

I keep botpress updated with the latest tag soon as it's out. I've seen this in multiple versions. I'll follow up here with a version next time it happens
Its like the model is completely gone (as if we never hit train)
I did not verify if there were any models left in the models folder, I'll look at that as well

franklevasseur commented 3 years ago

For point 1.

There is an option in nlu module config file which is disabled by default that goes something like trainOnBotMount or trainOnServerStart.

For point 2.

Can you talk to the bot ? Is there NLU predictions when you chat with it ?

franklevasseur commented 2 years ago

Yo @hacheybj,

As we've discussed privately today, I'll close the issue for the following reasons:

1 - When the engine version changes, it is expected that your bot needs to retrain. 2 - The fact that your model is deleted randomly is very problematic, but it has not been reported or reproduced since.

If you have any updates, please open a new issue.

François

hacheybj commented 2 years ago

Having this issue with 12.26.12 now.

I've seen:

No server reboots nor crashes
No bot unmounts/mounts
No server updates (constant version of 12.26.12)
Only correlation: long periods of inactivity
Shortest period so far has been 4 days

environment

botpress in official docker image (container)
Single node environment
sqLite - bpfs = fs && postgres - bpfs = db
12.26.12