TKL 16.1 Odoo issues when TKLBAM restore run.

[l-arnold]

I had an issue that had this included in December 2021 but closed as I had too many issues.

Just doing a site migration with TKLBAM and again all rendering gets lost in a TKLBAM restore.

I worked through this manually back then but need to see if I can do so again.

[JedMeister]

So is this a restore of a v16.1 Odoo backup to a v16.1 server? Or a restore from a different version?

If it's the first, then I can only assume that there is something that your site needs, that isn't being included in the backup (either you're storing stuff in places that ideally you shouldn't, or we're excluding a path from the backup that we shouldn't). There is also a possibility that you need to refresh your caches (I'm not sure how you do that with Odoo, but I'm sure it's possible).

If it's the second (i.e. your backup comes from an earlier TurnKey release, then that's a pain, but my guess is mismatching Odoo versions. Unfortunately, there isn't really a great way around that (other than us not changing Odoo versions or you restoring to the older version). Having said that, once you've updated everything, if you create a new backup of the new (working) Odoo instance, then that should restore cleanly and "just work" when restoring to v16.1.

[l-arnold]

I have had this happen both on the Hub and on Hub to Linode, and Linode to Linode migrations. I am not sure what is not properly migrated. Yesterday I tried, after TKLBAM to then backup the moved database. Delete the moved database, and then move it in. That did not work for me. What did work was deleting the moved database, then moving a manually backed up and restored database from server to server (this was Linode to Linode). This was followed by a 2 cycle restart I recall. It did render in the end.

[l-arnold]

if one goes to (DOMAIN)/web/database/manager you can normally do a backup of the database to your local machine in one of 2 formats... then with the new server, do the same. The risk is, and what I was not able to complete, is if the old machine goes south, it is not possible, at least for me so far, to use only the backed up Database. This is important to work through. This is a 16.1 to 16.1 migration.

[l-arnold]

Easiest way to test is to Build an Odoo Server. Add a second database to it. Run Backup. Then start a new Odoo Server, run TKLBAM restore and see if the data comes in. This is essentially what I did except that I had several months of data in the new database (had deleted the sample database from my original server so of course that does not migrate). I even deleted the sample database from the second database to be sure that was not somehow tripping things (try 2 or 3)

[JedMeister]

This is a 16.1 to 16.1 migration.

Great, thanks for that vital info.

And I assume that you are using the same version of Odoo on both systems? I.e. have you updated Odoo itself between the servers?

if one goes to (DOMAIN)/web/database/manager you can normally do a backup of the database to your local machine in one of 2 formats... then with the new server, do the same. The risk is, and what I was not able to complete, is if the old machine goes south, it is not possible, at least for me so far, to use only the backed up Database. This is important to work through.

So it seems that the DB restore part is the problem? Is there any error message in the restore log(s)? Can you please post the restore log from a machine that experiences this behaviour? Alternatively, do a manual restore from the commandline like this:

tklbam-restore BACKUP_ID | tee ~/tklbam-restore.log

(Where BACKUP_ID is the backup ID of your backup).

Easiest way to test is to Build an Odoo Server. Add a second database to it. Run Backup. Then start a new Odoo Server, run TKLBAM restore and see if the data comes in. This is essentially what I did except that I had several months of data in the new database (had deleted the sample database from my original server so of course that does not migrate). I even deleted the sample database from the second database to be sure that was not somehow tripping things (try 2 or 3)

Ok, thanks for this. I'm not sure when I'll get to it, but that's useful to know how to recreate the issue.

[JedMeister]

Hang on, Odoo uses a postgresql DB - so I think the problem you're experiencing might be this issue: https://github.com/turnkeylinux/tracker/issues/1640

Can you please have a look at that and try the workaround noted there?

We need to do some major work on TKLBAM and I need to complete v17.0 before I'll have the spare cycles to start that work. But that (this?) bug will need to be addressed.

[l-arnold]

16.1 to 16.1 TKL (Which is Odoo 14).
Have had the issue both with a "un updated" Odoo 14 and related attempted Restore and an "updated" Odoo (apt-update) + (apt-upgrade) Odoo system. (Which I did run on the new system as well and with the manual DB it seemed to help) I think one (of a few) issue also has to do with how Odoo Upgrades the Database..

The header on that Issue (1640) sounds similar perhaps, but realize that I have had this happen also on "very virgin systems". It even happened on a very New Hub system back when I first reported this. (November or December 2021).
Your description of "dropping" to allow the Restore is more or less what I was doing manually. It could be done programmatically. Again though, upgrading the database may also be needed.

I AM happy to report that through this I made the mistake of "restoring to my source" which was a VERY BAD Idea. AND tklbam-restore-rollback worked VERY Well! Yay. I had been trying to take a fresh backup and put to a new system but my Shell screens got confused and I restored there. Fixing TKLBAM so you CANNOT restore to the Source SYSTEM seems very a good idea. That rollback command is not well publicized.

[JedMeister]

I have had this happen also on "very virgin systems"

The only requirement of this bug is that the DB schemas don't match. IIRC more specifically, the bug occurs if the new DB has changes to existing table config and/or additional tables. So in the case of Odoo, I imagine that doing anything (of any value) would change the DB schema. I'd need to test further though to be sure.

Fixing TKLBAM so you CANNOT restore to the Source SYSTEM seems very a good idea.

Currently, that is the only scenario that should be (almost) guaranteed to work 100%! So I'm not really sure how to best approach that. Regarding the restore rollback, doesn't TKLBAM explicitly mention the option of rolling back the restore once the restore is completed? IIRC it's one of the last lines of the logging output?! (If it's not, then we should add that!) I'm not really sure how much more than that we could do to ensure that users are aware of it? Although I'm open to ideas if you have suggestions.

FWIW, the expectations of TKLBAM restores is noted on the TKLBAM doc page:

Assuming the exact same server type (e.g. LAMP backup restored to LAMP server) and a pre-tested backup (to ensure that it includes all the required files): A backup should always restore cleanly on the same server. A backup should usually restore cleanly on a server of the exact same version (e.g. v16.1 -> v16.1). A backup should most likely restore cleanly on a server of the same major version (e.g. v16.0 -> v16.1). A backup might restore ok on a server of a single newer major version (e.g. v15.2 -> v16.1) although it may require some tweaking. A backup will almost certainly need some manual intervention if you restore to a server of a (much) newer version (e.g. v14.2 -> v16.1).

[l-arnold]

This comes from an old note I had re updating the database.. I believe the process is looped in many places in Odoo but not in TKLBAM... perhaps it could be applied once we know what it actually does

`Odoo Postgress -update=all

Upgrade the database. For linux, start the odoo server with the –update=all parameter. For windows, as an administrator open the command prompt (cmd). Change to the odoo server directory and start the server with odoo-server.exe –update=all to update the modules. This process will take a short while (1-2minutes max) and the database won’t be accessible by the client`

[JedMeister]

That's good info, thanks Landis.

My suspicion is that running that command will only make changes if it needs to. So running that on restore shouldn't do any harm. Can you please double check?

If that's the case, we can make a tklbam restore hook script that runs that after restore.

[l-arnold]

Ok. Will do soon.

My thought is that for "in place" restores, the Database itself be "renamed" and left in place, then the Restored Database be written in to the system. It may be however that because of Postgres there also be "full backup" and "full restore"... I'll see how an in-place restore goes to a secondary server after "renaming" the database on it first.

[l-arnold]

Well, I spent a little time today. I have an active Odoo install and an Odoo that I created a few months ago and brought forward (described above). Today I tried a simple "tklbam-restore (backup#)" Two times it ran through the process (did not work the first time). It appears that it is updating the database, but it is not.

In order to bring the system up to date one needs to run the Odoo Database Manager on the Running System... Do a Backup, Then go to the Restore To System, Delete the database there, then Restore the newly backed up database.

I cannot get it to restore in any other way. Maybe I am missing something but i have done this before.

I had thought that since I had done this process a few month's ago the database could just be updated. Perhaps there is some encrypted id for the Database that does not match the one I am trying to restore to. Seems strange though. Everything should be identical.

All for now.

[l-arnold]

Ok. I have likely noted some of this earlier but want to sumarize.

If a new TKL Odoo 16.1 App is installed, then TKL-BAM restore run to populate it next one must

• Delete the Demo Database
• perhaps delete the restored database (ref the "loses rendering" part of this issues original description)
• manually upload a database (saved by Odoo) on original server. (this is basically a Dump file but works)

If in a previously restored and now working (by following the above) server which you want to bring "up to date" with a subsequent TKLBAM restore, the following occurs:

• Files are updated up to current
• TKBAM restore of the database (same name) over the running database (same name) will cause many many errors visible during the restore process in SSH/Webshell etc. This includes module set "constraints" visible in the screenshot attached.
• the restored database (if it was restored over and not excluded by a tklbam switch) will need to be deleted.
• (last line in outline above again) manually upload a database (saved by Odoo) on original server. (this is basically a Dump file but works)

Don't crash your server. Perhaps the rendering issue stated initially and above works itself out over time. Seems that it did one time recently.

OK. working through this in parallel Screenshot below from the end of tklbam-restore scroll.

as I write this.

[l-arnold]

Just tried the approach to simply First Delete the Database in Odoo Database Manager then run TKLBAM "restore". Basically confirms that "rendering" does not work. I believe that certain assets are not being brought in with the tklbam restored database.

Here is a screenshot of "unrendered" after restore Odoo.

After Odoo Database trans this is more of the rendering (plus some rectangles) that should be there.

Moving the files, TKLBAM is working well. We just need to not do the incremental database as it seems the content of the data is not complete. Even odoo has 2 options for how to backup the data and the ZIP level is the one that is complete.

[l-arnold]

This looks to be a "filestore" issue related to the restored database. This video shows restoring a database from within Postgresql, but also running into the same issue outlined here, then fixing it by moving the "filestore".

https://www.youtube.com/watch?v=XrPURF5my9s

I am not sure if TKLBAM is indeed placing the filestore in the wrong place with the restore process, but seems likely.

I note that when a manual backup takes place there is a Filestore Folder within the zip file that should be properly placed in restore process (as yet unposted issue w/ 16.1-17.1 migration is related)

[JedMeister]

Thanks for posting screenshots Landis (sorry I missed them last year - it was a hectic year...).

FWIW, the one that you note "isn't rendering", that looks to me like missing CSS (i.e. html styling). In modern apps, it's not uncommon for CSS to be dynamically generated on the fly and cached on the system. My guess is that the cache isn't included in the backup, so the cache would need to be refreshed (so new dynamic CSS can be generated). Often there is a command that can be run, or a button in the app ui (in admin config or similar).

Having said that, looking at the Odoo backup profile, it's not including or excluding anything (the new v17.1 release does, but not the older one). So perhaps it is storing stuff somewhere that isn't being backed up?!

Thanks for sharing that video, that certainly does seem to be consistent with the idea that there is something missing...

Could you perhaps share a backup with me (privately) so I can test it myself. To do that (without having to share access to your Hub account or your server):

mkdir /root/tklbam-dump
tklbam-backup --dump=/root/tklbam-dump
tar -cvf /root/tklbam-dump.tar.gz /root/tklbam-dump
rm -r /root/tklbam-dump

Then upload the /root/tklbam-dump.tar.gz file somewhere so I can download it. If you have a google account, you could upload it to google drive and share a link to it with me privately? My personal gmail account is jeremy jeremydavis org. Otherwise, use whatever service works for you and send me a link privately (my turnkeylinux.org email is a fine).

Also, as a total aside Odoo is now packaged in Debian and we've decided to switch to that for v17.x. There are pros and cons for going that way, but one of the big pros is that it will stay a consistent version for the life of a major TurnKey release (e.g. v17.x) and will get auto Debian security updates. It's also means tons less maintenance for us and hopefully might even make migration between Odoo versions easier/better? (Debian often do a pretty good job of handling those).

[l-arnold]

Hi Jeremy,

TKL Odoo 17.1 has Postgresql Filestore in:
/var/lib/odoo/.local/share/Odoo/filestore (then the database name)

TKL Odoo 16.1 has Postgresql Filestore in /var/lib/odoo/.local/share/odoo/filestore (then the database name)

I am pretty sure the individual database "filestore folders" are not being restored properly. There is a separate folder for each database that may be on the system. The "Turnkeylinux Example" database which is installed at install time may be elswhere (seems I saw something somewhere else when I was looking for these above)

We can communicate on side channels to find a way to look at this.

Landis

[JedMeister]

The v17.1 release does have that path included in the backup as it's a fairly standard Debian package location (I explicitly looked for it there). But prior to v17.x, that path wasn't included! That must be the issue.

TBH I'm a bit surprised that the upstream Odoo package (that we used in v16.x) puts it there?! Considering it doesn't use standard Debian install locations (it installs Odoo to /opt/odoo; usually it should be installed to /usr/lib/odoo or /usr/share/odoo) I would have expected the data to have been stored in a subdir of /opt/odoo. Anyway, that's clearly not the case and I probably should have checked a bit deeper.

Hopefully you still have the original (working) server still running? If so, add that path to /etc/tklbam/overrides. E.g.:

echo "/var/lib/odoo/.local/share/odoo" >> /etc/tklbam/overrides

Then rerun your backup.

That will ensure that /var/lib/odoo/.local/share/odoo (and all contained sub-directories and files) are included in your backup.

Give it a test and let me know how it goes.

As for the DB bug, I think that the way to go might be to create a hook script to do a DB dump (to the restore rollback storage directory if possible - just in case things go wrong), then drop the DB, and load the backup. In the meantime, I'd be happy to assist you to to create a hooks script to manually do that (although you'd need to manually restore teh DB if you need to do a "rollback".

[l-arnold]

I will work on the overrides there Jeremy. Yes, original server is still working. I will also look to see if the overrides change on a 17.1 server when a "restore to" has been run on that server. I will also check.

Back soon.

[l-arnold]

So I am not seeing anything in 16.1 Overrides (webmin) or 17.1 (post restore) Overrides file. I will boot a new temp server to see how it should look.

[l-arnold]

Quite strange digging into over rides in Webmin. First they show blank. Then if you click on the "backup profile" you get a populated file. But if you want to "save" a change you cannot. Then you go "back" it is indeed populated but it is a different file (no Odoo at the bottom to start with).

aiming with File Manager next but will run your Echo command first. echo "/var/lib/odoo/.local/share/odoo" >> /etc/tklbam/overrides

[l-arnold]

Well, I ran restore but the databases did not come in even though the override above was set in the 16.1 server (just before my backup). Restoring to 17.1 Server. As it was though, I actually got zero database coming in while in the past I had gotten elements of the database coming in.

I do note that first "restore" changed my root login and cert keys, so something definitely came in. I am running a second "restore" now in case somehow helps....
Well I see the following DB Error in TKLBAM:

DATABASE - Unserializing PgSQL databases from /tmp/tklbam-uCC7ap/TKLBAM/pgfs
============================================================================

database: test
pg_restore: error: one of -d/--dbname and -f/--file must be specified
database: root
pg_restore: error: one of -d/--dbname and -f/--file must be specified
database: mydatabase
pg_restore: error: one of -d/--dbname and -f/--file must be specified

[JedMeister]

Sorry to hear about the Webmin weirdness. For what it's worth, for v17.x, I learned some basic perl and did some major work on the Webmin TKLBAM module - which fixed a raft of bugs. But unfortunately, it's not available in v16.x.

So to eliminate the possibility that the weirdness you are experiencing is due to Webmin module bugs, I urge you to use the CLI.

FWIW, the default content of the overrides file can be found here.

As to the error, all I can suggest is a hook script (to drop the DB before restore) as I'm sure I've mentioned before). I'm pretty under the pump at the moment, but if I can find a minute, I'll try putting something together for you to try out.

Having said that, I'm pretty sure that those messages look different to what I've seen before?

[l-arnold]

Hi Jeremy, It is a restore of a 16.1 to 17.1 server.

Old Server is 16.1 new Server is 17.1

I have not tried a 16.x restore with the new override.

The issue did however start as a 16.1 to 16.1 restore.

We could put that in place if you want. Not sure if I have time just now but might.

Landis

From: "Jeremy Davis" @.> To: "turnkeylinux" @.> Cc: "Landis Arnold" @.>, "Author" @.> Sent: Sunday, May 1, 2022 10:42:56 PM Subject: Re: [turnkeylinux/tracker] TKL 16.1 Odoo looses proper rendering when TKLBAM restore run. (Issue #1726)

So is this a restore of a v16.1 Odoo backup to a v16.1 server? Or a restore from a different version?

If it's the first, then I can only assume that there is something that your site needs, that isn't being included in the backup (either you're storing stuff in places that ideally you shouldn't, or we're excluding a path from the backup that we shouldn't). There is also a possibility that you need to refresh your caches (I'm not sure how you do that with Odoo, but I'm sure it's possible).

If it's the second (i.e. your backup comes from an earlier TurnKey release, then that's a pain, but my guess is mismatching Odoo versions. Unfortunately, there isn't really a great way around that (other than us not changing Odoo versions or you restoring to the older version). Having said that, once you've updated everything, if you create a new backup of the new (working) Odoo instance, then that should restore cleanly and "just work" when restoring to v16.1.

— Reply to this email directly, [ https://github.com/turnkeylinux/tracker/issues/1726#issuecomment-1114492861 | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/ABLJD2FFI5FYFGHQ6YW6FJLVH5MNBANCNFSM5UT4IBPA | unsubscribe ] . You are receiving this because you authored the thread. Message ID: @.***>

[JedMeister]

It is a restore of a 16.1 to 17.1 server.

Old Server is 16.1 new Server is 17.1

I have not tried a 16.x restore with the new override.

The issue did however start as a 16.1 to 16.1 restore.

We could put that in place if you want. Not sure if I have time just now but might.

Ah ok. Thanks for the additional info. That's likely why I didn't recognise that error - it's likely a different (albeit possibly related) issue.

[JedMeister]

Hey Landis, my deepest apologies that I dropped the ball on this one. FWIW another user also reported it (against v17.x: #1867) and highlighted a specific error message (pg_restore: error: one of -d/--dbname and -f/--file must be specified) that I had missed in your report. Following on that lead I'm pretty sure that I was able to resolve that issue.

So far I have only rebuilt for v17.x so you'll need to start with a v17.x server to test it, but please do and let me know how you go. Check it out here: https://github.com/turnkeylinux/tklbam/releases/tag/v1.4.3.1