phil31 commented 4 years ago

Dear All,

happy new year of course !

PartKeepr Version: 1.4 JavaScript errors: No. PHP 7.1.33-1+0~20191026.27+debian10~1.gbpd10139 Apache 2.4.38 on Debian 10 mariaDB 10.3.17-MariaDB-0+deb10u1 - Debian 10

here it is a part of my partkeepr.log : https://pastebin.com/143iMwDy

I tried the wiki to import my data from 1.09 to 1.4.

I copied the data directory (without proxies) and the config.php as indicated. I updated the rights and relaunched web / setup. everything is green and I have import information.

now I can access Partkeepr and identify myself. i have always the message "Your database schema is not up-to-date! Please re-run setup immediately!" and none of my old components are there!

something shocked me .. I did nothing at the level of the database! do I have to import something into PhpMyAdmin? How? 'Or' What ?

Thank you for your help

baradhili commented 4 years ago

I know default partkeepr build asks for mysql.. sometimes mariadb version response alone can confuse some setup scripts.. have you tried this with mysql instead of mariadb?

christianlupus commented 4 years ago

The problem is that the MariaDB >= 10.3 is not supported by the currently used Symphony 2 framework. As a result, no correct database connection can be made. I am surprised that @phil31 could even start the import as I think he needs to finish the setup beforehand.

Definitively, we need more information, what he did and when it went south.

phil31 commented 4 years ago

Hello christian,

i'm Phil31 !!

which information you need ? i would be happy to help on this great project, but i'm totaly newbie with Linux stuff, so please explain what you expect !

yes i totaly finish the setup, all green. my issues are

"database schema is not up-to-date" : i understand that it is an incompatibility between mariaDB and Doctrine ==> i can setup a new VM using mySQL if that solve the issue.
i need to import all components from my old 0.1.9 to 1.4.0 : the wiki page don't help me enough. ==> i did it and when i open the 1.40, i dont see my components from the 0.1.9 but as explain i'm really surprise that i don't do any export with phpMyadmin from my 0.1.9 then import to my new VM .. is that normal ?

best regards phil

----- Mail original ----- De: "Christian" notifications@github.com À: "partkeepr/PartKeepr" PartKeepr@noreply.github.com Cc: "phil31" pognon.chagny@free.fr, "Mention" mention@noreply.github.com Envoyé: Mercredi 15 Janvier 2020 11:19:59 Objet: Re: [partkeepr/PartKeepr] problem to import components from 1.09 to 1.4 (#1063)

The problem is that the MariaDB >= 10.3 is not supported by the currently used Symphony 2 framework. As a result, no correct database connection can be made. I am surprised that @phil31 could even start the import as I think he needs to finish the setup beforehand.

Definitively, we need more information, what he did and when it went south.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe .

christianlupus commented 4 years ago

which information you need ? i would be happy to help on this great project, but i'm totaly newbie with Linux stuff, so please explain what you expect !

OK, no problem here. First we need to understand what your steps were in detail. I have some idea from your statements. Please correct me if I was wrong:

You had a PartKeepr (PK) version 0.1.9 successfully running
You tried to install in a VM PK 1.4.0, here you used MariaDB 10.3.17 as storage.
Upon login after the setup you get the Database is not up-to-date message.
You then exported the database from your old PK-0.1.9 installation (probably via phpMyAdmin)
You imported the DB dump using phpMyAdmin into the DB of PK-1.4.0
PK allows you to log in but none of the parts are there.

Is this roughly the process you did?

yes i totaly finish the setup, all green. my issues are

"database schema is not up-to-date" : i understand that it is an incompatibility between mariaDB and Doctrine ==> i can setup a new VM using mySQL if that solve the issue.

As long as you try with MariaDB 10.3 or later you will run into issues as long as there is not update to PK available. This needed update is going to be delayed for now, sorry. So it might be best go with an older version of MariaDB or MySQL/PostgreSQL for now.

i need to import all components from my old 0.1.9 to 1.4.0 : the wiki page don't help me enough. ==> i did it and when i open the 1.40, i dont see my components from the 0.1.9 but as explain i'm really surprise that i don't do any export with phpMyadmin from my 0.1.9 then import to my new VM .. is that normal ?

I have to admit, that I did not do such an update personally yet. So I cannot 100% sure tell you what will happen but you should for sure not loose all your parts. I suspect that the problem is that you are still using the unsupported MariaDB version. As long as you do not change that, you have basically no functional DB storage. If you have another DB in place (preferably MySQL) import your dump there. Then you can rerun web/setup as documented.

So let's get this sorted out piece by piece. Fist step is that you describe what you did exactly and where exactly something went wrong.

phil31 commented 4 years ago

well actually : i have a PC which run PK 0.1.9 i have a VM wich run PK 1.4.0 - DEBIAN10 / Apache 2.4.38 / mariaDB 10.3.17 (i dont know why this is this version and not the mentioned one in the wiki !?)

1/ on PK1.4.0 VM, i succesull follow the setup, all was green; but each time i logon, i get the message "Database is not up-to-date message" i can manually add a component i try differents tips found on the web, no one was succesful :( (anyway, the PK is used only in my LAN, no access from outside - so i can live with that if that is not a real problem)

2/ i follow the wiki to update/upgrade from 0.1.9 to 1.4.0 VM ( https://wiki.partkeepr.org/wiki/How_to_update ) but anywhere i can found any mention to export the database from 0.1.9 and import it to 1.4.0 i only copy the differents directory and config.php, as describe in the wiki page.

so yes, my components database is safe on the previous/old PC maybe i just need to import the database in the new VM ? what is your advise please ? setup a new VM, using DEBIAN9 and mariaDB 10.1 ?

thanks christian for your support ! regards

----- Mail original ----- De: "Christian" notifications@github.com À: "partkeepr/PartKeepr" PartKeepr@noreply.github.com Cc: "phil31" pognon.chagny@free.fr, "Mention" mention@noreply.github.com Envoyé: Mercredi 15 Janvier 2020 17:15:51 Objet: Re: [partkeepr/PartKeepr] problem to import components from 1.09 to 1.4 (#1063)

which information you need ? i would be happy to help on this great project, but i'm totaly newbie with Linux stuff, so please explain what you expect !

OK, no problem here. First we need to understand what your steps were in detail. I have some idea from your statements. Please correct me if I was wrong:

* You had a PartKeepr (PK) version 0.1.9 successfully running 
* You tried to install in a VM PK 1.4.0, here you used MariaDB 10.3.17 as storage. 
* Upon login after the setup you get the Database is not up-to-date message. 
* You then exported the database from your old PK-0.1.9 installation (probably via phpMyAdmin) 
* You imported the DB dump using phpMyAdmin into the DB of PK-1.4.0 
* PK allows you to log in but none of the parts are there.

Is this roughly the process you did?

yes i totaly finish the setup, all green. my issues are

* "database schema is not up-to-date" : i understand that it is an incompatibility between mariaDB and Doctrine ==> i can setup a new VM using mySQL if that solve the issue.

As long as you try with MariaDB 10.3 or later you will run into issues as long as there is not update to PK available. This needed update is going to be delayed for now, sorry. So it might be best go with an older version of MariaDB or MySQL/PostgreSQL for now.

* i need to import all components from my old 0.1.9 to 1.4.0 : the wiki page don't help me enough. ==> i did it and when i open the 1.40, i dont see my components from the 0.1.9 but as explain i'm really surprise that i don't do any export with phpMyadmin from my 0.1.9 then import to my new VM .. is that normal ?

I have to admit, that I did not do such an update personally yet. So I cannot 100% sure tell you what will happen but you should for sure not loose all your parts. I suspect that the problem is that you are still using the unsupported MariaDB version. As long as you do not change that, you have basically no functional DB storage. If you have another DB in place (preferably MySQL) import your dump there. Then you can rerun web/setup as documented .

So let's get this sorted out piece by piece. Fist step is that you describe what you did exactly and where exactly something went wrong.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe .

christianlupus commented 4 years ago

well actually : i have a PC which run PK 0.1.9 i have a VM wich run PK 1.4.0 - DEBIAN10 / Apache 2.4.38 / mariaDB 10.3.17 (i dont know why this is this version and not the mentioned one in the wiki !?)

You are referring to this wiki page? This is using stretch (aka Debian 9) while you are using Debian Buster (aka Debian 10). Thus there is a difference in the versions provided with the releases.

1/ on PK1.4.0 VM, i succesull follow the setup, all was green; but each time i logon, i get the message "Database is not up-to-date message" i can manually add a component i try differents tips found on the web, no one was succesful :( (anyway, the PK is used only in my LAN, no access from outside - so i can live with that if that is not a real problem)

I doubt this is working completely. I would not guarantee that the created parts are correctly saved to DB. In fact, the DB is not correctly connected at all. Otherwise the message about the not up-to-date would vanish.

2/ i follow the wiki to update/upgrade from 0.1.9 to 1.4.0 VM ( https://wiki.partkeepr.org/wiki/How_to_update ) but anywhere i can found any mention to export the database from 0.1.9 and import it to 1.4.0 i only copy the differents directory and config.php, as describe in the wiki page.

It is the first part:

Create a database backup of your PartKeepr installation

As you have set up everything it is assumed, you know how to handle things like a db backup.

Nevertheless, I will try to help here. You need to know the credentials to access the database (user and password like in phpMyAdmin) and the name of the database (probably something like partkeepr).

Before I type a hell of commands here: Do your host run linux or windows?

so yes, my components database is safe on the previous/old PC maybe i just need to import the database in the new VM ? what is your advise please ? setup a new VM, using DEBIAN9 and mariaDB 10.1 ?

I am not yet sure what might be best. Personally I like the variant using docker and docker-compose most. This allows very fine grained configuration of the various parts that interact as well as their versions.

Would you be ok with that? I could test it out here and give you the commands to redo things....

phil31 commented 4 years ago

My PK 1.4.0 VM is running under Hyper-V on a new windows server 2019.

I think the best is to create a new VM with Debian 9, as the wiki explain, then I will be back to you when it’s done.

And if you can, you will assist me to import the old database to the new.

I hope than during the install, it will correctly install MariaDB 10.1 !

because I can only follow what the wiki explain ..

if i get an error, i'll be like an idiot ... a penguin in front of a keyboard: o)

sorry for my poor English, but when you write : “It is the first part: Create a database backup of your PartKeepr installation”…

I don’t understand that I need to export the database then import in the new PK !

I understand that I need to create the backup for purpose security .. but as I work on a new VM, not on the original PC ..

I mean “bypass this sentence” !

Thanks for your time, will be back to you soon when the new VM is ready !

Regards

Phil

De : Christian notifications@github.com Envoyé : mercredi 15 janvier 2020 21:30 À : partkeepr/PartKeepr PartKeepr@noreply.github.com Cc : phil31 xxxx.xxxx@free.fr; Mention mention@noreply.github.com Objet : Re: [partkeepr/PartKeepr] problem to import components from 1.09 to 1.4 (#1063)

well actually : i have a PC which run PK 0.1.9 i have a VM wich run PK 1.4.0 - DEBIAN10 / Apache 2.4.38 / mariaDB 10.3.17 (i dont know why this is this version and not the mentioned one in the wiki !?)

You are referring to this wiki page https://wiki.partkeepr.org/wiki/PartKeepr_on_Debian_%22Stretch%22 ? This is using stretch (aka Debian 9) while you are using Debian Buster (aka Debian 10). Thus there is a difference in the versions provided with the releases.

1/ on PK1.4.0 VM, i succesull follow the setup, all was green; but each time i logon, i get the message "Database is not up-to-date message" i can manually add a component i try differents tips found on the web, no one was succesful :( (anyway, the PK is used only in my LAN, no access from outside - so i can live with that if that is not a real problem)

I doubt this is working completely. I would not guarantee that the created parts are correctly saved to DB. In fact, the DB is not correctly connected at all. Otherwise the message about the not up-to-date would vanish.

2/ i follow the wiki to update/upgrade from 0.1.9 to 1.4.0 VM ( https://wiki.partkeepr.org/wiki/How_to_update ) but anywhere i can found any mention to export the database from 0.1.9 and import it to 1.4.0 i only copy the differents directory and config.php, as describe in the wiki page.

It is the first part:

Create a database backup of your PartKeepr installation As you have set up everything it is assumed, you know how to handle things like a db backup.

Nevertheless, I will try to help here. You need to know the credentials to access the database (user and password like in phpMyAdmin) and the name of the database (probably something like partkeepr).

Before I type a hell of commands here: Do your host run linux or windows?

so yes, my components database is safe on the previous/old PC maybe i just need to import the database in the new VM ? what is your advise please ? setup a new VM, using DEBIAN9 and mariaDB 10.1 ?

I am not yet sure what might be best. Personally I like the variant using docker and docker-compose most. This allows very fine grained configuration of the various parts that interact as well as their versions.

Would you be ok with that? I could test it out here and give you the commands to redo things....

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/partkeepr/PartKeepr/issues/1063?email_source=notifications&email_token=ACJOE2ULZ5UA7ALH3WJOXYDQ55W5RA5CNFSM4KCCRGW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJBWPXI#issuecomment-574842845 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJOE2RPEWXSZYUGOSG6ENDQ55W5RANCNFSM4KCCRGWQ . https://github.com/notifications/beacon/ACJOE2SN7YOX4M2XU7TBCLDQ55W5RA5CNFSM4KCCRGW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJBWPXI.gif

phil31 commented 4 years ago

Well Christian

i have setup a new VM PK 1.4.0 / debian9 / mariaDB 10.1 and no more trouble when i logon :o)

now please let me know the more detailled manner to import the stuffs from 0.1.9 to 1.4.0. i already export my .sql database from my old PC. does the different fields, from the previous database, compatible ?

thanks for your support, regards

christianlupus commented 4 years ago

OK, then let's start this. There should be an automatical update of the schema version during the setup method. My suggestion is that you clone the VM to be save if something is going wrong.

It would be best, if we start from scratch here.

Database import

Does your database dump has DROP TABLE statements in it? If not, you will have to issue manual commands before continuing. Write here and wait for comments.
Stop the partkeepr instance (or do not use it) during the update process.
In your VM issue the command mysql -u root -p partkeepr < dump.sql, replacing root with the mysql user name you installed, partkeepr with the name of the created database and dump.sql the filename of the dump of the old database. This will import the dump into the database.

Filesystem import

On your old machine make a backup of the data/ directory. For example call tar czf /tmp/data.tgz data/ in the old partkeepr installation folder to create /tmp/data.tgz.
Copy the file to the VM to /tmp.
Go to the /tmp folder and untar the file you just copied: tar xzf data.tgz.
Check if there is a folder /tmp/data/proxies. If yes, remove it.
Go to the PK folder.
Copy the folder /tmp/data over the existing data folder and adopt the rights accordingly.
Copy the config.php from your old installation to the new installation (partkeepr root folder).
Remove the file app/config/parameters.php in the new installation.

Migration of the data

Start the web server if you stopped it.
Go in a browser to the address http://IP-OF-VM/web/setup.
Continue the setup as normal.
You should be done.

christianlupus commented 4 years ago

There was quite some back-and-forth using mails. I will insert the mails here to keep track of the progress.

christianlupus commented 4 years ago

From 01/19/20 22:50

Christian, I will try tomorrow what you explain me to import the database. But I’m not sure to right understand this sentence : “Does your database dump has DROP TABLE statements in it? If not, you will have to issue manual commands before continuing. Write here and wait for comments.“ How to verify it ? Or what is the right method to export the base from my old PC ? I can access to the base with phpMyAdmin. Thanks, phil

christianlupus commented 4 years ago

From 01/20/20 10:36:

Hello Phil,

But I'm not sure to right understand this sentence :

"Does your database dump has DROP TABLE statements in it? If not, you will have to issue manual commands before continuing. Write here and wait for comments."

How to verify it ?

You said you created already a dump of the database (typically a *.sql file). Just open it in a plain text editor. It contains just plain text.

The language you see is called SQL. Depending on the setting you gave when creating the dump, there are various parts omitted. The file contains blocks like this (here for table foo):

DROP TABLE IF EXISTS foo;
CREATE TABLE foo IF NOT EXISTS (...);
INSERT INTO TABLE foo (...) VALUES (...), (...), ... (...);

The blocks might get lengthy. So do not be surprised if you get problems to manually spot them completely. Simply look for the first CREATE TABLE statement. Take the name of the table and look quite above the line if there is some DROP TABLE before. This will clean out any preexisting data from the database (necessary).

I hope you get the idea.

Christian

christianlupus commented 4 years ago

From 01/20/20 11:07:

Hi,

again, thanks for your patience and time to help me ! here it is my dump file. i don't see any "DROP TABLE" sentence maybe i do badly the export ! i don't know too much things on SQL stuffs ..?

FIY, the old PK 0.1.9 was installed on a Windows server with IIS server ! .. so we can see some paths with "C:\inetpub\wwwroot". that maybe bad for the import step ..? i remember that it take me long time to successful install it, but finaly it was perfectly working for several years ! i had write some tutorial on my web site : https://eko-fpga.com/index.php/partkeepr

regards

christianlupus commented 4 years ago

From 01/20/20 12:38:

Hello Phil,

I changed the SQL file by adding the DROP TABLE commands. No big deal there with regular expressions.

The paths are in the cache, I would try what happens if the warming cache step in the setup manages this or if there will be a problem with it. In the latter case we can restart and adopt the paths in the file by regex replacing as well (I need the installation path on the new machine then).

Cheers Christian

christianlupus commented 4 years ago

From 01/20/20 14:43:

the new path is "/var/www/partkeepr-1.4.0/"

well, may i try now to import, as you describe in the thread : https://github.com/partkeepr/PartKeepr/issues/1063 let me know ?

i will copy the VM before anything !

thanks

christianlupus commented 4 years ago

From 01/21/20 10:23:

Hello Phil,

yes, try to go forward. I do not have your data, so I cannot try it myself. I can only advice you the best I can.

If something odd happens you are welcome to come back to me.

Sorry for the late answer Christian

PS: Keep me updated so we can close the issue once you are done with it.

christianlupus commented 4 years ago

From 01/21/20 14:32:

Christian,

i try the first thing .. already some trouble :
root@PARTKEEPR1:/home/partkeepr# mysql -u Partkeepr -p Partkeepr < 20190915_eko_partkeepr_bdc_new.sql
Enter password:
ERROR 1005 (HY000) at line 4295: Can't create table `Partkeepr`.`#sql-229_5e` (errno: 121 "Duplicate key on write or update" )
and now, i just try to login in PK .. can't anymore :((( i'm tired .. i don't know what to do !

phil

christianlupus commented 4 years ago

From 01/21/20 14:38:

i see that some informations from my old PK, are now in the new database !

i will continue with the copy directory ..

but i think that the user i define during the install of PK1.4.0, is not more usable .. how can i retreive the old user/pass from my old PK ?

thanks

christianlupus commented 4 years ago

From 01/21/20 15:03:

the old config.php file contain some information from the old database, which is not identical as the new one .. is that a problem ?

so i dont copy the config.php. i delete the app/parameters.php then rerun /setup in my browser. all green as before. step "Existing Configuration" : No configuration found step "Database Parameters" : i set the parameters for the new database and try with phpMyadmin to set the username/password. so, i logon to phpMyadmin, but it seem that i have no more any right on the Partkeepr database !!! i can't set "GRANT USAGE ON . ..." :((((

christianlupus commented 4 years ago

From 01/21/20 15:23:

well i manage to logon phpMyAdmin with root access now i see my database "Partkeepr" and re setup the grant usage to the user "Partkeepr"

the i go back to my /setup web page and click on next :

testing for databse connectivity : green

setting up the database schema : red "the table with name 'partkeepr.cachedimage' already exists

... what to do next ? sorry, i'm really a pain !

christianlupus commented 4 years ago

From 01/22/20 11:14:

Hallo Phil,

root@PARTKEEPR1:/home/partkeepr# mysql -u Partkeepr -p Partkeepr < 20190915_eko_partkeepr_bdc_new.sql Enter password: ERROR 1005 (HY000) at line 4295: Can't create table Partkeepr.#sql-229_5e (errno: 121 "Duplicate key on write or update"
)

this is sort of weird. I can try it on my machine if the file is importable.

and now, i just try to login in PK .. can't anymore :((( i'm tired .. i don't know what to do !

Of course, you cannot log into. You must first do the whole migration before you can use the installation again. So trying to login or running the setup mid-process might at best not help and at worst destroy your data in the VM. E.g. if a file is not found, it might get removed from the DB although you just did not yet migrate the files from the data folder.

I will answer the other questions individually.

Christian

christianlupus commented 4 years ago

From 01/22/20 11:17:

Hello Phil,

i see that some informations from my old PK, are now in the new database !

That should be exactly the case you should not have installed PK in the first moment. You could as well have dropped the WHOLE DB and recreated it.

As I said: We have to start from scratch.

i will continue with the copy directory ..

You should have done that by now.

but i think that the user i define during the install of PK1.4.0, is not more usable .. how can i retreive the old user/pass from my old PK ?

First make sure, you get it running. You do know the username/password combination of the old PK installation or not?

Christian

christianlupus commented 4 years ago

From 01/22/20 11:19:

the old config.php file contain some information from the old database, which is not identical as the new one .. is that a problem ?

You mean the wrong db host and user name? This needs to be updated.

so i dont copy the config.php. i delete the app/parameters.php then rerun /setup in my browser. all green as before. step "Existing Configuration" : No configuration found step "Database Parameters" : i set the parameters for the new database and try with phpMyadmin to set the username/password. so, i logon to phpMyadmin, but it seem that i have no more any right on the Partkeepr database !!! i can't set "GRANT USAGE ON . ..." :((((

Yes, PK will avoid overwriting existing data in the DB to ensure that a configuration error (like yours) destroys the data.

christianlupus commented 4 years ago

From 01/22/20 11:42:

well i manage to logon phpMyAdmin with root access now i see my database "Partkeepr" and re setup the grant usage to the user "Partkeepr"

the i go back to my /setup web page and click on next :

testing for databse connectivity : green

setting up the database schema : red "the table with name 'partkeepr.cachedimage' already exists

... what to do next ? sorry, i'm really a pain !

OK, I think, you might have done quite some things.

I just tried it: I can successfully import the SQL file into a mariadb server. I do not see any problem here.

So I think it is time to start over at this moment.

First log into the phpMyAdmin and select the database. Remove any existing table from there. Normally you use a non-root account for the productive access to the DB. Make sure, you can log into the DB (server) using these credentials and that you have granted the rights on the database to the user. To do so, create a simple table. If this works, everything is all right. If this does not work, you need to call the command

GRANT ALL ON foo.* TO bar; where foo is the name of the database and bar the name of the user. Remove the test table in case it worked.

Now import the SQL (either yours or the one I sent to you, they should work equally). You gave already the command to do so. Do NOT try to open PK.

Remove the PK data files. Recopy/resync the files from the old installation to the new PK folder. Adopt the config.php to respect the new DB settings (db user, host, pwd, name) plus potential similar changes.

Run the web installer, afterwards you are hopefully ready to use your installation.

If this does not work, we could also try to migrate in 2 steps: First we just migrate to a new machine (keeping at 0.19) and then in a second step update to the latest version. This way, you should be able to check the intermediate result if this is wokring sufficiently.

Christian

christianlupus commented 4 years ago

From 01/23/20 10:18:

Hello Christian,

on the new VM, i delete the database and create a new one with phpMyAdmin, using the SQL commands provided in the tab "database parameters" (I just run /setup to obtain the 3 SQL commands lines)

I import the old SQL file (the one not modified by you) using 'import' function under phpMyAdmin : no error

i delete "data" directory in the new VM

i copy the old "data" directory to the new VM (without the proxies directory) and adapt the rights for the user/group

i copy the config.php to the new VM and modify the parameters inside to be compatible with the new database

i rerun /setup, all green until tab "setup 1/2" : "setting up the database schema" : RED

Does the "partkeepr" user right configured under the SQL table : (again I just type the 3 lines given by the Database parameters tab ).

I don't see < ALL PRIVILEGES > for the partkeepr user .. is it right ?

Thanks, Phil

christianlupus commented 4 years ago

From 01/23/20 11:35:

Hello Phil,

sorry, my French is quite rusty so I cannot answer in French, excuse-moi.

Your PK instance should use the user partkeepr in the database, right?

Was the previous configuration detected correctly (tab "existing configuration")? The fact that you are in the step "Setting up database schema" and not in "Updating database schema" indicates for me that you are trying a plain new install, so your configuration might not have been detected/imported.

About the rights: This looks indeed suspicious. However I do not know what phpMyAdmin shows here exactly. So I'd log in as the partkeepr user (or whichever it should be) and try to do two things:

Add a new test table and remove afterward
Add a column to an existing table and remove again You might want to look especially at the footprintcategory table. Both times you can see, if you can alter the data structure correctly. If not, you might have to insert (as root) the commands
```
GRANT ALL ON Partkeer.* TO partkeepr;
FLUSH PRIVILEGES;
```
to provide the rights. Then retry the failed add/remove table and column test.

What db system (mysql, mariadb, ...) and version are you using? I would try the exact version to ensure, it is not a db version incompatibility. I doubt it but you never know where the problem comes from until you find it.

Christian

christianlupus commented 4 years ago

From 01/23/20 11:56:

No problem, my German is not very well, we will continue in English ;o)

My reply with screens copy (hope you can see them !)

Yes the PK setup use the partkeepr user

Yes I see that the previous config Is right detected :

About the rights for partkeepr user : I logon with my root user to phpMyAdmin and select / set all privileges to the partkeepr user Now I see that :

I rerun the /setup and obtain the same error (I think) as previous :

Here it is the SQL configuration :

It's hard ! I'm sorry to disturb you so much ! Thanks

christianlupus commented 4 years ago

From 01/23/20 16:35:

Hello once more,

I am running into bigger problems than I thought. Trying to install the 0.1.9 version of PK but failing to set up the infrastructure at the moment (needs time to get it right).

Christian

christianlupus commented 4 years ago

From 01/23/20 16:55:

"lol" if the expert failed too, how can I do it successful ! All my wishes to help you ;o)

Wait your reply, I can understand for sure that you don't have only this stuff to do !

christianlupus commented 4 years ago

@phil31 I ask you to proceed here in github and not via mail as this allows to track the progress of the issue and others might get involved. The only exception might be in case of private data you do not want to share with the public internet (thus I did not attach the sql files as one could try to hack your password from the included hash).

phil31 commented 4 years ago

Hi Christian, any news about your 0.1.9 setup ?

thanks, phil

christianlupus commented 4 years ago

I need to do it the manual way. As documentation is updated in the meantime and dependencies are not all available anymore as it seems, I need to do quite soe work to get this running.

You might try something different: Can you try to clone the old installation to the VM without updating? So just copy over the whole file structure plus the DB dump and insert the dump into a (another) database of your choice (just take another db name). Let's see if we can get you there.

The goal is that you have the old installation in the VM with a more recent HTTP daemon and Mariadb 10.1 or whatever you used to install. Then you make a clone of the VM ;-).

phil31 commented 4 years ago

hi Christian well was really busy last week ! you want that i copy the old 0.1.9 files to the new VM ? but which directory / files exactly ? then import the old database to a new one (with a new name) (using a copy of the VM of course)

that is already what we try as describe in the wiki help ?...i'm not sure to right understand what you mean

regards

christianlupus commented 4 years ago

I intend to make smaller steps. So the idea is to first migrate the installation completely to your VM without upgrading. As soon as it is running you can make a backup thereof. With the backup saved, you can try to migrate to the latest version of PK. You might need to do this in multiple steps as we had problems during the migration phase in our first attempt.

So, you need to copy the complete PK installation from the old machine to the VM (all folders) and set up the webserver accordingly. Then you make a simple 1:1 import of the database. This should get you running in general with the older 0.1.9 version of PK.

phil31 commented 4 years ago

mm you mean copy the 0.1.9 to a fresh VM apply modification to work on this new VM then try an update to 1.4.0 .. seem complex for my poor linux knowledge ! and remember, my 0.1.9 was working on a Windows server .. so there is surely a lot of things to modify before it will work on a new debian server !

and folowings the differents thread, it seem not a good idea !?

i think the best and simplest method is to find why my database is not right imported/used by the new 1.4.0 !.. i can send you my sql export to have a try on your side ?

christianlupus commented 4 years ago

The database is most probably imported already. This was not the issue here.

I assume that the newer PK has a different database structure than the older version. It detects this on the invocation of the setup script. As a result, the database schema is to be updated. This update process seems to be your culprit as far as I understand things here.

As you are using a webserver the difference between Linux and Windows should not be that large. The benefit of doing it in multiple steps is to split the big problem into smaller chunks. Otherwise, it might get really hard to get to the main issue.

phil31 commented 4 years ago

I am convinced that it will be very complicated to try to run my old installation in a new VM. it exceeds my knowledge of Linux and we will waste a lot of time exchanging questions and answers!

do you have a VM 1.4.0 to try to import my database and run the /setup ? if yes, message me to my personnal email, i will send you what you required. or i can try to send you a zip of my old windows setup .. but again, i'm not convince about this method :(

christianlupus commented 4 years ago

As far as I understand it, you are running Debian Stretch inside your VM, right?
Where are you planning on publishing the PK instance? In /var/http/www?

I will send you a link to my private Nextcloud instance. There you can upload a ZIP file of your old installation (complete Partkeepr folder, there should be multiple folders like app, data, scripts, setup and the like). I will install a local virtual machine of your OS and try to process the migration. I hope, you are ok with this.

christianlupus commented 4 years ago

OK, @phil31, I think I got this sorted out.

I uploaded three files to the Nextcloud folder I sent you earlier. Download all to the VM. Additionally download the latest PK version 1.4.0 release. I was using VirtualBox thus cloning the whole VM seems unusable. Also, it might get quite large. Thus you need to install locally for yourself. It should be straight forward.

Now proceed as following:

File system migration

I advise making a backup as always.
Go to /var/www/html. If there are any files, remove them (rm -rf /var/www/html/*). Uncompress the PK tarball into /var/www/html.
Remove the data folder: rm -rf data
Uncompress the provided data tarball from me in the folder. Go to /var/www/html and issue tar xJf </path/to/data2.tar.xz>
Copy the provided parameters.php to the folder app/config/.
Set the owner of all files to www-data: chown -R www-data /var/www/html

Database migration

You need to adopt the database as well.

Drop the old database if it was already there.
Create a new database partkeepr.
Create a user partkeepr if not done yet.
Grant all privileges to the user. The corresponding SQL is GRANT ALL ON partkeepr.* TO partkeepr; FLUSH PRIVILEGES;.
Ensure you know the password of the user.
Import the modified database dump. Use something like mysql -u partkeepr -p partkeepr < /path/to/db-finished2.sql and provide the password manually. The import might take a few seconds.

Apache configuration

On the VM you should make some modifications to the default apache configuration.

Issue (as root) the command a2enmod rewrite.
Adopt the file /etc/apache2/sites-enabled/000-default.conf
- Make sure DocumentRoot is /var/www/html/web (the web at the ending is importand)
- Add the following to the config if needed:
```
ServerSignature Off
AcceptPathInfo on
ErrorDocument 403 "<h1>Demo Site update in progress. Check back in a few minutes.</h1>"
```

<Directory /var/www/html/web/> Require all granted AllowOverride All


  - Restart Apache: `systemctl restart apache2`

# Finalizing the setup
- Open a browser on your main machine.
- Point to `http://<IP-of-partkeepr>/setup`.
- Proceed with the setup

You should be able to finish it. I added a new user `userfoo`. The password is `passworduserfoo`. I did not know your admin credentials. As soon as the installation is running you might want to remove those debug stubs.

phil31 commented 4 years ago

I warmly thank Christian for his precious help! it took a long time to detail and test at home. the result is perfect, I now have my old base 0.1.9, functional in 1.4.0 this tutorial is a must! Thank you very much Christian

christianlupus commented 4 years ago

I am closing this as it seems to be 100% solved.

For those running into a similar issue (and to remember things on my own): The problem here was a migration from a Windows host to a Linux host. As a result, the case sensitivity of the filesystem was used in the name of the databases as well (standard settings on Windows and Linux differ here). Thus, there was a naming clash in the database triggering the issue mentioned above. I had to manually rename all tables in the database to their corresponding mixed case for them to be compatible with the Linux default installation. After that, it was merely a simple regex replacement on the SQL dump to get the absolute paths in the cache table to match up with the values from the real installation.

partkeepr / PartKeepr

problem to import components from 1.09 to 1.4 #1063

Database import

Filesystem import

Migration of the data

File system migration

Database migration

Apache configuration