Unable to upgrade from 2.10.2 > 3.0.0

antwonw commented 2 years ago

I seem to be having a similar issue when upgrading from 2.10.2 > 3.0.0

How Shlink is set-up Shlink Version: 3.0.0 PHP Version: 8.0.14 How do you serve Shlink: Self-hosted, Apache 2.4.4, Ubuntu 20.04 LTS Database engine used: SQLite

Summary Trying to upgrade from 2.10.2 to 3.0.0 and v3 is unable to migrate v2 database.

Behavior

link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 1304
drwxr-x--- 12 link link    4096 Feb  7 16:14 ./
drwxr-xr-x  4 link link    4096 Feb  6 02:03 ../
-rw-r--r--  1 link link      44 Feb  6 01:49 .virtualmin-src
drwxr-x---  2 link link    4096 Aug 10 12:34 cgi-bin/
drwxrwxr-x  2 link link    4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link    4096 Aug 10 12:34 etc/
drwxr-xr-x  2 link link    4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link    4096 Sep  1  2020 homes/
drwxr-x---  2 link link    4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link 1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link      13 Sep  1  2020 public_html -> shlink/public/
drwxrwxr-x  8 link link    4096 Feb  7 15:38 shlink/
drwxrwxr-x  8 link link    4096 Feb  7 15:38 shlink-old-old/
drwxr-xr-x  8 link link    4096 Jan  7 12:55 shlink2.10.2_php8.0_dist/
-rwxr-xr-x  1 link link    3750 Dec 12 15:07 ssl.ca*
-rwx------  1 link link    2264 Dec 12 15:07 ssl.cert*
-rwx------  1 link link    6016 Dec 12 15:07 ssl.combined*
-rwx------  1 link link    7725 Dec 12 15:07 ssl.everything*
-rwx------  1 link link    1708 Dec 12 15:07 ssl.key*
drwxr-x---  2 link link    4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ wget https://github.com/shlinkio/shlink/releases/download/v3.0.0/shlink3.0.0_php8.0_dist.zip && unzip shlink3.0.0_php8.0_dist.zip && rm shlink3.0.0_php8.0_dist.zip && mv shlink/ shlink-old/ && cp -ap shlink3.0.0_php8.0_dist/ shlink/ && cd shlink/ && vendor/bin/shlink-installer update -vvv
--2022-02-07 16:18:15--  https://github.com/shlinkio/shlink/releases/download/v3.0.0/shlink3.0.0_php8.0_dist.zip
Resolving github.com (github.com)... 140.82.113.3
Connecting to github.com (github.com)|140.82.113.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/55801807/d38f8fc6-0532-447c-9411-57fb23181c7e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220208%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220208T001815Z&X-Amz-Expires=300&X-Amz-Signature=bc10f0b7362843a3b4342a018258b15710559a41be68ae705154cd1e74495a0d&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=55801807&response-content-disposition=attachment%3B%20filename%3Dshlink3.0.0_php8.0_dist.zip&response-content-type=application%2Foctet-stream [following]
--2022-02-07 16:18:15--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/55801807/d38f8fc6-0532-447c-9411-57fb23181c7e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220208%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220208T001815Z&X-Amz-Expires=300&X-Amz-Signature=bc10f0b7362843a3b4342a018258b15710559a41be68ae705154cd1e74495a0d&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=55801807&response-content-disposition=attachment%3B%20filename%3Dshlink3.0.0_php8.0_dist.zip&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22311182 (21M) [application/octet-stream]
Saving to: ‘shlink3.0.0_php8.0_dist.zip’

shlink3.0.0_php8.0_dist.zip    100%[===================================================>]  21.28M  44.6MB/s    in 0.5s    

2022-02-07 16:18:16 (44.6 MB/s) - ‘shlink3.0.0_php8.0_dist.zip’ saved [22311182/22311182]

Archive:  shlink3.0.0_php8.0_dist.zip

 Welcome to Shlink!!
 This tool will guide you through the installation process.

 Do you want to import configuration from previous installation? (You will still be asked for any new config option that did not exist in previous shlink versions) (yes/no) [yes]:
 > yes

 Previous shlink installation path from which to import config:
 > ../shlink-old/

DATABASE
========

 Database name [shlink]:
 >

The CLI accepts the previous instance path, but then it asks to setup the database, though one already exists. Completing the CLI questions results in a broken install and database not migrated. If I tried to enter other paths, but the CLI will tell me those aren't valid, so obviously my initial path is correct.

Originally posted by @antwonw in https://github.com/shlinkio/shlink/issues/1357#issuecomment-1032087525

acelaya commented 2 years ago

Thanks for reporting.

The structure of the generated configuration has drastically changed from previous version, so I guess I might have missed some combination when migrating the old structure.

I'll try to investigate.

antwonw commented 2 years ago

Thanks! Let me know if you'd like me to test anything or provide any other reports.

acelaya commented 2 years ago

I found the root cause. I'll release a fix between tomorrow afternoon and the weekend.

acelaya commented 2 years ago

I have just released Shlink 3.0.2. You should be able to update from 2.10.x directly to it and it should properly migrate the database config when using SQLite.

https://github.com/shlinkio/shlink/releases/tag/v3.0.2

antwonw commented 2 years ago

Alright, so am I doing something wrong here, @acelaya? It's still not migrating.

Here is my command wget https://github.com/shlinkio/shlink/releases/download/v3.0.2/shlink3.0.2_php8.0_dist.zip && unzip -q shlink3.0.2_php8.0_dist.zip && rm shlink3.0.2_php8.0_dist.zip && cp -a shlink/ shlinkbackup/ && mv shlink/ shlink-old/ && cp -ap shlink3.0.0_php8.0_dist/ shlink/ && cd shlink/ && vendor/bin/shlink-installer update -vvv

link@seattle:~/domains/serve.link.nwadventists.com$ wget https://github.com/shlinkio/shlink/releases/download/v3.0.2/shlink3.0.2_php8.0_dist.zip && unzip -q shlink3.0.2_php8.0_dist.zip && rm shlink3.0.2_php8.0_dist.zip && cp -a shlink/ shlinkbackup/ && mv shlink/ shlink-old/ && cp -ap shlink3.0.0_php8.0_dist/ shlink/ && cd shlink/ && vendor/bin/shlink-installer update -vvv
--2022-02-10 13:23:43--  https://github.com/shlinkio/shlink/releases/download/v3.0.2/shlink3.0.2_php8.0_dist.zip
Resolving github.com (github.com)... 140.82.114.3
Connecting to github.com (github.com)|140.82.114.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/55801807/0383d3a7-0182-41b3-b910-14088c9adf39?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220210%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220210T212344Z&X-Amz-Expires=300&X-Amz-Signature=24cdbc98119b8f3741a56e6ce9ad12be534c3c373adb36d657ddbe1cca4c9638&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=55801807&response-content-disposition=attachment%3B%20filename%3Dshlink3.0.2_php8.0_dist.zip&response-content-type=application%2Foctet-stream [following]
--2022-02-10 13:23:44--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/55801807/0383d3a7-0182-41b3-b910-14088c9adf39?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220210%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220210T212344Z&X-Amz-Expires=300&X-Amz-Signature=24cdbc98119b8f3741a56e6ce9ad12be534c3c373adb36d657ddbe1cca4c9638&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=55801807&response-content-disposition=attachment%3B%20filename%3Dshlink3.0.2_php8.0_dist.zip&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.109.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22325339 (21M) [application/octet-stream]
Saving to: ‘shlink3.0.2_php8.0_dist.zip’

shlink3.0.2_php8.0_dist.zip                 100%[=========================================================================================>]  21.29M  37.5MB/s    in 0.6s    

2022-02-10 13:23:44 (37.5 MB/s) - ‘shlink3.0.2_php8.0_dist.zip’ saved [22325339/22325339]

 Welcome to Shlink!!
 This tool will guide you through the installation process.

 Do you want to import configuration from previous installation? (You will still be asked for any new config option that did not exist in previous shlink versions) (yes/no) [yes]:
 > yes

 Previous shlink installation path from which to import config:
 > ../shlink-old/

DATABASE
========

 Database name [shlink]:
 >

acelaya commented 2 years ago

Hmm, that is weird. It should not be asking you that.

I'll check tomorrow again. I'm not sure what I might have missed this time.

acelaya commented 2 years ago

Would you mind sharing your existing generated config here?

It's the config/params/generated_config.php file. The only credential you need to hide from it is the GeoLite2 license.

acelaya commented 2 years ago

I have been doing some tests, and I cannot reproduce it anymore with Shlink 3.0.2

I have followed these steps:

Download fresh Shlink 2.10.2: Installed it from scratch selecting SQLite
Download fresh Shlink 3.0.0: Tried to update it, importing from v2.10.2 -> The behavior described here occurs
Download fresh Shlink 3.0.1: Tried to update it, importing from v2.10.2 -> The behavior described here occurs
Download fresh Shlink 3.0.2 Tried to update it, importing from v2.10.2 -> Import worked as expected.

The only option I can think of is that there's some config combination I might be missing, that's why i was asking about your config.

antwonw commented 2 years ago

Here is my config file.

link@seattle:~$ cat domains/serve.link.nwadventists.com/shlink/config/params/generated_config.php
<?php
return array(
    'entity_manager' => array(
        'connection' => array(
            'driver' => 'pdo_sqlite',
            'path' => 'data/database.sqlite',
        ),
    ),
    'url_shortener' => array(
        'domain' => array(
            'hostname' => 'nwadvent.st',
            'schema' => 'https',
        ),
        'validate_url' => false,
        'default_short_codes_length' => 5,
        'redirect_status_code' => 302,
        'auto_resolve_titles' => true,
        'append_extra_path' => true,
    ),
    'geolite2' => array(
        'license_key' => '****************',
    ),
    'not_found_redirects' => array(
        'base_url' => 'https://nwadventists.com/',
        'invalid_short_url' => 'https://nwadventists.com/',
        'regular_404' => 'https://nwadventists.com/',
    ),
    'delete_short_urls' => array(
        'check_visits_threshold' => true,
        'visits_threshold' => 15,
    ),
    'router' => array(
        'base_path' => '',
    ),
    'cache' => array(
        'redis' => null,
    ),
    'tracking' => array(
        'track_orphan_visits' => true,
        'disable_track_param' => null,
        'disable_tracking' => false,
        'disable_ip_tracking' => false,
        'anonymize_remote_addr' => true,
        'disable_ua_tracking' => false,
        'disable_referrer_tracking' => false,
        'disable_tracking_from' => array(),
    ),
    'qr_codes' => array(
        'size' => 1000,
        'margin' => 0,
        'format' => 'svg',
        'error_correction' => 'l',
        'round_block_size' => false,
    ),
);

antwonw commented 2 years ago

So, I wasn't looking very carefully at the single command I use to download, move and update shlink each new version.

It appears I didn't update it to reflect 3.0.2 and it still had 3.0.0.

$ wget https://github.com/shlinkio/shlink/releases/download/v3.0.2/shlink3.0.2_php8.0_dist.zip && unzip -q shlink3.0.2_php8.0_dist.zip && rm shlink3.0.2_php8.0_dist.zip && cp -a shlink/ shlinkbackup/ && mv shlink/ shlink-old/ && cp -ap shlink3.0.0_php8.0_dist/ shlink/ && cd shlink/ && vendor/bin/shlink-installer update -vvv`

So, I redid each command individually this time. And it migrated....HOWEVER...I'm still running into a critical error.

link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 1304
drwxr-x--- 12 link link    4096 Feb 11 10:28 ./
drwxr-xr-x  4 link link    4096 Feb  6 02:03 ../
-rw-r--r--  1 link link      44 Feb  6 01:49 .virtualmin-src
drwxr-x---  2 link link    4096 Aug 10  2021 cgi-bin/
drwxrwxr-x  2 link link    4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link    4096 Aug 10  2021 etc/
drwxr-xr-x  2 link link    4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link    4096 Sep  1  2020 homes/
drwxr-x---  2 link link    4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link 1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link      13 Sep  1  2020 public_html -> shlink/public/
drwxrwxr-x  8 link link    4096 Feb  7 15:38 shlink/
drwxrwxr-x  8 link link    4096 Feb  7 15:38 shlink-old-old/
drwxr-xr-x  8 link link    4096 Jan  7 12:55 shlink2.10.2_php8.0_dist/
-rwxr-xr-x  1 link link    3750 Dec 12 15:07 ssl.ca*
-rwx------  1 link link    2264 Dec 12 15:07 ssl.cert*
-rwx------  1 link link    6016 Dec 12 15:07 ssl.combined*
-rwx------  1 link link    7725 Dec 12 15:07 ssl.everything*
-rwx------  1 link link    1708 Dec 12 15:07 ssl.key*
drwxr-x---  2 link link    4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ wget -q --show-progress https://github.com/shlinkio/shlink/releases/download/v3.0.2/shlink3.0.2_php8.0_dist.zip
shlink3.0.2_php8.0_dist.zip       100%[============================================================>]  21.29M  37.2MB/s    in 0.6s    
link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 23108
drwxr-x--- 12 link link     4096 Feb 11 10:31 ./
drwxr-xr-x  4 link link     4096 Feb  6 02:03 ../
-rw-r--r--  1 link link       44 Feb  6 01:49 .virtualmin-src
drwxr-x---  2 link link     4096 Aug 10  2021 cgi-bin/
drwxrwxr-x  2 link link     4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link     4096 Aug 10  2021 etc/
drwxr-xr-x  2 link link     4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link     4096 Sep  1  2020 homes/
drwxr-x---  2 link link     4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link  1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link       13 Sep  1  2020 public_html -> shlink/public/
drwxrwxr-x  8 link link     4096 Feb  7 15:38 shlink/
drwxrwxr-x  8 link link     4096 Feb  7 15:38 shlink-old-old/
drwxr-xr-x  8 link link     4096 Jan  7 12:55 shlink2.10.2_php8.0_dist/
-rw-rw-r--  1 link link 22325339 Feb 10 12:52 shlink3.0.2_php8.0_dist.zip
-rwxr-xr-x  1 link link     3750 Dec 12 15:07 ssl.ca*
-rwx------  1 link link     2264 Dec 12 15:07 ssl.cert*
-rwx------  1 link link     6016 Dec 12 15:07 ssl.combined*
-rwx------  1 link link     7725 Dec 12 15:07 ssl.everything*
-rwx------  1 link link     1708 Dec 12 15:07 ssl.key*
drwxr-x---  2 link link     4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ unzip -q shlink3.0.2_php8.0_dist.zip
link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 23112
drwxr-x--- 13 link link     4096 Feb 11 10:32 ./
drwxr-xr-x  4 link link     4096 Feb  6 02:03 ../
-rw-r--r--  1 link link       44 Feb  6 01:49 .virtualmin-src
drwxr-x---  2 link link     4096 Aug 10  2021 cgi-bin/
drwxrwxr-x  2 link link     4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link     4096 Aug 10  2021 etc/
drwxr-xr-x  2 link link     4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link     4096 Sep  1  2020 homes/
drwxr-x---  2 link link     4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link  1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link       13 Sep  1  2020 public_html -> shlink/public/
drwxrwxr-x  8 link link     4096 Feb  7 15:38 shlink/
drwxrwxr-x  8 link link     4096 Feb  7 15:38 shlink-old-old/
drwxr-xr-x  8 link link     4096 Jan  7 12:55 shlink2.10.2_php8.0_dist/
drwxr-xr-x  8 link link     4096 Feb 10 12:49 shlink3.0.2_php8.0_dist/
-rw-rw-r--  1 link link 22325339 Feb 10 12:52 shlink3.0.2_php8.0_dist.zip
-rwxr-xr-x  1 link link     3750 Dec 12 15:07 ssl.ca*
-rwx------  1 link link     2264 Dec 12 15:07 ssl.cert*
-rwx------  1 link link     6016 Dec 12 15:07 ssl.combined*
-rwx------  1 link link     7725 Dec 12 15:07 ssl.everything*
-rwx------  1 link link     1708 Dec 12 15:07 ssl.key*
drwxr-x---  2 link link     4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ mv shlink/ shlink-old/
link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 23112
drwxr-x--- 13 link link     4096 Feb 11 10:34 ./
drwxr-xr-x  4 link link     4096 Feb  6 02:03 ../
-rw-r--r--  1 link link       44 Feb  6 01:49 .virtualmin-src
drwxr-x---  2 link link     4096 Aug 10  2021 cgi-bin/
drwxrwxr-x  2 link link     4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link     4096 Aug 10  2021 etc/
drwxr-xr-x  2 link link     4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link     4096 Sep  1  2020 homes/
drwxr-x---  2 link link     4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link  1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link       13 Sep  1  2020 public_html -> shlink/public
drwxrwxr-x  8 link link     4096 Feb  7 15:38 shlink-old/
drwxrwxr-x  8 link link     4096 Feb  7 15:38 shlink-old-old/
drwxr-xr-x  8 link link     4096 Jan  7 12:55 shlink2.10.2_php8.0_dist/
drwxr-xr-x  8 link link     4096 Feb 10 12:49 shlink3.0.2_php8.0_dist/
-rw-rw-r--  1 link link 22325339 Feb 10 12:52 shlink3.0.2_php8.0_dist.zip
-rwxr-xr-x  1 link link     3750 Dec 12 15:07 ssl.ca*
-rwx------  1 link link     2264 Dec 12 15:07 ssl.cert*
-rwx------  1 link link     6016 Dec 12 15:07 ssl.combined*
-rwx------  1 link link     7725 Dec 12 15:07 ssl.everything*
-rwx------  1 link link     1708 Dec 12 15:07 ssl.key*
drwxr-x---  2 link link     4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ cp -ap shlink3.0.2_php8.0_dist/ shlink/
link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 23116
drwxr-x--- 14 link link     4096 Feb 11 10:34 ./
drwxr-xr-x  4 link link     4096 Feb  6 02:03 ../
-rw-r--r--  1 link link       44 Feb  6 01:49 .virtualmin-src
drwxr-x---  2 link link     4096 Aug 10  2021 cgi-bin/
drwxrwxr-x  2 link link     4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link     4096 Aug 10  2021 etc/
drwxr-xr-x  2 link link     4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link     4096 Sep  1  2020 homes/
drwxr-x---  2 link link     4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link  1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link       13 Sep  1  2020 public_html -> shlink/public/
drwxr-xr-x  8 link link     4096 Feb 10 12:49 shlink/
drwxrwxr-x  8 link link     4096 Feb  7 15:38 shlink-old/
drwxrwxr-x  8 link link     4096 Feb  7 15:38 shlink-old-old/
drwxr-xr-x  8 link link     4096 Jan  7 12:55 shlink2.10.2_php8.0_dist/
drwxr-xr-x  8 link link     4096 Feb 10 12:49 shlink3.0.2_php8.0_dist/
-rw-rw-r--  1 link link 22325339 Feb 10 12:52 shlink3.0.2_php8.0_dist.zip
-rwxr-xr-x  1 link link     3750 Dec 12 15:07 ssl.ca*
-rwx------  1 link link     2264 Dec 12 15:07 ssl.cert*
-rwx------  1 link link     6016 Dec 12 15:07 ssl.combined*
-rwx------  1 link link     7725 Dec 12 15:07 ssl.everything*
-rwx------  1 link link     1708 Dec 12 15:07 ssl.key*
drwxr-x---  2 link link     4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ cd shlink/
link@seattle:~/domains/serve.link.nwadventists.com/shlink$ vendor/bin/shlink-installer update -vvv

 Welcome to Shlink!!
 This tool will guide you through the installation process.

 Do you want to import configuration from previous installation? (You will still be asked for any new config option that did not exist in previous shlink versions) (yes/no) [yes]:
 > yes

 Previous shlink installation path from which to import config:
 > ../shlink-old/

INTEGRATIONS
============

 Do you want to use a redis instance, redis cluster or redis sentinels as a shared cache for Shlink? (recommended if you run a cluster of Shlink instances) (yes/no) [no]:
 > no

 Custom configuration properly generated!

Updating database... [Running "/usr/bin/php8.0 bin/cli db:migrate"]   RUN  '/usr/bin/php8.0' 'bin/cli' 'db:migrate'
  OUT  Migrating database...
  OUT  
  ERR    RUN  '/usr/bin/php8.0' 'vendor/doctrine/migrations/bin/doctrine-migrations.php' 'migrations:migrate' '--no-interaction'
  ERR    ERR  
  ERR    ERR  In SchemaException.php line 35:
  ERR    ERR                                                                    
  ERR    ERR    [Doctrine\DBAL\Schema\SchemaException (10)]                     
  ERR    ERR    There is no table with name 'public.short_urls' in the schema.  
  ERR    ERR                                                                    
  ERR    ERR  
  ERR    ERR  Exception trace:
  ERR    ERR    at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/dbal/src/Schema/SchemaException.php:35
  ERR    ERR   Doctrine\DBAL\Schema\SchemaException::tableDoesNotExist() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/dbal/src/Schema/Schema.php:189
  ERR    ERR   Doctrine\DBAL\Schema\Schema->getTable() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/friendsofphp/proxy-manager-lts/src/ProxyManager/GeneratorStrategy/EvaluatingGeneratorStrategy.php(54) : eval()'d code:49
  ERR    ERR   ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generated83d21ccf7853cfc07dce9110515971db->getTable() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/friendsofphp/proxy-manager-lts/src/ProxyManager/GeneratorStrategy/EvaluatingGeneratorStrategy.php(54) : eval()'d code:49
  ERR    ERR   ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generated83d21ccf7853cfc07dce9110515971db->getTable() at /home/link/domains/serve.link.nwadventists.com/shlink/data/migrations/Version20160819142757.php:28
  ERR    ERR   ShlinkMigrations\Version20160819142757->up() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Version/DbalExecutor.php:167
  ERR    ERR   Doctrine\Migrations\Version\DbalExecutor->executeMigration() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Version/DbalExecutor.php:102
  ERR    ERR   Doctrine\Migrations\Version\DbalExecutor->execute() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/DbalMigrator.php:112
  ERR    ERR   Doctrine\Migrations\DbalMigrator->executePlan() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/DbalMigrator.php:76
  ERR    ERR   Doctrine\Migrations\DbalMigrator->executeMigrations() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/DbalMigrator.php:162
  ERR    ERR   Doctrine\Migrations\DbalMigrator->migrate() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Tools/Console/Command/MigrateCommand.php:210
  ERR    ERR   Doctrine\Migrations\Tools\Console\Command\MigrateCommand->execute() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Command/Command.php:291
  ERR    ERR   Symfony\Component\Console\Command\Command->run() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:989
  ERR    ERR   Symfony\Component\Console\Application->doRunCommand() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:299
  ERR    ERR   Symfony\Component\Console\Application->doRun() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:171
  ERR    ERR   Symfony\Component\Console\Application->run() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Tools/Console/ConsoleRunner.php:97
  ERR    ERR   Doctrine\Migrations\Tools\Console\ConsoleRunner::run() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/bin/doctrine-migrations.php:45
  ERR    ERR   Doctrine\Migrations\{closure}() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/bin/doctrine-migrations.php:46
  ERR    ERR  
  ERR    ERR  migrations:migrate [--write-sql [WRITE-SQL]] [--dry-run] [--query-time] [--allow-no-migration] [--all-or-nothing [ALL-OR-NOTHING]] [--configuration CONFIGURATION] [--em EM] [--conn CONN] [--] [<version>]
  ERR    ERR  
  ERR    ERR  
  ERR  In Process.php line 267:
  ERR                                                                                 
  ERR    [Symfony\Component\Process\Exception\ProcessFailedException]                 
  ERR    The command "'/usr/bin/php8.0' 'vendor/doctrine/migrations/bin/doctrine-mig  
  ERR    rations.php' 'migrations:migrate' '--no-interaction'" failed.                
  ERR                                                                                 
  ERR    Exit Code: 10(Unknown error)                                                 
  ERR                                                                                 
  ERR    Working directory: /home/link/domains/serve.link.nwadventists.com/shlink     
  ERR                                                                                 
  ERR    Output:                                                                      
  ERR    ================                                                             
  ERR                                                                                 
  ERR                                                                                 
  ERR    Error Output:                                                                
  ERR    ================                                                             
  ERR                                                                                 
  ERR    In SchemaException.php line 35:                                              
  ERR                                                                                 
  ERR      [Doctrine\DBAL\Schema\SchemaException (10)]                                
  ERR      There is no table with name 'public.short_urls' in the schema.             
  ERR                                                                                 
  ERR                                                                                 
  ERR    Exception trace:                                                             
  ERR      at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/  
  ERR    dbal/src/Schema/SchemaException.php:35                                       
  ERR     Doctrine\DBAL\Schema\SchemaException::tableDoesNotExist() at /home/link/do  
  ERR    mains/serve.link.nwadventists.com/shlink/vendor/doctrine/dbal/src/Schema/Sc  
  ERR    hema.php:189                                                                 
  ERR     Doctrine\DBAL\Schema\Schema->getTable() at /home/link/domains/serve.link.n  
  ERR    wadventists.com/shlink/vendor/friendsofphp/proxy-manager-lts/src/ProxyManag  
  ERR    er/GeneratorStrategy/EvaluatingGeneratorStrategy.php(54) : eval()'d code:49  
  ERR     ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generated83d  
  ERR    21ccf7853cfc07dce9110515971db->getTable() at /home/link/domains/serve.link.  
  ERR    nwadventists.com/shlink/vendor/friendsofphp/proxy-manager-lts/src/ProxyMana  
  ERR    ger/GeneratorStrategy/EvaluatingGeneratorStrategy.php(54) : eval()'d code:4  
  ERR    9                                                                            
  ERR     ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generated83d  
  ERR    21ccf7853cfc07dce9110515971db->getTable() at /home/link/domains/serve.link.  
  ERR    nwadventists.com/shlink/data/migrations/Version20160819142757.php:28         
  ERR     ShlinkMigrations\Version20160819142757->up() at /home/link/domains/serve.l  
  ERR    ink.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrati  
  ERR    ons/Version/DbalExecutor.php:167                                             
  ERR     Doctrine\Migrations\Version\DbalExecutor->executeMigration() at /home/link  
  ERR    /domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/  
  ERR    Doctrine/Migrations/Version/DbalExecutor.php:102                             
  ERR     Doctrine\Migrations\Version\DbalExecutor->execute() at /home/link/domains/  
  ERR    serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/  
  ERR    Migrations/DbalMigrator.php:112                                              
  ERR     Doctrine\Migrations\DbalMigrator->executePlan() at /home/link/domains/serv  
  ERR    e.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migr  
  ERR    ations/DbalMigrator.php:76                                                   
  ERR     Doctrine\Migrations\DbalMigrator->executeMigrations() at /home/link/domain  
  ERR    s/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrin  
  ERR    e/Migrations/DbalMigrator.php:162                                            
  ERR     Doctrine\Migrations\DbalMigrator->migrate() at /home/link/domains/serve.li  
  ERR    nk.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migratio  
  ERR    ns/Tools/Console/Command/MigrateCommand.php:210                              
  ERR     Doctrine\Migrations\Tools\Console\Command\MigrateCommand->execute() at /ho  
  ERR    me/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/C  
  ERR    ommand/Command.php:291                                                       
  ERR     Symfony\Component\Console\Command\Command->run() at /home/link/domains/ser  
  ERR    ve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:989   
  ERR     Symfony\Component\Console\Application->doRunCommand() at /home/link/domain  
  ERR    s/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php  
  ERR    :299                                                                         
  ERR     Symfony\Component\Console\Application->doRun() at /home/link/domains/serve  
  ERR    .link.nwadventists.com/shlink/vendor/symfony/console/Application.php:171     
  ERR     Symfony\Component\Console\Application->run() at /home/link/domains/serve.l  
  ERR    ink.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrati  
  ERR    ons/Tools/Console/ConsoleRunner.php:97                                       
  ERR     Doctrine\Migrations\Tools\Console\ConsoleRunner::run() at /home/link/domai  
  ERR    ns/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/bin/doctri  
  ERR    ne-migrations.php:45                                                         
  ERR     Doctrine\Migrations\{closure}() at /home/link/domains/serve.link.nwadventi  
  ERR    sts.com/shlink/vendor/doctrine/migrations/bin/doctrine-migrations.php:46     
  ERR                                                                                 
  ERR    migrations:migrate [--write-sql [WRITE-SQL]] [--dry-run] [--query-time] [--  
  ERR    allow-no-migration] [--all-or-nothing [ALL-OR-NOTHING]] [--configuration CO  
  ERR    NFIGURATION] [--em EM] [--conn CONN] [--] [<version>]                        
  ERR                                                                                 
  ERR  
  ERR  Exception trace:
  ERR    at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/process/Process.php:267
  ERR   Symfony\Component\Process\Process->mustRun() at /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Util/ProcessRunner.php:48
  ERR   Shlinkio\Shlink\CLI\Util\ProcessRunner->run() at /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Command/Db/AbstractDatabaseCommand.php:30
  ERR   Shlinkio\Shlink\CLI\Command\Db\AbstractDatabaseCommand->runPhpCommand() at /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Command/Db/MigrateDatabaseCommand.php:30
  ERR   Shlinkio\Shlink\CLI\Command\Db\MigrateDatabaseCommand->lockedExecute() at /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Command/Util/AbstractLockedCommand.php:35
  ERR   Shlinkio\Shlink\CLI\Command\Util\AbstractLockedCommand->execute() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Command/Command.php:291
  ERR   Symfony\Component\Console\Command\Command->run() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:989
  ERR   Symfony\Component\Console\Application->doRunCommand() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:299
  ERR   Symfony\Component\Console\Application->doRun() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:171
  ERR   Symfony\Component\Console\Application->run() at /home/link/domains/serve.link.nwadventists.com/shlink/bin/cli:10
  ERR  
  ERR  db:migrate
  ERR  
  ERR  
  RES  1 Command did not run successfully
link@seattle:~/domains/serve.link.nwadventists.com/shlink$

acelaya commented 2 years ago

Could it be that in this last try, when you did the database backup (I see the original_database.sqlite file in the root folder) you actually moved it instead of copied it? That could make the shlink-old folder to not have a database file to copy from, which in turn would result in a new empty database being created, and therefore, the migrations throw these errors where the tables do not exist.

I'm asking because I just repeated the whole process once again, even using your config, and everything worked: https://youtu.be/Yvg0Ao9vnzA

If that's not the case, I would recommend copying it manually to the new folder (if there's already a database.sqlite file in the new one, just remove it), and then, run bin/cli db:migrate -vvv, which will get the database up to date.

antwonw commented 2 years ago

So that original_database.sqlite is an old database from almost a year ago when I needed to make some modifications to the db alone and wanted a backup.

It appears that the current issue might be a permissions item with my shlink/ directory as I am unable to get access to even the cli on 2.10.2. So, this might be part of the reason for the migration issue?

link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 1300
drwxr-x--- 11 link link    4096 Feb 11 11:04 ./
drwxr-xr-x  4 link link    4096 Feb  6 02:03 ../
-rw-r--r--  1 link link      44 Feb  6 01:49 .virtualmin-src
drwxr-x---  2 link link    4096 Aug 10  2021 cgi-bin/
drwxrwxr-x  2 link link    4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link    4096 Aug 10  2021 etc/
drwxr-xr-x  2 link link    4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link    4096 Sep  1  2020 homes/
drwxr-x---  2 link link    4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link 1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link      13 Sep  1  2020 public_html -> shlink/public/
drwxrwxr-x  8 link link    4096 Feb  7 15:38 shlink/
drwxr-xr-x  8 link link    4096 Jan  7 12:55 shlink2.10.2_php8.0_dist/
-rwxr-xr-x  1 link link    3750 Dec 12 15:07 ssl.ca*
-rwx------  1 link link    2264 Dec 12 15:07 ssl.cert*
-rwx------  1 link link    6016 Dec 12 15:07 ssl.combined*
-rwx------  1 link link    7725 Dec 12 15:07 ssl.everything*
-rwx------  1 link link    1708 Dec 12 15:07 ssl.key*
drwxr-x---  2 link link    4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ cd shlink/
link@seattle:~/domains/serve.link.nwadventists.com/shlink$ bin/cli short-url:list
-bash: bin/cli: Permission denied

antwonw commented 2 years ago

If I attempt to access the original download of 2.10.2 and access the cli I of course get an db access error.

link@seattle:~/domains/serve.link.nwadventists.com/shlink$ cd ../
link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 1300
drwxr-x--- 11 link link    4096 Feb 11 11:04 ./
drwxr-xr-x  4 link link    4096 Feb  6 02:03 ../
-rw-r--r--  1 link link      44 Feb  6 01:49 .virtualmin-src
drwxr-x---  2 link link    4096 Aug 10  2021 cgi-bin/
drwxrwxr-x  2 link link    4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link    4096 Aug 10  2021 etc/
drwxr-xr-x  2 link link    4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link    4096 Sep  1  2020 homes/
drwxr-x---  2 link link    4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link 1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link      13 Sep  1  2020 public_html -> shlink/public/
drwxrwxr-x  8 link link    4096 Feb  7 15:38 shlink/
drwxr-xr-x  8 link link    4096 Jan  7 12:55 shlink2.10.2_php8.0_dist/
-rwxr-xr-x  1 link link    3750 Dec 12 15:07 ssl.ca*
-rwx------  1 link link    2264 Dec 12 15:07 ssl.cert*
-rwx------  1 link link    6016 Dec 12 15:07 ssl.combined*
-rwx------  1 link link    7725 Dec 12 15:07 ssl.everything*
-rwx------  1 link link    1708 Dec 12 15:07 ssl.key*
drwxr-x---  2 link link    4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ cd shlink2.10.2_php8.0_dist/
link@seattle:~/domains/serve.link.nwadventists.com/shlink2.10.2_php8.0_dist$ bin/cli short-url:list

In ExceptionConverter.php line 55:

  An exception occurred while executing a query: SQLSTATE[HY000]: General error: 1 no such table: short_urls  

In Exception.php line 26:

  SQLSTATE[HY000]: General error: 1 no such table: short_urls  

In Connection.php line 72:

  SQLSTATE[HY000]: General error: 1 no such table: short_urls  

short-url:list [-p|--page PAGE] [-st|--search-term SEARCH-TERM] [--searchTerm SEARCHTERM] [-t|--tags TAGS] [-o|--order-by ORDER-BY] [--orderBy ORDERBY] [--show-tags] [--showTags] [-k|--show-api-key] [-m|--show-api-key-name] [-a|--all] [-s|--start-date START-DATE] [--startDate STARTDATE] [-e|--end-date END-DATE] [--endDate ENDDATE]

acelaya commented 2 years ago

If I attempt to access the original download of 2.10.2 and access the cli I of course get an db access error.

But that is probably expected. Have you ever initialized Shlink on that one? Otherwise, it won't have a database at all, and trying to do anything will initialize an empty one, throwing that kind of error, because the tables do not exist.

So, this might be part of the reason for the migration issue?

We might be onto something here. What I think happened in your last attempt is that everything worked except copying the database file itself, hence, this suggestion:

If that's not the case, I would recommend copying it manually to the new folder (if there's already a database.sqlite file in the new one, just remove it), and then, run bin/cli db:migrate -vvv, which will get the database up to date.

After that you will end up with a working copy of Shlink 3.0.2 with your older database.

antwonw commented 2 years ago

Alright, so this worked....kind of.

Here is what I did:

$ wget https://github.com/shlinkio/shlink/releases/download/v3.0.2/shlink3.0.2_php8.0_dist.zip && unzip -q shlink3.0.2_php8.0_dist.zip && rm shlink3.0.2_php8.0_dist.zip
$ mv shlink/ shlink-old/ && cp -ap shlink3.0.2_php8.0_dist/ shlink/
$ cd shlink/
$ vendor/bin/shlink-installer update -vvv
$ rm data/database.sqlite
$ cp ../shlink-old/data/database.sqlite data/
$ bin/cli db:migrate -vvv
Migrating database...
  RUN  '/usr/bin/php8.0' 'vendor/doctrine/migrations/bin/doctrine-migrations.php' 'migrations:migrate' '--no-interaction'
  OUT  
  OUT  
  RES  Command ran successfully                                                                                                                       
 [OK] Database properly migrated!

However, I am getting an error when trying to pull up the short-url:list

$ bin/cli short-url:list
PHP Warning:  require(data/proxies/__CG__ShlinkioShlinkRestEntityApiKey.php): Failed to open stream: No such file or directory in /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php on line 197
PHP Fatal error:  Uncaught Error: Failed opening required 'data/proxies/__CG__ShlinkioShlinkRestEntityApiKey.php' (include_path='.:/usr/share/php') in /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php:197
Stack trace:
#0 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php(109): Doctrine\Common\Proxy\AbstractProxyFactory->getProxyDefinition()
#1 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php(2872): Doctrine\Common\Proxy\AbstractProxyFactory->getProxy()
#2 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php(255): Doctrine\ORM\UnitOfWork->createEntity()
#3 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php(478): Doctrine\ORM\Internal\Hydration\ObjectHydrator->getEntity()
#4 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php(143): Doctrine\ORM\Internal\Hydration\ObjectHydrator->hydrateRowData()
#5 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/AbstractHydrator.php(268): Doctrine\ORM\Internal\Hydration\ObjectHydrator->hydrateAllData()
#6 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php(1194): Doctrine\ORM\Internal\Hydration\AbstractHydrator->hydrateAll()
#7 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php(1135): Doctrine\ORM\AbstractQuery->executeIgnoreQueryCache()
#8 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php(871): Doctrine\ORM\AbstractQuery->execute()
#9 /home/link/domains/serve.link.nwadventists.com/shlink/module/Core/src/Repository/ShortUrlRepository.php(45): Doctrine\ORM\AbstractQuery->getResult()
#10 /home/link/domains/serve.link.nwadventists.com/shlink/module/Core/src/ShortUrl/Paginator/Adapter/ShortUrlRepositoryAdapter.php(26): Shlinkio\Shlink\Core\Repository\ShortUrlRepository->findList()
#11 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/pagerfanta/core/Pagerfanta.php(286): Shlinkio\Shlink\Core\ShortUrl\Paginator\Adapter\ShortUrlRepositoryAdapter->getSlice()
#12 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/pagerfanta/core/Pagerfanta.php(272): Pagerfanta\Pagerfanta->getCurrentPageResultsFromAdapter()
#13 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/pagerfanta/core/Pagerfanta.php(443): Pagerfanta\Pagerfanta->getCurrentPageResults()
#14 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/lstrojny/functional-php/src/Functional/Map.php(31): Pagerfanta\Pagerfanta->getIterator()
#15 /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Command/ShortUrl/ListShortUrlsCommand.php(174): Functional\map()
#16 /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Command/ShortUrl/ListShortUrlsCommand.php(148): Shlinkio\Shlink\CLI\Command\ShortUrl\ListShortUrlsCommand->renderPage()
#17 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Command/Command.php(291): Shlinkio\Shlink\CLI\Command\ShortUrl\ListShortUrlsCommand->execute()
#18 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php(989): Symfony\Component\Console\Command\Command->run()
#19 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php(299): Symfony\Component\Console\Application->doRunCommand()
#20 /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php(171): Symfony\Component\Console\Application->doRun()
#21 /home/link/domains/serve.link.nwadventists.com/shlink/bin/cli(10): Symfony\Component\Console\Application->run()
#22 {main}
  thrown in /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php on line 197

In fact, my entire DB is now without domains. It has short urls, but no domains. So I get short urls like this: https:/v5dan

acelaya commented 2 years ago

What you are missing now are all the steps of the update.

You don't have the generated config (that's why it's not showing the domain, not because you don't have domains in the database), and you don't have the generated proxies (that's why it throws those errors in which it fails to open some files).

antwonw commented 2 years ago

So, I had to do the update, again. Still not clear why I had to do it again, since all the generated_config's migrated over, but...

I finally got it. I decided to use the original untouched 2.10.2 download to kick this all off and migrate my current 2.10.2 to the untouched download of 2.10.2.

$ mv shlink/ shlink-old/ && cp -ap shlink2.10.2_php8.0_dist/ shlink/ && cd shlink/
$ bin/update -vvv

Then I downloaded 3.0.2 again and performed the update from 2.10.2, which I got the previously mentioned migration error.

$ wget -q --show-progress https://github.com/shlinkio/shlink/releases/download/v3.0.2/shlink3.0.2_php8.0_dist.zip && unzip -q shlink3.0.2_php8.0_dist.zip && rm shlink3.0.2_php8.0_dist.zip && mv shlink-old/ shlink-old-old/ && mv shlink/ shlink-old/ && cp -ap shlink3.0.2_php8.0_dist/ shlink/ && cd shlink/ && vendor/bin/shlink-installer update -vvv

I then removed the database from 3.0.2 and copied my old 2.10.2 database over and did the db migrate.

$ rm data/database.sqlite && cp ../shlink-old/data/database.sqlite data/
$ bin/cli db:migrate -vvv

I then performed the install updated again and referenced my 2.10.2 backup directory.

$ vendor/bin/shlink-installer update -vvv

Thank you for the assistance with this. Not sure where and how I messed it all up originally.

One last thing, why do I get this error message each time I update -vvv? I've gotten this in both v2 and now here in v3.

Custom configuration properly generated!

Updating database... [Running "/usr/bin/php8.0 bin/cli db:migrate"]   RUN  '/usr/bin/php8.0' 'bin/cli' 'db:migrate'
  OUT  Migrating database...
  OUT  
  ERR    RUN  '/usr/bin/php8.0' 'vendor/doctrine/migrations/bin/doctrine-migrations.php' 'migrations:migrate' '--no-interaction'
  ERR    OUT  
  ERR    OUT  
  ERR    RES  Command ran successfully
  ERR  
  OUT  
  OUT   [OK] Database properly migrated!

Again, thank you for your patience with this.

acelaya commented 2 years ago

I finally got it.

I'm super glad to hear that. I'm not sure what happened in the middle of the process, but at least everything is there now.

One last thing, why do I get this error message each time I update -vvv? I've gotten this in bot v2 and now here in v3.

It's actually not an error. For some reason, the library I use to run Shlink commands from the CLI, displays that ERR prefix when running a sub-process in verbose mode, which is quite confusing. But if you check, it's actually a successful output.

I'll try to report it to them. Or maybe it's me, that I'm redirecting the standard output of the sub-process to the error output of the parent process. I'll check that as well.

antwonw commented 2 years ago

Hi @acelaya.

So, I'm really not sure what's going on with my install. I just tried to update from 3.0.2 to 3.0.3 and I'm still getting error messages and a failure to migrate.

link@seattle:~/domains/serve.link.nwadventists.com$ ll
total 1304
drwxr-x--- 12 link link    4096 Mar 27 02:03 ./
drwxr-xr-x  4 link link    4096 Mar 27 02:18 ../
-rw-r--r--  1 link link      44 Mar 27 02:02 .virtualmin-src
drwxr-x---  2 link link    4096 Aug 10  2021 cgi-bin/
drwxrwxr-x  2 link link    4096 Feb  7 15:59 data/
drwxr-xr-x  4 link link    4096 Aug 10  2021 etc/
drwxr-xr-x  2 link link    4096 Jul 21  2021 fcgi-bin/
drwxr-xr-x  2 link link    4096 Sep  1  2020 homes/
drwxr-x---  2 link link    4096 Sep  1  2020 logs/
-rw-rw-r--  1 link link 1253376 May 24  2021 original_database.sqlite
lrwxrwxrwx  1 link link      13 Sep  1  2020 public_html -> shlink/public/
drwxr-xr-x  8 link link    4096 Feb 10 12:49 shlink/
drwxr-xr-x  8 link link    4096 Jan  7 12:55 shlink-old/
drwxr-xr-x  8 link link    4096 Feb 10 12:49 shlink3.0.2_php8.0_dist/
-rwxr-xr-x  1 link link    3750 Feb 19 13:17 ssl.ca*
-rwx------  1 link link    2260 Feb 19 13:17 ssl.cert*
-rwx------  1 link link    6012 Feb 19 13:17 ssl.combined*
-rwx------  1 link link    7717 Feb 19 13:17 ssl.everything*
-rwx------  1 link link    1704 Feb 19 13:17 ssl.key*
drwxr-x---  2 link link    4096 Sep  1  2020 tmp/
link@seattle:~/domains/serve.link.nwadventists.com$ wget -q --show-progress https://github.com/shlinkio/shlink/releases/download/v3.0.3/shlink3.0.3_php8.0_dist.zip && unzip -q shlink3.0.3_php8.0_dist.zip && rm shlink3.0.3_php8.0_dist.zip && mv shlink-old/ shlink-old-old/ && mv shlink/ shlink-old/ && cp -ap shlink3.0.3_php8.0_dist/ shlink/ && cd shlink/ && vendor/bin/shlink-installer update -vvv
shlink3.0.3_php8.0_ 100%[===================>]  21.29M  18.3MB/s    in 1.2s    

 Welcome to Shlink!!
 This tool will guide you through the installation process.

 Do you want to import configuration from previous installation? (You will still be asked for any new config option that did not exist in previous shlink versions) (yes/no) [yes]:
 > yes

 Previous shlink installation path from which to import config:
 > ../shlink-old 

 Custom configuration properly generated!

Updating database... [Running "/usr/bin/php8.1 bin/cli db:migrate"]   RUN  '/usr/bin/php8.1' 'bin/cli' 'db:migrate'
  OUT  Migrating database...
  OUT  
  ERR    RUN  '/usr/bin/php8.1' 'vendor/doctrine/migrations/bin/doctrine-migrations.php' 'migrations:migrate' '--no-interaction'
  ERR    ERR  
  ERR    ERR  In SchemaException.php line 35:
  ERR    ERR                                                                    
  ERR    ERR    [Doctrine\DBAL\Schema\SchemaException (10)]                     
  ERR    ERR    There is no table with name 'public.short_urls' in the schema.  
  ERR    ERR                                                                    
  ERR    ERR  
  ERR    ERR  Exception trace:
  ERR    ERR    at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/dbal/src/Schema/SchemaException.php:35
  ERR    ERR   Doctrine\DBAL\Schema\SchemaException::tableDoesNotExist() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/dbal/src/Schema/Schema.php:189
  ERR    ERR   Doctrine\DBAL\Schema\Schema->getTable() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/friendsofphp/proxy-manager-lts/src/ProxyManager/GeneratorStrategy/EvaluatingGeneratorStrategy.php(54) : eval()'d code:49
  ERR    ERR   ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generated83d21ccf7853cfc07dce9110515971db->getTable() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/friendsofphp/proxy-manager-lts/src/ProxyManager/GeneratorStrategy/EvaluatingGeneratorStrategy.php(54) : eval()'d code:49
  ERR    ERR   ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generated83d21ccf7853cfc07dce9110515971db->getTable() at /home/link/domains/serve.link.nwadventists.com/shlink/data/migrations/Version20160819142757.php:28
  ERR    ERR   ShlinkMigrations\Version20160819142757->up() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Version/DbalExecutor.php:167
  ERR    ERR   Doctrine\Migrations\Version\DbalExecutor->executeMigration() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Version/DbalExecutor.php:102
  ERR    ERR   Doctrine\Migrations\Version\DbalExecutor->execute() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/DbalMigrator.php:112
  ERR    ERR   Doctrine\Migrations\DbalMigrator->executePlan() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/DbalMigrator.php:76
  ERR    ERR   Doctrine\Migrations\DbalMigrator->executeMigrations() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/DbalMigrator.php:162
  ERR    ERR   Doctrine\Migrations\DbalMigrator->migrate() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Tools/Console/Command/MigrateCommand.php:210
  ERR    ERR   Doctrine\Migrations\Tools\Console\Command\MigrateCommand->execute() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Command/Command.php:291
  ERR    ERR   Symfony\Component\Console\Command\Command->run() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:989
  ERR    ERR   Symfony\Component\Console\Application->doRunCommand() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:299
  ERR    ERR   Symfony\Component\Console\Application->doRun() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:171
  ERR    ERR   Symfony\Component\Console\Application->run() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrations/Tools/Console/ConsoleRunner.php:97
  ERR    ERR   Doctrine\Migrations\Tools\Console\ConsoleRunner::run() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/bin/doctrine-migrations.php:45
  ERR    ERR   Doctrine\Migrations\{closure}() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/bin/doctrine-migrations.php:46
  ERR    ERR  
  ERR    ERR  migrations:migrate [--write-sql [WRITE-SQL]] [--dry-run] [--query-time] [--allow-no-migration] [--all-or-nothing [ALL-OR-NOTHING]] [--configuration CONFIGURATION] [--em EM] [--conn CONN] [--] [<version>]
  ERR    ERR  
  ERR    ERR  
  ERR  In Process.php line 267:
  ERR                                                                                 
  ERR    [Symfony\Component\Process\Exception\ProcessFailedException]                 
  ERR    The command "'/usr/bin/php8.1' 'vendor/doctrine/migrations/bin/doctrine-mig  
  ERR    rations.php' 'migrations:migrate' '--no-interaction'" failed.                
  ERR                                                                                 
  ERR    Exit Code: 10(Unknown error)                                                 
  ERR                                                                                 
  ERR    Working directory: /home/link/domains/serve.link.nwadventists.com/shlink     
  ERR                                                                                 
  ERR    Output:                                                                      
  ERR    ================                                                             
  ERR                                                                                 
  ERR                                                                                 
  ERR    Error Output:                                                                
  ERR    ================                                                             
  ERR                                                                                 
  ERR    In SchemaException.php line 35:                                              
  ERR                                                                                 
  ERR      [Doctrine\DBAL\Schema\SchemaException (10)]                                
  ERR      There is no table with name 'public.short_urls' in the schema.             
  ERR                                                                                 
  ERR                                                                                 
  ERR    Exception trace:                                                             
  ERR      at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/doctrine/  
  ERR    dbal/src/Schema/SchemaException.php:35                                       
  ERR     Doctrine\DBAL\Schema\SchemaException::tableDoesNotExist() at /home/link/do  
  ERR    mains/serve.link.nwadventists.com/shlink/vendor/doctrine/dbal/src/Schema/Sc  
  ERR    hema.php:189                                                                 
  ERR     Doctrine\DBAL\Schema\Schema->getTable() at /home/link/domains/serve.link.n  
  ERR    wadventists.com/shlink/vendor/friendsofphp/proxy-manager-lts/src/ProxyManag  
  ERR    er/GeneratorStrategy/EvaluatingGeneratorStrategy.php(54) : eval()'d code:49  
  ERR     ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generated83d  
  ERR    21ccf7853cfc07dce9110515971db->getTable() at /home/link/domains/serve.link.  
  ERR    nwadventists.com/shlink/vendor/friendsofphp/proxy-manager-lts/src/ProxyMana  
  ERR    ger/GeneratorStrategy/EvaluatingGeneratorStrategy.php(54) : eval()'d code:4  
  ERR    9                                                                            
  ERR     ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generated83d  
  ERR    21ccf7853cfc07dce9110515971db->getTable() at /home/link/domains/serve.link.  
  ERR    nwadventists.com/shlink/data/migrations/Version20160819142757.php:28         
  ERR     ShlinkMigrations\Version20160819142757->up() at /home/link/domains/serve.l  
  ERR    ink.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrati  
  ERR    ons/Version/DbalExecutor.php:167                                             
  ERR     Doctrine\Migrations\Version\DbalExecutor->executeMigration() at /home/link  
  ERR    /domains/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/  
  ERR    Doctrine/Migrations/Version/DbalExecutor.php:102                             
  ERR     Doctrine\Migrations\Version\DbalExecutor->execute() at /home/link/domains/  
  ERR    serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/  
  ERR    Migrations/DbalMigrator.php:112                                              
  ERR     Doctrine\Migrations\DbalMigrator->executePlan() at /home/link/domains/serv  
  ERR    e.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migr  
  ERR    ations/DbalMigrator.php:76                                                   
  ERR     Doctrine\Migrations\DbalMigrator->executeMigrations() at /home/link/domain  
  ERR    s/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrin  
  ERR    e/Migrations/DbalMigrator.php:162                                            
  ERR     Doctrine\Migrations\DbalMigrator->migrate() at /home/link/domains/serve.li  
  ERR    nk.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migratio  
  ERR    ns/Tools/Console/Command/MigrateCommand.php:210                              
  ERR     Doctrine\Migrations\Tools\Console\Command\MigrateCommand->execute() at /ho  
  ERR    me/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/C  
  ERR    ommand/Command.php:291                                                       
  ERR     Symfony\Component\Console\Command\Command->run() at /home/link/domains/ser  
  ERR    ve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:989   
  ERR     Symfony\Component\Console\Application->doRunCommand() at /home/link/domain  
  ERR    s/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php  
  ERR    :299                                                                         
  ERR     Symfony\Component\Console\Application->doRun() at /home/link/domains/serve  
  ERR    .link.nwadventists.com/shlink/vendor/symfony/console/Application.php:171     
  ERR     Symfony\Component\Console\Application->run() at /home/link/domains/serve.l  
  ERR    ink.nwadventists.com/shlink/vendor/doctrine/migrations/lib/Doctrine/Migrati  
  ERR    ons/Tools/Console/ConsoleRunner.php:97                                       
  ERR     Doctrine\Migrations\Tools\Console\ConsoleRunner::run() at /home/link/domai  
  ERR    ns/serve.link.nwadventists.com/shlink/vendor/doctrine/migrations/bin/doctri  
  ERR    ne-migrations.php:45                                                         
  ERR     Doctrine\Migrations\{closure}() at /home/link/domains/serve.link.nwadventi  
  ERR    sts.com/shlink/vendor/doctrine/migrations/bin/doctrine-migrations.php:46     
  ERR                                                                                 
  ERR    migrations:migrate [--write-sql [WRITE-SQL]] [--dry-run] [--query-time] [--  
  ERR    allow-no-migration] [--all-or-nothing [ALL-OR-NOTHING]] [--configuration CO  
  ERR    NFIGURATION] [--em EM] [--conn CONN] [--] [<version>]                        
  ERR                                                                                 
  ERR  
  ERR  Exception trace:
  ERR    at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/process/Process.php:267
  ERR   Symfony\Component\Process\Process->mustRun() at /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Util/ProcessRunner.php:48
  ERR   Shlinkio\Shlink\CLI\Util\ProcessRunner->run() at /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Command/Db/AbstractDatabaseCommand.php:30
  ERR   Shlinkio\Shlink\CLI\Command\Db\AbstractDatabaseCommand->runPhpCommand() at /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Command/Db/MigrateDatabaseCommand.php:30
  ERR   Shlinkio\Shlink\CLI\Command\Db\MigrateDatabaseCommand->lockedExecute() at /home/link/domains/serve.link.nwadventists.com/shlink/module/CLI/src/Command/Util/AbstractLockedCommand.php:35
  ERR   Shlinkio\Shlink\CLI\Command\Util\AbstractLockedCommand->execute() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Command/Command.php:291
  ERR   Symfony\Component\Console\Command\Command->run() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:989
  ERR   Symfony\Component\Console\Application->doRunCommand() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:299
  ERR   Symfony\Component\Console\Application->doRun() at /home/link/domains/serve.link.nwadventists.com/shlink/vendor/symfony/console/Application.php:171
  ERR   Symfony\Component\Console\Application->run() at /home/link/domains/serve.link.nwadventists.com/shlink/bin/cli:10
  ERR  
  ERR  db:migrate
  ERR  
  ERR  
  RES  1 Command did not run successfully

Am I going to have to manually migrate my database each time I upgrade shlink?

$ rm data/database.sqlite && cp ../shlink-old/data/database.sqlite data/
$ bin/cli db:migrate -vvv
$ vendor/bin/shlink-installer update -vvv

shlinkio / shlink

Unable to upgrade from 2.10.2 > 3.0.0 #1373