Tables appear empty (No entries) after update from 0.6.6 to 0.7.0 official (via App Store)

[osm-frasch]

Steps to reproduce

Steps to reproduce Have Tables 0.6.6 installed Have a table with data Upgrade to Tables 0.7.0 official (via App Store)

Expected behavior

Upgrade works fine
Tables still have their table

Actual behavior

Tables appear empty (No entries/rows are displayed in the table)

Tables app version

0.7.0 official

Browser

Firefox latest, Chrome latest

Client operating system

Linux OpenSuse

Operating system

Linux 4.18.0-513.18.2.el8_9.x86_64 x86_64

Web server

None

PHP engine version

PHP 8.2

Database

MySQL

Additional info

Nextcloud 28.0.4

Systemprotocol: InternalError No column found to build filter with for id -2

[tables] Fehler: No column found to build filter with for id -2 GET /index.php/apps/tables/table von 178.13.21.33 von MaxMuster um 25.04.2024, 10:56:19

{
  "reqId": "ZioaswDdI8MDFKhqcyDKKAAAAAY",
  "level": 3,
  "time": "2024-04-25T08:56:19+00:00",
  "remoteAddr": "178.13.21.33",
  "user": "MaxMuster",
  "app": "tables",
  "method": "GET",
  "url": "/index.php/apps/tables/table",
  "message": "No column found to build filter with for id -2",
  "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0",
  "version": "28.0.4.1",
  "exception": {
    "Exception": "OCA\\Tables\\Errors\\InternalError",
    "Message": "No column found to build filter with for id -2",
    "Code": 0,
    "Trace": [
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Db/Row2Mapper.php",
        "line": 263,
        "function": "getFilter",
        "class": "OCA\\Tables\\Db\\Row2Mapper",
        "type": "->",
        "args": [
          [
            "OC\\DB\\QueryBuilder\\QueryBuilder"
          ],
          [
            [
              "Max Muster",
              -2,
              "is-equal"
            ]
          ]
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Db/Row2Mapper.php",
        "line": 241,
        "function": "getFilterGroups",
        "class": "OCA\\Tables\\Db\\Row2Mapper",
        "type": "->",
        "args": [
          [
            "OC\\DB\\QueryBuilder\\QueryBuilder"
          ],
          [
            [
              [
                "Max Muster",
                -2,
                "is-equal"
              ]
            ]
          ]
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Db/Row2Mapper.php",
        "line": 143,
        "function": "addFilterToQuery",
        "class": "OCA\\Tables\\Db\\Row2Mapper",
        "type": "->",
        "args": [
          [
            "OC\\DB\\QueryBuilder\\QueryBuilder"
          ],
          [
            [
              [
                "Max Muster",
                -2,
                "is-equal"
              ]
            ]
          ],
          "MaxMuster"
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Db/Row2Mapper.php",
        "line": 827,
        "function": "getWantedRowIds",
        "class": "OCA\\Tables\\Db\\Row2Mapper",
        "type": "->",
        "args": [
          "MaxMuster",
          41,
          [
            [
              [
                "@my-name",
                -2,
                "is-equal"
              ]
            ]
          ]
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Service/RowService.php",
        "line": 535,
        "function": "countRowsForView",
        "class": "OCA\\Tables\\Db\\Row2Mapper",
        "type": "->",
        "args": [
          [
            "OCA\\Tables\\Db\\View",
            6
          ],
          "MaxMuster",
          [
            [
              "OCA\\Tables\\Db\\Column",
              168
            ],
            [
              "OCA\\Tables\\Db\\Column",
              169
            ],
            [
              "OCA\\Tables\\Db\\Column",
              177
            ],
            [
              "OCA\\Tables\\Db\\Column",
              198
            ],
            [
              "OCA\\Tables\\Db\\Column",
              199
            ],
            "And 9 more entries, set log level to debug to see all entries"
          ]
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Service/ViewService.php",
        "line": 413,
        "function": "getViewRowsCount",
        "class": "OCA\\Tables\\Service\\RowService",
        "type": "->",
        "args": [
          [
            "OCA\\Tables\\Db\\View",
            6
          ],
          "MaxMuster"
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Service/ViewService.php",
        "line": 90,
        "function": "enhanceView",
        "class": "OCA\\Tables\\Service\\ViewService",
        "type": "->",
        "args": [
          [
            "OCA\\Tables\\Db\\View",
            6
          ],
          "MaxMuster"
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Service/TableService.php",
        "line": 227,
        "function": "findAll",
        "class": "OCA\\Tables\\Service\\ViewService",
        "type": "->",
        "args": [
          [
            "OCA\\Tables\\Db\\Table",
            41
          ]
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Service/TableService.php",
        "line": 149,
        "function": "enhanceTable",
        "class": "OCA\\Tables\\Service\\TableService",
        "type": "->",
        "args": [
          [
            "OCA\\Tables\\Db\\Table",
            41
          ],
          "MaxMuster"
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Controller/TableController.php",
        "line": 39,
        "function": "findAll",
        "class": "OCA\\Tables\\Service\\TableService",
        "type": "->",
        "args": [
          "MaxMuster"
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Controller/Errors.php",
        "line": 16,
        "function": "OCA\\Tables\\Controller\\{closure}",
        "class": "OCA\\Tables\\Controller\\TableController",
        "type": "->",
        "args": [
          "*** sensitive parameters replaced ***"
        ]
      },
      {
        "file": "/home/www/nextcloud/apps/tables/lib/Controller/TableController.php",
        "line": 38,
        "function": "handleError",
        "class": "OCA\\Tables\\Controller\\TableController",
        "type": "->",
        "args": [
          [
            "Closure"
          ]
        ]
      },
      {
        "file": "/home/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php",
        "line": 230,
        "function": "index",
        "class": "OCA\\Tables\\Controller\\TableController",
        "type": "->",
        "args": []
      },
      {
        "file": "/home/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php",
        "line": 137,
        "function": "executeController",
        "class": "OC\\AppFramework\\Http\\Dispatcher",
        "type": "->",
        "args": [
          [
            "OCA\\Tables\\Controller\\TableController"
          ],
          "index"
        ]
      },
      {
        "file": "/home/www/nextcloud/lib/private/AppFramework/App.php",
        "line": 184,
        "function": "dispatch",
        "class": "OC\\AppFramework\\Http\\Dispatcher",
        "type": "->",
        "args": [
          [
            "OCA\\Tables\\Controller\\TableController"
          ],
          "index"
        ]
      },
      {
        "file": "/home/www/nextcloud/lib/private/Route/Router.php",
        "line": 315,
        "function": "main",
        "class": "OC\\AppFramework\\App",
        "type": "::",
        "args": [
          "OCA\\Tables\\Controller\\TableController",
          "index",
          [
            "OC\\AppFramework\\DependencyInjection\\DIContainer"
          ],
          [
            "tables.table.index"
          ]
        ]
      },
      {
        "file": "/home/www/nextcloud/lib/base.php",
        "line": 1069,
        "function": "match",
        "class": "OC\\Route\\Router",
        "type": "->",
        "args": [
          "/apps/tables/table"
        ]
      },
      {
        "file": "/home/www/nextcloud/index.php",
        "line": 39,
        "function": "handleRequest",
        "class": "OC",
        "type": "::",
        "args": []
      }
    ],
    "File": "/home/www/nextcloud/apps/tables/lib/Db/Row2Mapper.php",
    "Line": 276,
    "message": "No column found to build filter with for id -2",
    "exception": [],
    "CustomMessage": "No column found to build filter with for id -2"
  },
  "id": "662a1bbc0c62f"
}

[osm-frasch]

I checked the system log again and found the first entry after the update:

"Try to set no user in context, but request is not allowed. "

{"reqId":"XXXXXXXXXX","level" 2,"time":"2024-04-25T08:44:39+00:00","remoteAddr":"xxx.xx.xx.xx","user":"Max Muster","app":"tables","method":"GET","url":"/index.php/settings/apps/update/tables","message":"Try to set no user in context, but request is not allowed.","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0","version":"28.0.4.1","data":{"app":"tables"},"id":"XXXXXXXXXX"}

Just under a minute later, this message was displayed in the system log

Exception Undefined constant OCA\Tables\AppInfo\Application::NODE_TYPE_TABLE in file '/home/www/nextcloud/apps/tables/lib/Service/TableService.php' line 230

    /home/www/nextcloud/lib/private/AppFramework/App.phpLine 184

    OC\AppFramework\Http\Dispatcher->dispatch(
      [
        "OCA\\Tables\\Controller\\TableController"
      ],
      "index"
    )

    /home/www/nextcloud/lib/private/Route/Router.phpLine 315

    OC\AppFramework\App::main(
      "OCA\\Tables\\Controller\\TableController",
      "index",
      [
        "OC\\AppFramework\\DependencyInjection\\DIContainer"
      ],
      [
        "tables.table.index"
      ]
    )

    /home/www/nextcloud/lib/base.phpLine 1069

    OC\Route\Router->match(
      "/apps/tables/table"
    )

    /home/www/nextcloud/index.phpLine 39

    OC::handleRequest()

Caused by Error Undefined constant OCA\Tables\AppInfo\Application::NODE_TYPE_TABLE

[juliushaertl]

Thanks for reporting. Could you please share the output of the following two database queries?

select * from oc_tables_views where id = 6;
select * from oc_tables_columns where table_id IN (select table_id from oc_tables_views where id = 6);

[osm-frasch]

@juliushaertl I have sent the extract directly to you via github mail

[juliushaertl]

Thanks, I've pushed a potential fix to https://github.com/nextcloud/tables/pull/1002

Testing would be very welcome. You can apply the patch by following the guide in https://docs.nextcloud.com/server/latest/admin_manual/issues/applying_patch.html

[daCaPo]

Applied the patch, but did not resolve #999

[juliushaertl]

Applied the patch, but did not resolve https://github.com/nextcloud/tables/issues/999

Any new or different error?

[daCaPo]

Applied the patch, but did not resolve #999

Any new or different error?

I actually don't receive any error message at all when accessing the tables app.

[osm-frasch]

@juliushaertl Our hoster has just applied the patch. No success so far.

Now the system log says: PHP
Undefined array key -2 at /home/www/nextcloud/apps/tables/lib/Db/Row2Mapper.php#279

[daCaPo]

In the database, the old data is stored in "oc_tables_rows", with the whole record in field "data".

I just added a new record to the "Tutorial" table. Data is stored distributed over the tables "oc_tables_row_cells_datetime", "..._number", "..._selection" and "..._text". Looks like the update script did not migrate the data from the old to the new schema.

[juliushaertl]

The PR has been updated with a few more fixes.

@daCaPo Your issue seems slightly different then. Could you check your logs during the app ugprade for errors and open a separate issue for this?

You can also run the migration from the old to the new database schema manually using the occ tables:legacy:transfer:rows command

[daCaPo]

@juliushaertl Running this gives 🤷🏻‍ Add at least one table id or add the option --all to transfer all tables., and I can't find any documentation for the tables:legacy:transfer:rows command. Is it safe to use --all, or how can I specify to transfer just the Tables app's tables?

@juliushaertl Thanks, that fixed the "empty table" problem! For everyone having the same issue:

occ tables:legacy:transfer:rows --all migrates all tables that require migration. Already migrated tables will just throw an error without further damage.

[osm-frasch]

Yes i can confirm (from my point of view) @juliushaertl the "occ tables:legacy:transfer:rows" command has solved the problem.

• All tables are displayed.
• All table entries are displayed correctly (446 Entries!).
• I was able to create an entry
• and then delete it again.

I was able to make releases and change permissions Many thanks to all involved for their contribution

[pand-herber]

I have a problem after running occ tables:legacy:transfer:rows:

I do see the content of my table with 980 entries, but I can't add new rows to it.

The following error is shown in the logs:

"message": "An internal error or exception occurred: OCA\\Tables\\Service\\RowService - create: An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint \"oc_tables_row_sleeves_pkey\"\nDETAIL: Key (id)=(9) already exists.",

Somehow it seems as if it tries to create entries with key id starting from 1 again, although it is already at 1848.

Is there any way to delete the duplicats and set the key id to 1849 for new entries?

> sudo -u www-data php occ tables:legacy:transfer:rows 21 fails now as well:

Look for given table(s)

-- Start transfer for table 21 (XXXXXXXX) [1/1]
---- Found 11 columns
---- Found 981 rows
⚠️  Could not transfer data. Continue with next table. The logs will have more information about the error: An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constraint "oc_tables_row_sleeves_pkey"
DETAIL:  Key (id)=(1049) already exists.

Thanks for any advice!

[juliushaertl]

If anyone can share their nextcloud logs during the upgrade process that would be much appreciated to see why the automatic migration failed

[kenoiyan]

{"reqId":"","level":1,"time":"2024-04-26T10:25:04+00:00","remoteAddr":"","user":"--","app":"updater","method":"GET","url":"/nextcloud/core/ajax/update.php?requesttoken=","message":"\\OC\\Updater::upgradeAppStoreApp: Update app \"tables\" from appstore","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0","version":"28.0.4.1","data":{"app":"updater"}}

{"reqId":"","level":1,"time":"2024-04-26T10:25:34+00:00","remoteAddr":"","user":"--","app":"updater","method":"GET","url":"/nextcloud/core/ajax/update.php?requesttoken=","message":"OC\\Repair\\Events\\RepairStepEvent: Repair step: Copy the data into the new db structure","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0","version":"28.0.4.1","data":{"app":"updater"}}

{"reqId":"","level":1,"time":"2024-04-26T10:25:34+00:00","remoteAddr":"","user":"--","app":"updater","method":"GET","url":"/nextcloud/core/ajax/update.php?requesttoken=","message":"OC\\Repair\\Events\\RepairInfoEvent: Repair info: Look for tables","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0","version":"28.0.4.1","data":{"app":"updater"}}

{"reqId":"","level":2,"time":"2024-04-26T10:25:34+00:00","remoteAddr":"","user":"--","app":"tables","method":"GET","url":"/nextcloud/core/ajax/update.php?requesttoken=","message":"Try to set no user in context, but request is not allowed.","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0","version":"28.0.4.1","data":{"app":"tables"}}

{"reqId":"","level":2,"time":"2024-04-26T10:25:34+00:00","remoteAddr":"","user":"--","app":"updater","method":"GET","url":"/nextcloud/core/ajax/update.php?requesttoken=","message":"OC\\Repair\\Events\\RepairWarningEvent: Repair warning: Error while fetching tables. Will aboard.","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0","version":"28.0.4.1","data":{"app":"updater"}}

[Tokens and IP addresses removed from log.]

[kinimodmeyer]

https://github.com/nextcloud/tables/blob/main/lib/Db/Row2Mapper.php#L612

Are you sure that method "createRowSleeveFromExistingData" should do a setId() before the insert? :thinking:

[kenoiyan]

The #1002 update has unfortunately not fixed the issue. The tables are still empty.

[thisiszeev]

Thank you all for this thread.

I had a near panic attack when I saw all the business leads we had been capturing were missing. Checked the MariaDB tables for the tables and app and found it all there, but in a different format to what I was used to. Came to github to see if anyone else had reported it and if anyone had solved it.

occ tables:legacy:transfer:rows --all worked like a charm.

[Tr4sK]

I got worried :fearful: for a while before googling the issue once I saw that all my content was still in the DB.

:relaxed: