NCSC-NL / taranis3

Taranis
Other
59 stars 17 forks source link

db archive problem taranis 3.6.2 #25

Closed dagobertdebug closed 4 years ago

dagobertdebug commented 5 years ago

hi

the following error occurs

taranis@VM:~/taranis-3.6.2/bin$ ./taranis db archive -b 20190701
Archiving records before 2019-07-01
(+    0s) 13:32:30 record count before archiving:
  item                                         184028
  item_archive                                      0
  TOTAL                                        184028

  email_item                                        0
  email_item_archive                                0
  TOTAL                                             0

  identifier                                        0
  identifier_archive                                0
  TOTAL                                             0

(+    0s) 13:32:30 Collecting tagged items, to be excluded
excluding 0 tagged items
(+    0s) 13:32:30 Archiving email records (table email_item)
(+    0s) 13:32:30 Delete archived email items
(+    0s) 13:32:30 Archiving feed records (table item)
DBD::Pg::st execute failed: ERROR:  value too long for type character varying(16) at /home/taranis/taranis-3.6.2/perl5/Taranis/Command/DB.pm line 218.

Any workaround or bugfix?

Many thanks!

markov2 commented 5 years ago

In the database schema, item_archive.id is a varchar(16) which contains a copy of item.id of type integer. Wow, your id has grown very big.

Release 3.7 will repair this inconsistency with schema update 3602. This will execute

ALTER TABLE item_archive
   ALTER COLUMN id TYPE integer USING id::integer;

You could execute this now, manually, but be aware that your upgrade to 3.7 will break. During upgrading towards 3.7 (to be released after the holiday season) you need to put "exit 0" in the sub in file install/db-upgrade-scripts/3602.item-archive-id-type

markov2 commented 5 years ago

HINT: since we use Postgress' "modern" full text search indices, we stopped archiving items: performance does not degrade with the number of items anymore.

dagobertdebug commented 4 years ago

Thanks for your response, will wait for release 3.7

And I have no idea why the id has grown so big.