search

impresso / impresso-pycommons

Python module with bits of code (objects, functions) highly reusable within impresso.
http://impresso-pycommons.rtfd.io/
GNU Affero General Public License v3.0
3 stars 3 forks source link

[rebuild] non existing page ID in FedGazDe #58

Open mromanello opened 4 years ago

mromanello commented 4 years ago

The rebuild of FedGazDe-1849 runs through but 3 content items present problems (I got 171 items in input, and only 168 in output). The reason is some page IDs that are referenced in the issue JSON document do not exist in the page JSON (see logs below).

2020-04-30 17:03:31,314 impresso_commons.text.helpers ERROR    Page p0014 not found for item FedGazDe-1849-11-24-a-i0001Issue FedGazDe-1849-11-24-a has pages ['FedGazDe-1849-11-24-a-p0001', 'FedGazDe-1849-11-24-a-p0002', 'FedGazDe-1849-11-24-a-p0003', 'FedGazDe-1849-11-24-a-p0004', 'FedGazDe-1849-11-24-a-p0005', 'FedGazDe-1849-11-24-a-p0006', 'FedGazDe-1849-11-24-a-p0007', 'FedGazDe-1849-11-24-a-p0008', 'FedGazDe-1849-11-24-a-p0009', 'FedGazDe-1849-11-24-a-p0010', 'FedGazDe-1849-11-24-a-p0011', 'FedGazDe-1849-11-24-a-p0012', 'FedGazDe-1849-11-24-a-p0013']
2020-04-30 17:03:32,585 impresso_commons.text.helpers ERROR    Page p0019 not found for item FedGazDe-1849-05-22-a-i0001Issue FedGazDe-1849-05-22-a has pages ['FedGazDe-1849-05-22-a-p0001', 'FedGazDe-1849-05-22-a-p0002', 'FedGazDe-1849-05-22-a-p0003', 'FedGazDe-1849-05-22-a-p0004', 'FedGazDe-1849-05-22-a-p0005', 'FedGazDe-1849-05-22-a-p0006', 'FedGazDe-1849-05-22-a-p0007', 'FedGazDe-1849-05-22-a-p0008', 'FedGazDe-1849-05-22-a-p0009', 'FedGazDe-1849-05-22-a-p0010', 'FedGazDe-1849-05-22-a-p0011', 'FedGazDe-1849-05-22-a-p0012', 'FedGazDe-1849-05-22-a-p0013', 'FedGazDe-1849-05-22-a-p0014', 'FedGazDe-1849-05-22-a-p0015', 'FedGazDe-1849-05-22-a-p0016', 'FedGazDe-1849-05-22-a-p0017', 'FedGazDe-1849-05-22-a-p0018', 'FedGazDe-1849-05-22-a-p0020', 'FedGazDe-1849-05-22-a-p0021', 'FedGazDe-1849-05-22-a-p0022', 'FedGazDe-1849-05-22-a-p0023', 'FedGazDe-1849-05-22-a-p0024', 'FedGazDe-1849-05-22-a-p0025', 'FedGazDe-1849-05-22-a-p0026', 'FedGazDe-1849-05-22-a-p0027', 'FedGazDe-1849-05-22-a-p0028', 'FedGazDe-1849-05-22-a-p0029', 'FedGazDe-1849-05-22-a-p0030', 'FedGazDe-1849-05-22-a-p0031', 'FedGazDe-1849-05-22-a-p0032', 'FedGazDe-1849-05-22-a-p0033', 'FedGazDe-1849-05-22-a-p0034', 'FedGazDe-1849-05-22-a-p0035', 'FedGazDe-1849-05-22-a-p0036', 'FedGazDe-1849-05-22-a-p0037', 'FedGazDe-1849-05-22-a-p0038', 'FedGazDe-1849-05-22-a-p0039', 'FedGazDe-1849-05-22-a-p0040', 'FedGazDe-1849-05-22-a-p0041', 'FedGazDe-1849-05-22-a-p0042', 'FedGazDe-1849-05-22-a-p0043', 'FedGazDe-1849-05-22-a-p0044', 'FedGazDe-1849-05-22-a-p0045', 'FedGazDe-1849-05-22-a-p0046', 'FedGazDe-1849-05-22-a-p0047', 'FedGazDe-1849-05-22-a-p0048', 'FedGazDe-1849-05-22-a-p0049']
2020-04-30 17:04:29,972 impresso_commons.text.helpers ERROR    Page p0021 not found for item FedGazDe-1849-03-14-a-i0001Issue FedGazDe-1849-03-14-a has pages ['FedGazDe-1849-03-14-a-p0001', 'FedGazDe-1849-03-14-a-p0002', 'FedGazDe-1849-03-14-a-p0003', 'FedGazDe-1849-03-14-a-p0004', 'FedGazDe-1849-03-14-a-p0005', 'FedGazDe-1849-03-14-a-p0006', 'FedGazDe-1849-03-14-a-p0007', 'FedGazDe-1849-03-14-a-p0008', 'FedGazDe-1849-03-14-a-p0009', 'FedGazDe-1849-03-14-a-p0010', 'FedGazDe-1849-03-14-a-p0011', 'FedGazDe-1849-03-14-a-p0012', 'FedGazDe-1849-03-14-a-p0013', 'FedGazDe-1849-03-14-a-p0014', 'FedGazDe-1849-03-14-a-p0015', 'FedGazDe-1849-03-14-a-p0016', 'FedGazDe-1849-03-14-a-p0017', 'FedGazDe-1849-03-14-a-p0018', 'FedGazDe-1849-03-14-a-p0019', 'FedGazDe-1849-03-14-a-p0020', 'FedGazDe-1849-03-14-a-p0022', 'FedGazDe-1849-03-14-a-p0023', 'FedGazDe-1849-03-14-a-p0024', 'FedGazDe-1849-03-14-a-p0025', 'FedGazDe-1849-03-14-a-p0026', 'FedGazDe-1849-03-14-a-p0027', 'FedGazDe-1849-03-14-a-p0028', 'FedGazDe-1849-03-14-a-p0029', 'FedGazDe-1849-03-14-a-p0030', 'FedGazDe-1849-03-14-a-p0031', 'FedGazDe-1849-03-14-a-p0032', 'FedGazDe-1849-03-14-a-p0033', 'FedGazDe-1849-03-14-a-p0034', 'FedGazDe-1849-03-14-a-p0035', 'FedGazDe-1849-03-14-a-p0036', 'FedGazDe-1849-03-14-a-p0037']

@aflueckiger can you have a look to see what's happening there?

mromanello commented 4 years ago

here below the complete list of years that present likely the same problem (extracted from the rebuild log):

Skipped articles: ['FedGazDe-1849-05-22-a-i0001', 'FedGazDe-1849-05-22-a-i0001', 'FedGazDe-1849-03-14-a-i0001', 'FedGazDe-1849-03-14-a-i0001', 'FedGazDe-1849-11-24-a-i0001', 'FedGazDe-1849-11-24-a-i0001']
Skipped articles: ['FedGazDe-1850-01-14-a-i0003', 'FedGazDe-1850-01-14-a-i0003']
Skipped articles: ['FedGazDe-1875-04-17-a-i0004', 'FedGazDe-1875-04-17-a-i0004']
Skipped articles: ['FedGazDe-1876-01-08-a-i0001', 'FedGazDe-1876-01-08-a-i0001']
Skipped articles: ['FedGazDe-1880-09-25-a-i0006', 'FedGazDe-1880-09-25-a-i0006', 'FedGazDe-1880-08-14-a-i0005', 'FedGazDe-1880-08-14-a-i0005', 'FedGazDe-1880-08-14-a-i0005', 'FedGazDe-1880-08-21-a-i0005', 'FedGazDe-1880-08-21-a-i0005', 'FedGazDe-1880-07-31-a-i0005', 'FedGazDe-1880-07-31-a-i0005', 'FedGazDe-1880-07-31-a-i0005']
Skipped articles: ['FedGazDe-1881-04-02-a-i0007', 'FedGazDe-1881-04-02-a-i0007', 'FedGazDe-1881-04-02-a-i0007', 'FedGazDe-1881-11-26-a-i0007', 'FedGazDe-1881-11-26-a-i0007', 'FedGazDe-1881-11-26-a-i0007']
Skipped articles: ['FedGazDe-1883-04-14-a-i0005', 'FedGazDe-1883-04-14-a-i0005', 'FedGazDe-1883-03-17-a-i0008', 'FedGazDe-1883-03-17-a-i0008', 'FedGazDe-1883-03-17-a-i0008']
Skipped articles: ['FedGazDe-1884-03-01-a-i0004', 'FedGazDe-1884-03-01-a-i0004', 'FedGazDe-1884-11-22-a-i0006', 'FedGazDe-1884-11-22-a-i0006', 'FedGazDe-1884-01-19-a-i0005', 'FedGazDe-1884-01-19-a-i0005', 'FedGazDe-1884-01-19-a-i0005']
Skipped articles: ['FedGazDe-1886-01-30-a-i0002', 'FedGazDe-1886-01-30-a-i0002', 'FedGazDe-1886-01-30-a-i0002']
Skipped articles: ['FedGazDe-1904-05-18-a-i0004', 'FedGazDe-1904-05-18-a-i0004']
Skipped articles: ['FedGazDe-1914-02-25-a-i0005', 'FedGazDe-1914-02-25-a-i0005', 'FedGazDe-1914-02-25-a-i0005', 'FedGazDe-1914-08-05-a-i0006', 'FedGazDe-1914-08-05-a-i0006', 'FedGazDe-1914-08-05-a-i0006', 'FedGazDe-1914-06-17-a-i0005', 'FedGazDe-1914-06-17-a-i0005', 'FedGazDe-1914-06-17-a-i0005', 'FedGazDe-1914-05-20-a-i0009', 'FedGazDe-1914-05-20-a-i0009', 'FedGazDe-1914-05-20-a-i0009', 'FedGazDe-1914-04-22-a-i0003', 'FedGazDe-1914-04-22-a-i0003', 'FedGazDe-1914-04-22-a-i0003', 'FedGazDe-1914-03-11-a-i0008', 'FedGazDe-1914-03-11-a-i0008', 'FedGazDe-1914-03-11-a-i0008', 'FedGazDe-1914-11-25-a-i0008', 'FedGazDe-1914-11-25-a-i0008', 'FedGazDe-1914-11-25-a-i0008', 'FedGazDe-1914-01-14-a-i0004', 'FedGazDe-1914-01-14-a-i0004', 'FedGazDe-1914-01-14-a-i0004', 'FedGazDe-1914-01-28-a-i0006', 'FedGazDe-1914-01-28-a-i0006', 'FedGazDe-1914-01-28-a-i0006']
Skipped articles: ['FedGazDe-1915-12-15-a-i0004', 'FedGazDe-1915-12-15-a-i0004', 'FedGazDe-1915-12-15-a-i0004', 'FedGazDe-1915-10-06-a-i0005', 'FedGazDe-1915-10-06-a-i0005', 'FedGazDe-1915-10-06-a-i0005', 'FedGazDe-1915-09-15-a-i0004', 'FedGazDe-1915-09-15-a-i0004', 'FedGazDe-1915-09-15-a-i0004', 'FedGazDe-1915-07-21-a-i0004', 'FedGazDe-1915-07-21-a-i0004', 'FedGazDe-1915-07-21-a-i0004', 'FedGazDe-1915-06-23-a-i0005', 'FedGazDe-1915-06-23-a-i0005', 'FedGazDe-1915-06-23-a-i0005', 'FedGazDe-1915-05-19-a-i0004', 'FedGazDe-1915-05-19-a-i0004', 'FedGazDe-1915-05-19-a-i0004', 'FedGazDe-1915-05-05-a-i0004', 'FedGazDe-1915-05-05-a-i0004', 'FedGazDe-1915-05-05-a-i0004', 'FedGazDe-1915-04-28-a-i0007', 'FedGazDe-1915-04-28-a-i0007', 'FedGazDe-1915-04-28-a-i0007']
Skipped articles: ['FedGazDe-1916-12-13-a-i0008', 'FedGazDe-1916-12-13-a-i0008', 'FedGazDe-1916-12-13-a-i0008', 'FedGazDe-1916-02-16-a-i0003', 'FedGazDe-1916-02-16-a-i0003', 'FedGazDe-1916-02-16-a-i0003', 'FedGazDe-1916-02-02-a-i0005', 'FedGazDe-1916-02-02-a-i0005', 'FedGazDe-1916-02-02-a-i0005', 'FedGazDe-1916-10-25-a-i0005', 'FedGazDe-1916-10-25-a-i0005', 'FedGazDe-1916-10-25-a-i0005', 'FedGazDe-1916-10-18-a-i0003', 'FedGazDe-1916-10-18-a-i0003', 'FedGazDe-1916-10-18-a-i0003', 'FedGazDe-1916-10-11-a-i0005', 'FedGazDe-1916-10-11-a-i0005', 'FedGazDe-1916-10-11-a-i0005', 'FedGazDe-1916-07-12-a-i0004', 'FedGazDe-1916-07-12-a-i0004', 'FedGazDe-1916-07-12-a-i0004', 'FedGazDe-1916-06-21-a-i0003', 'FedGazDe-1916-06-21-a-i0003', 'FedGazDe-1916-06-21-a-i0003', 'FedGazDe-1916-03-08-a-i0006', 'FedGazDe-1916-03-08-a-i0006', 'FedGazDe-1916-03-08-a-i0006', 'FedGazDe-1916-11-08-a-i0007', 'FedGazDe-1916-11-08-a-i0007', 'FedGazDe-1916-11-08-a-i0007']
Skipped articles: ['FedGazDe-1917-07-25-a-i0004', 'FedGazDe-1917-07-25-a-i0004', 'FedGazDe-1917-07-25-a-i0004', 'FedGazDe-1917-07-04-a-i0004', 'FedGazDe-1917-07-04-a-i0004', 'FedGazDe-1917-07-04-a-i0004', 'FedGazDe-1917-06-20-a-i0008', 'FedGazDe-1917-06-20-a-i0008', 'FedGazDe-1917-06-20-a-i0008', 'FedGazDe-1917-05-02-a-i0006', 'FedGazDe-1917-05-02-a-i0006', 'FedGazDe-1917-05-02-a-i0006', 'FedGazDe-1917-03-07-a-i0007', 'FedGazDe-1917-03-07-a-i0007', 'FedGazDe-1917-03-07-a-i0007', 'FedGazDe-1917-03-21-a-i0009', 'FedGazDe-1917-03-21-a-i0009', 'FedGazDe-1917-03-21-a-i0009', 'FedGazDe-1917-01-10-a-i0004', 'FedGazDe-1917-01-10-a-i0004', 'FedGazDe-1917-01-10-a-i0004']
Skipped articles: ['FedGazDe-1918-02-06-a-i0003', 'FedGazDe-1918-02-06-a-i0003', 'FedGazDe-1918-02-06-a-i0003', 'FedGazDe-1918-07-10-a-i0010', 'FedGazDe-1918-07-10-a-i0010', 'FedGazDe-1918-07-10-a-i0010', 'FedGazDe-1918-03-13-a-i0006', 'FedGazDe-1918-03-13-a-i0006', 'FedGazDe-1918-03-13-a-i0006', 'FedGazDe-1918-01-30-a-i0003', 'FedGazDe-1918-01-30-a-i0003', 'FedGazDe-1918-01-30-a-i0003']
Skipped articles: ['FedGazDe-1953-05-28-a-i0005', 'FedGazDe-1953-05-28-a-i0005', 'FedGazDe-1953-05-28-a-i0005']
aflueckiger commented 4 years ago

I am on it. Right now, there are other priorities though. Controll the new heuristics is still due.

Related issue: https://github.com/impresso/impresso-text-acquisition/issues/70