search

LeMyst / WikibaseIntegrator

A Python module to manipulate data on a Wikibase instance (like Wikidata) through the MediaWiki Wikibase API and the Wikibase SPARQL endpoint.
MIT License
68 stars 14 forks source link

RC1: WBI tries to upload "corrupt datavalue" #294

Closed dpriskorn closed 2 years ago

dpriskorn commented 2 years ago

code is here https://github.com/dpriskorn/OpenAlexBot/tree/72edd412417e3b4c9c7f91ceaada039e77e8dbd4

DEBUG:openalexbot:Printing the item json

{
    'labels': {
        'en': {
            'language': 'en',
            'value': 'Study of Chromium Activity in the Cr-Fe-N System by 
Galvanic Cell Method'
        }
    },
    'descriptions': {
        'en': {'language': 'en', 'value': 'scientific article from 2010'}
    },
    'aliases': {},
    'type': 'item',
    'claims': {
        'P921': [
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 725,
                            'id': 'Q725'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C511782168',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 209440,
                            'id': 'Q209440'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C29633239',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 228736,
                            'id': 'Q228736'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C192562407',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 11467,
                            'id': 'Q11467'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C191897082',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 2660666,
                            'id': 'Q2660666'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C2781468121',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 2329,
                            'id': 'Q2329'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C185592680',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 1087583,
                            'id': 'Q1087583'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C205688572',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 243545,
                            'id': 'Q243545'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C13965031',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 37756,
                            'id': 'Q37756'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C2780026712',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            },
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P921',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 11165,
                            'id': 'Q11165'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'C179104552',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            }
        ],
        'P2860': [
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P2860',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 21563759,
                            'id': 'Q21563759'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'W2024095314',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            }
        ],
        'P356': [
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P356',
                    'datatype': 'external-id',
                    'datavalue': {
                        'value': '10.1007/s11661-009-0053-5',
                        'type': 'string'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'W2024095314',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            }
        ],
        'P31': [
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P31',
                    'datatype': 'wikibase-item',
                    'datavalue': {
                        'value': {
                            'entity-type': 'item',
                            'numeric-id': 13442814,
                            'id': 'Q13442814'
                        },
                        'type': 'wikibase-entityid'
                    }
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'W2024095314',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            }
        ],
        'P1476': [
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P1476',
                    'datatype': 'monolingualtext',
                    'datavalue': {}
                },
                'type': 'statement',
                'rank': 'normal',
                'references': [
                    {
                        'snaks': {
                            'P813': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P813',
                                    'datatype': 'time',
                                    'datavalue': {
                                        'value': {
                                            'time': '+2022-02-16T00:00:00Z',
                                            'before': 0,
                                            'after': 0,
                                            'precision': 11,
                                            'timezone': 0,
                                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                                        },
                                        'type': 'time'
                                    }
                                }
                            ],
                            'P248': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P248',
                                    'datatype': 'wikibase-item',
                                    'datavalue': {
                                        'value': {
                                            'entity-type': 'item',
                                            'numeric-id': 107507571,
                                            'id': 'Q107507571'
                                        },
                                        'type': 'wikibase-entityid'
                                    }
                                }
                            ],
                            'P10283': [
                                {
                                    'snaktype': 'value',
                                    'property': 'P10283',
                                    'datatype': 'external-id',
                                    'datavalue': {
                                        'value': 'W2024095314',
                                        'type': 'string'
                                    }
                                }
                            ]
                        },
                        'snaks-order': []
                    }
                ]
            }
        ],
        'P577': [
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P577',
                    'datatype': 'time',
                    'datavalue': {
                        'value': {
                            'time': '+2010-01-01T00:00:00Z',
                            'before': 0,
                            'after': 0,
                            'precision': 11,
                            'timezone': 0,
                            'calendarmodel': 
'http://www.wikidata.org/entity/Q1985727'
                        },
                        'type': 'time'
                    }
                },
                'type': 'statement',
                'rank': 'normal'
            }
        ],
        'P433': [
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P433',
                    'datatype': 'string',
                    'datavalue': {'value': '1', 'type': 'string'}
                },
                'type': 'statement',
                'rank': 'normal'
            }
        ]
    }
}

DEBUG:wikibaseintegrator.entities.baseentity:{'action': 'wbeditentity', 'data': '{"labels":{"en":{"language":"en","value":"Study of Chromium Activity in the Cr-Fe-N System by Galvanic Cell Method"}},"descriptions":{"en":{"language":"en","value":"scientific article from 2010"}},"aliases":{},"type":"item","claims":{"P921":[{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":725,"id":"Q725"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C511782168","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":209440,"id":"Q209440"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C29633239","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":228736,"id":"Q228736"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C192562407","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":11467,"id":"Q11467"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C191897082","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":2660666,"id":"Q2660666"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C2781468121","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":2329,"id":"Q2329"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C185592680","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":1087583,"id":"Q1087583"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C205688572","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":243545,"id":"Q243545"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C13965031","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":37756,"id":"Q37756"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C2780026712","type":"string"}}]},"snaks-order":[]}]},{"mainsnak":{"snaktype":"value","property":"P921","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":11165,"id":"Q11165"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"C179104552","type":"string"}}]},"snaks-order":[]}]}],"P2860":[{"mainsnak":{"snaktype":"value","property":"P2860","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":21563759,"id":"Q21563759"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"W2024095314","type":"string"}}]},"snaks-order":[]}]}],"P356":[{"mainsnak":{"snaktype":"value","property":"P356","datatype":"external-id","datavalue":{"value":"10.1007\\/s11661-009-0053-5","type":"string"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"W2024095314","type":"string"}}]},"snaks-order":[]}]}],"P31":[{"mainsnak":{"snaktype":"value","property":"P31","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":13442814,"id":"Q13442814"},"type":"wikibase-entityid"}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"W2024095314","type":"string"}}]},"snaks-order":[]}]}],"P1476":[{"mainsnak":{"snaktype":"value","property":"P1476","datatype":"monolingualtext","datavalue":{}},"type":"statement","rank":"normal","references":[{"snaks":{"P813":[{"snaktype":"value","property":"P813","datatype":"time","datavalue":{"value":{"time":"+2022-02-16T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}}],"P248":[{"snaktype":"value","property":"P248","datatype":"wikibase-item","datavalue":{"value":{"entity-type":"item","numeric-id":107507571,"id":"Q107507571"},"type":"wikibase-entityid"}}],"P10283":[{"snaktype":"value","property":"P10283","datatype":"external-id","datavalue":{"value":"W2024095314","type":"string"}}]},"snaks-order":[]}]}],"P577":[{"mainsnak":{"snaktype":"value","property":"P577","datatype":"time","datavalue":{"value":{"time":"+2010-01-01T00:00:00Z","before":0,"after":0,"precision":11,"timezone":0,"calendarmodel":"http:\\/\\/www.wikidata.org\\/entity\\/Q1985727"},"type":"time"}},"type":"statement","rank":"normal"}],"P433":[{"mainsnak":{"snaktype":"value","property":"P433","datatype":"string","datavalue":{"value":"1","type":"string"}},"type":"statement","rank":"normal"}]}}', 'format': 'json', 'summary': 'New item imported from OpenAlex', 'new': 'item'}
WARNING:wikibaseintegrator.wbi_helpers:WARNING: Please set an user agent if you interact with a Wikibase instance from the Wikimedia Foundation.
WARNING:wikibaseintegrator.wbi_helpers:More information in the README.md and https://meta.wikimedia.org/wiki/User-Agent_policy
DEBUG:urllib3.connectionpool:https://www.wikidata.org:443 "POST /w/api.php HTTP/1.1" 200 461

Traceback (most recent call last):
  File "C:/Users/E5450/PycharmProjects/OpenAlexBot/swepub.py", line 4, in <module>
    oa.start()
  File "C:\Users\E5450\PycharmProjects\OpenAlexBot\openalexbot\__init__.py", line 404, in start
    self.__process_dois__()
  File "C:\Users\E5450\PycharmProjects\OpenAlexBot\openalexbot\__init__.py", line 367, in __process_dois__
    self.__import_new_item__(doi=doi, work=work, wbi=wbi)
  File "C:\Users\E5450\PycharmProjects\OpenAlexBot\openalexbot\__init__.py", line 116, in __import_new_item__
    self.__upload_new_item__(item=self.__prepare_new_item__(doi=doi, work=work, wbi=wbi))
  File "C:\Users\E5450\PycharmProjects\OpenAlexBot\openalexbot\__init__.py", line 388, in __upload_new_item__
    new_item = item.write(summary="New item imported from OpenAlex")
  File "C:\Users\E5450\PycharmProjects\OpenAlexBot\venv\lib\site-packages\wikibaseintegrator\entities\item.py", line 88, in write
    json_data = super()._write(data=self.get_json(), **kwargs)
  File "C:\Users\E5450\PycharmProjects\OpenAlexBot\venv\lib\site-packages\wikibaseintegrator\entities\baseentity.py", line 154, in _write
    json_result: dict = mediawiki_api_call_helper(data=payload, login=self.api.login, allow_anonymous=allow_anonymous, is_bot=self.api.is_bot, **kwargs)
  File "C:\Users\E5450\PycharmProjects\OpenAlexBot\venv\lib\site-packages\wikibaseintegrator\wbi_helpers.py", line 187, in mediawiki_api_call_helper
    return mediawiki_api_call('POST', mediawiki_api_url=mediawiki_api_url, session=login_session, data=data, headers=headers, max_retries=max_retries,
  File "C:\Users\E5450\PycharmProjects\OpenAlexBot\venv\lib\site-packages\wikibaseintegrator\wbi_helpers.py", line 112, in mediawiki_api_call
    raise MWApiError(response.json() if response else {})
wikibaseintegrator.wbi_exceptions.MWApiError: {'error': {'code': 'modification-failed', 'info': 'Data value corrupt:', 'messages': [{'name': 'wikibase-validator-bad-value', 'parameters': [''], 'html': {'*': 'Data value corrupt: '}}], '*': 'See https://www.wikidata.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/&gt; for notice of API deprecations and breaking changes.'}, 'servedby': 'mw1428'}
Error while writing to the Wikibase instance
LeMyst commented 2 years ago

I wonder what part is "corrupted" in this 🤔

dpriskorn commented 2 years ago

I wonder what part is "corrupted" in this 🤔

Me too. I have not taken a closer look. I could try to edit the code to 0 references and see if that helps. I'll try to nail down which part triggers the MW error.

dpriskorn commented 2 years ago

I tried reproducing the issue, but have not succeeded yet. Here is the latest item made with the code. https://www.wikidata.org/wiki/Q110951928 https://www.wikidata.org/wiki/Q110951998 here is another. It seems to work fine. I think we should close this and reopen if anyone can reproduce it.

micgro42 commented 1 year ago

I would expect this to be the corrupt data value causing the error:

        'P1476': [
            {
                'mainsnak': {
                    'snaktype': 'value',
                    'property': 'P1476',
                    'datatype': 'monolingualtext',
                    'datavalue': {}
                },