Integrity issue when running pihole-updatelists - solutions for the same problem have not worked

[O-J1]

Hi,

I attempted to load both the tick list for my 'relaxed' client group and the nocross for my default client group but ran into SQL integrity issues that I dont really understand. Apologies if its something bleedingly obvious to you

I have tried all the suggestions in the prior 3 existing closed issues for "integrity" problems to no avail. Attached below is my verbose debug log - password is github

https://bin.0xfc.de/?1ca69cb1f4775492#4zANeyEgh6LLQDMaVEKGQe5JUCFYPnCWQPyPjaPoGuas

Regards, OJ

[jacklul]

It appears that GROUPA_ADLISTS section is empty in the config? Would you mind sharing your full config so I can test on a clean database ?

This might actually be a bug in the code because it tries to insert something that already exists but is of different type. Maybe redirector.googlevideo.com already exists in your Domains list?

[O-J1]

Hey mate, how might I share this full config? also apologies your response I completely missed the email notification on this :(

[jacklul]

You can just paste /etc/pihole-updatelists.conf here, in code block, I guess?

[O-J1]

Commented out Group A as a workaround. Essentially I have two tiers on my network. Less restricted and more restricted.

; https://github.com/jacklul/pihole-updatelists
; For a full list of available variables please see the readme.

; Remote list URL containing list of adlists to import
; URLs to single adlists are not supported here!
[GroupA_adlists]
#ADLISTS_URL="https://v.firebog.net/hosts/lists.php?type=tick"
#GROUP_ID=-1
#COMMENT="https://firebog.net - safe lists only"

[GroupB_adlists]
ADLISTS_URL="https://v.firebog.net/hosts/lists.php?type=nocross"
GROUP_ID=-0
COMMENT="Firebog - nocross"

; Remote list URL containing exact domains to whitelist
WHITELIST_URL="https://raw.githubusercontent.com/anudeepND/whitelist/master/dom>
COMMENT="anudeepND whitelist"

[jacklul]

I can already tell there is misconfiguration in your config file - sections should be at the bottom of the file. In your config WHITELIST_URL is assigned to GroupB_adlists section.

In your log entry redirector.googlevideo.com is causing constrain violation - meaning the record somehow already exists (my script fetches full list of entries before doing the job to prevent this issue), in my test I changed type of redirector.googlevideo.com to be a blacklisted domain and my script correctly detected it as a conflict and did not error out.

Can you check if redirector.googlevideo.com does not already exist in your setup and is it blacklisted or whitelisted (or maybe mistakenly added to regex list)?

[O-J1]

It is somehow both whitelisted and blacklisted, Ive also readjusted the config file as well to have the sections at the bottom

[jacklul]

Please try the develop branch version and tell me if the constrain error still happens - I wasn't able to reproduce the issue on my end. _Set GIT_BRANCH=develop and run pihole-updatelists --update to update to the develop branch._

[O-J1]

Hi Jack,

I can report back with success (not using the dev branch, just had IRL stuff to do). After deleting the redirector.googlevideo.com from blacklist and whitelist + editing the config file to be constructed propely then running pihole-updatelists > pihole updateGravity. All seems to have worked without a hitch after those crucial edits and deleting.

My apologies for the user error 🤦‍♂️

[jacklul]

Glad you got it working, if you encounter this issue again do not hesitate to reopen this issue!