K-vanc / Tempest-EPG-Generator

The fastest XMLTV formatted EPG Generator with GUI and the lowest memory usage
https://hub.docker.com/r/kvanc/tempest_epg
Other
100 stars 17 forks source link

[QUESTION] Possibility to delete dublicates #121

Closed cyb3rgh05t closed 1 month ago

cyb3rgh05t commented 1 month ago

Hi,

will it be possible to add an option to delete all dublicated xmltv id's at once in Channel Configurator?

K-vanc commented 1 month ago

Hi,

I dont think so. There is no way to know without looking 1 by 1 about what you want to do. Xmltv_id is just an indicator for channel in epg file. 1 channel from France and another channel from Mexico may have same xmltv_id. If you add both of them to same configuration, then you should know why and which one will be renamed. Tempest is showing you those 2 as duplicate. Which one do you expect from Tempest to auto-delete? France or Mexico? What will be the criteria?

In general no need to be that lazy, i believe 😃

K-vanc commented 1 month ago

If you are asking this for duplicate xmltv_ids which belong to same siteconfig, then on channel creation part, you can enable duplicate channel name delete option so same channel names(which are xmltv_ids) will be deleted on created channel list

cyb3rgh05t commented 1 month ago

If you are asking this for duplicate xmltv_ids which belong to same siteconfig, then on channel creation part, you can enable duplicate channel name delete option so same channel names(which are xmltv_ids) will be deleted on created channel list

yes that schould be it but i cant find that option?

cyb3rgh05t commented 1 month ago

image if you mean this than ok but it stucks at image

K-vanc commented 1 month ago

It is not stucking.. not every channel creation happens in a blink. Some will take minutes.......

cyb3rgh05t commented 1 month ago

ok

cyb3rgh05t commented 1 month ago

using docker i use this command to create the new list but it wont load.

docker exec -u root tempest php /var/www/html/tempest.php chengine=Generate siteconfig=[ENC][EX]tv.blue.ch_0 rm_dupe_id=on

K-vanc commented 1 month ago

Sorry my friend but i am having really difficulties to help you since you are continuesly refusing to say what is your tempest version, which siteconfig you are trying and similar totally useless information so all i can tell you, try again, maybe works..

K-vanc commented 1 month ago

If you updated 1.7, i wrote everywhere since last week as command syntax changed

cyb3rgh05t commented 1 month ago

ok sorry mate i am on latest version. ok i will recheck syntaxes. sorry didnt wanted to make you angry

K-vanc commented 1 month ago

When you dont tell me the minimum needed info, i am wasting my time to check for many things.

Just look below post, also TempestWIKI.pdf updated for new syntax usage;

https://github.com/K-vanc/Tempest-EPG-Generator/issues/17#issuecomment-2089903182

cyb3rgh05t commented 1 month ago

When you dont tell me the minimum needed info, i am wasting my time to check for many things.

Just look below post, also TempestWIKI.pdf updated for new syntax usage;

#17 (comment)

yes understood, i checked new syntaxes and now it works as expected. only thing is like for example if i use tv.blue.ch siteconfig and want to add it to a new config.xml and then start epg grabber it doesnt find any shows. is this due that the site.config doesnt work anymore or is it due to something else? i am using latest tempest version 1.7 in docker mode

Tempest Version : rv1.7.0.0-2405061442 System Info : Linux 23cad1d70090 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 x86_64 PHP Version : 8.2.12

K-vanc commented 1 month ago

Again, same thing..

which channel is not working ???

K-vanc commented 1 month ago

Yes, siteconfig issue. I will check it tomorrow

cyb3rgh05t commented 1 month ago

Again, same thing..

which channel is not working ???

all of them

Yes, siteconfig issue. I will check it tomorrow

ok thankyou very much

K-vanc commented 1 month ago

Hi, siteconfig is updated. There were some changes in data structure. Also I added program categories

cyb3rgh05t commented 1 month ago

Hi, siteconfig is updated. There were some changes in data structure. Also I added program categories

great thank you very much.

Yesterday i also made sponsoring for your project

K-vanc commented 1 month ago

I saw it, thanks for your support

cyb3rgh05t commented 1 month ago

what could be the issue if i create in Tempest Configurator a new config called switzerland adding all channls form tv.blue.ch siteconfig and after save&exit - the switzerland config isnt created?

Tempest Version : rv1.7.0.0-2405061442 System Info : Linux 23cad1d70090 5.15.0-105-generic Ubuntu PHP Version : 8.2.12

K-vanc commented 1 month ago
K-vanc commented 1 month ago

also

cyb3rgh05t commented 1 month ago

hmm i think i added all folders manually but i am not sure.

this is my docker-compose.yml

version: "3.9"
services:
  gluetun-epg:
    #hostname: "gluetun-epg"
    container_name: "gluetun-epg"
    image: "qmcgaw/gluetun:latest"
    cap_add:
      - NET_ADMIN
    devices:
      - "/dev/net/tun:/dev/net/tun"
    restart: "unless-stopped"
    networks:
    - proxy
    ports:
      - "8095:8095/tcp"
    environment:
      - "PUID=1000"
      - "PGID=1000"
      - "TZ=Europe/Berlin"
      - "UMASK=022"
      - "VPN_SERVICE_PROVIDER=mullvad"
      - "VPN_TYPE=wireguard"
      - "WIREGUARD_PRIVATE_KEY=sB4G/VV1/ugDQa3Gwmw7xxxxxxxx="
      - "WIREGUARD_ADDRESSES=10.69.102.240/32"
      - "SERVER_CITIES=Frankfurt"
      - "OWNED_ONLY=yes"
      - "HTTPPROXY=on"
      - "HTTPPROXY_LISTENING_ADDRESS=:8888"
    volumes:
      - "/opt/appdata/gluetun-epg/:/gluetun:rw"

  tempest:
    #hostname: "tempest"
    container_name: "tempest"
    environment:
      - "PGID=1000"
      - "PUID=1000"
      - "TZ=Europe/Berlin"   
      - "UMASK=022" 
    image: "kvanc/tempest_epg"
    restart: "unless-stopped"
    network_mode: "service:gluetun-epg"
    volumes:
      - "/opt/appdata/tempest/:/var/www/html/tempest_config/:rw"
      - "/opt/appdata/tempest/php-user.ini:/etc/php82/conf.d/custom.ini"
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=proxy"
      - "dockupdater.enable=true"
      - "traefik.http.routers.-rtr.entrypoints=https"
      - "traefik.http.routers.tempest-rtr.rule=Host(`tempest.mystreamnet.club`)"
      - "traefik.http.routers.tempest-rtr.tls=true"
      - "traefik.http.routers.tempest-rtr.tls.certresolver=dns-cloudflare"
      - "traefik.http.routers.tempest-rtr.middlewares=chain-authelia@file"
      - "traefik.http.routers.tempest-rtr.service=tempest-svc"
      - "traefik.http.services.tempest-svc.loadbalancer.server.port=8095"
networks:
  proxy:
    driver: bridge
    external: true
cyb3rgh05t commented 1 month ago

i had to go over vpn because sky magenta etc blocked my ips.

cyb3rgh05t commented 1 month ago

i am using Chrome Browser but i didnt checked it in another browser.

K-vanc commented 1 month ago

hmm this docker compose thing. Probably you have permission issue as you remember you created folders manually which you shouldn't. Tempest docker image using nobody as user and group. What you set as PGID and PUID are belongs to your system user. "nobody" user has no permission to write over something belongs to system. Try to sudo chown your mount folder with nobody;

sudo chown nobody:nobody -R /opt/appdata/tempest/

some system using nogroup as group name so if above not works, use below;

sudo chown nobody:nogroup -R /opt/appdata/tempest/

cyb3rgh05t commented 1 month ago

alright i will test it thanks

cyb3rgh05t commented 1 month ago

ok so i reinstalled tempest from scratch again with my docker-compose but i defenitly created the folders myself my tempest root folder is empty after installation. what could be that issue?

cyb3rgh05t commented 1 month ago

sudo chown nobody:nobody -R /opt/appdata/tempest/

nope that doesnt work eighter it doesnt save anything to configs

K-vanc commented 1 month ago

delete container again, also the folder you created as "/opt/appdata/tempest/", and remove PGID and PUID lines from your composer file under tempest: , then;

sudo mkdir /opt/appdata/tempest/ sudo chown nobody:nobody -R /opt/appdata/tempest/

now folder created with correct ownership.

use your edited composer file again to build container. When finished run the container and call tempest for folder creation;

docker exec -u root yourcontainername php /var/www/html/tempest.php

Now, folders shall be generated inside your /opt/appdata/tempest/ path so you can update

docker exec -u root yourcontainername php /var/www/html/tempest.php --update

cyb3rgh05t commented 1 month ago

i have to make sudo chown nobody:nogroup -R /opt/appdata/tempest/ becouse nobody doesnt work

so this worked?

cyb3rgh05t commented 1 month ago

after creation of the container and called tempest docker exec -u root tempest php /var/www/html/tempest.php still nothing inside the folder

K-vanc commented 1 month ago

but did you create the container with modified file? if you used again below;

tempest:

hostname: "tempest"

container_name: "tempest"
environment:
  - "PGID=1000"
  - "PUID=1000"

It will not work. Also check permissions and set folder to 755

sudo chmod 755 -R /opt/appdata/tempest/

cyb3rgh05t commented 1 month ago

yes i changed the composer file, removed the PGID PUID.

gluetun-epg
tempest
root@plexBeast ~ # docker rm gluetun-epg tempest
gluetun-epg
tempest
root@plexBeast ~ # mkdir /opt/appdata/tempest
root@plexBeast ~ # sudo chown nobody:nobody -R /opt/appdata/tempest/
chown: invalid group: ‘nobody:nobody’
root@plexBeast ~ # sudo chown nobody:nogroup -R /opt/appdata/tempest/
root@plexBeast ~ # docker-compose -f gluetun-epg.yml up -d
Creating gluetun-epg ... done
Creating tempest     ... done
root@plexBeast ~ # docker exec -u root tempest php /var/www/html/tempest.php
root@plexBeast ~ # docker exec -u root tempest php /var/www/html/tempest.php
root@plexBeast ~ # docker exec -u root tempest php /var/www/html/tempest.php --update
[                   ]
[                   ]-------------------------------------
[                   ]        Tempest EPG Generator
[                   ]
[                   ]   The Most Advanced Programmable
[                   ]       GUI XMLTV EPG Generator
[                   ]     by Kivanc Altug aka "Kvanc"
[                   ]-------------------------------------
[                   ]
[09-05-2024 14:37:44] Tempest Version : rv1.6.0.0-2310141014
[09-05-2024 14:37:44] System Info : Linux d0bc96f9727a 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 x86_64
[09-05-2024 14:37:44] PHP Version : 8.2.12
[09-05-2024 14:37:44] Server Info : Command Line Interface (Cli)
[09-05-2024 14:37:44] Tempest Updater enabled...
[                   ]
[09-05-2024 14:37:44] Your Tempest version is rv1.6.0.0-2310141014
[09-05-2024 14:37:44] Checking for updates...
[09-05-2024 14:37:44] New Tempest version found as rv1.7.0.0-2405061442
[09-05-2024 14:37:44] Updating...
[09-05-2024 14:37:45] Update completed...
K-vanc commented 1 month ago

so update is working. Mount should also work

cyb3rgh05t commented 1 month ago

yes update worked but not the mount.

root@plexBeast ~ # sudo chmod 755 -R /opt/appdata/tempest/
root@plexBeast ~ # docker exec -u root tempest php /var/www/html/tempest.php
root@plexBeast ~ # cd /opt/appdata/tempest
root@plexBeast /opt/appdata/tempest # ls -al
total 8
drwxr-xr-x  2 nobody nogroup 4096 May  9 13:31 .
drwxr-xr-x 47   1000    1000 4096 May  9 13:31 ..
root@plexBeast /opt/appdata/tempest #
K-vanc commented 1 month ago

sorry it is byte

K-vanc commented 1 month ago

may I ask why you set this in composer?

image

cyb3rgh05t commented 1 month ago

ha wait..

cyb3rgh05t commented 1 month ago

sorry that was my bad :(

total 36
drwxrwxrwx  7 nobody nogroup 4096 May  9 13:56 .
drwxr-xr-x 47   1000    1000 4096 May  9 13:54 ..
drwxrwxrwx  2 nobody nogroup 4096 May  9 13:56 cookies
drwxr-xr-x  2 nobody nogroup 4096 May  9 13:56 epg
drwxr-xr-x  2 nobody nogroup 4096 May  9 13:56 log
-rwxr-xr-x  1 nobody nogroup   63 Apr 24 18:49 php-user.ini
drwxrwxrwx  2 nobody nogroup 4096 May  9 13:56 site_config
drwxrwxrwx  2 nobody nogroup 4096 May  9 13:56 temp
-rwxrwxrwx  1 nobody nogroup  393 May  9 13:56 tempest.config.xml
root@plexBeast /opt/appdata/tempest #
cyb3rgh05t commented 1 month ago

i think there is still an issue with the tv.blue.ch siteconfig. if i add all channels to my config nothing is saved but if i only take selected channels than it works. or is this due to something else?

K-vanc commented 1 month ago

Probably there is a duplicate in the list(there are many on that siteconfig channel list) and your browser pop-ups, or javascript or both disabled. There is no difference on sending all or selecting some

K-vanc commented 1 month ago

I tested on chrome by sending all. Found many duplicete, after changing all, it saved the list

cyb3rgh05t commented 1 month ago

you say "after changing all" what do you change at a dublicated channel?

cyb3rgh05t commented 1 month ago

i made a channel generator to remove all dublicated ids but it didnt saved it after to my config and no pop came up to say that there some dublicates. after channel generator it reduced the list to 1072 channels

K-vanc commented 1 month ago

Xmltv_id of duplicate channels. In default list, Tempest is finding +20 dup xmltv_id. Don't you see all red/black highlighted boxes??

This page is pure html and javascript rendered by browser. If something is not working for you, it means it is your browser (mostly settings such disabled javascript etc.)

K-vanc commented 1 month ago

It will look like this;

https://user-images.githubusercontent.com/97025515/148506365-d299b2dd-1939-4947-b18d-2c4b1fb99061.jpg

cyb3rgh05t commented 1 month ago

It will look like this;

https://user-images.githubusercontent.com/97025515/148506365-d299b2dd-1939-4947-b18d-2c4b1fb99061.jpg

yes i see what you mean. in this redboxes can i just change the id? eg your example "xxx HD" to "xxxx HD (backup)"?

K-vanc commented 1 month ago

You can set there whatever you want or just add an extra Letter/Number etc. Important thing is those xmltv_ids shall be unique for each channel otherwise your generated epg file will not be readable for players. That photo is example. When they are unique, they will be blue/black again. When all list is blue/black, you are good to go saving

K-vanc commented 1 month ago

But dont forget that some players are using it for channel mapping so better using something identifable. Like if you have 2 Rtl for Belgium and Germany, use something like rtl.be and rtl.de so you will be able to understand which is which

cyb3rgh05t commented 1 month ago

alright thanks. i made for every country i need one config