No data in web ui

[kruska7]

Describe the bug No data shows in web ui after source is selected. Log shows playlist_processor processed both groups and channels, with no errors.

Expected behavior Tree view of playlist should show in ui.

Screenshots

Desktop (please complete the following information):

• OS: macOS 14.5
• Browser: Chrome
• Version: 126.0.6478.127

Additional context I have the most basic setup, no fluff added, but nothing shows in ui. I can update the playlist just fine in settings section of web ui though.

Please tell me what info you need from me to fix this. Would love to get this app working as it should :)

Log:

@:~/m3u-filter$ docker compose up [+] Running 2/2 ✔ Network mf_default Created 0.1s ✔ Container m3u-filter Created 0.1s Attaching to m3u-filter m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter] Log Level INFO m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter] Version: 2.0.2 m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter] Current time: 2024-07-06 10:36:01 m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter] Working dir: "/data" m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter] Config dir: "/config" m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter] Config file: /config/config.yml m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter] Source file: /config/source.yml m3u-filter | [2024-07-06T08:36:01Z WARN m3u_filter::utils::config_reader] cant read mapping file: mapping.yml m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter::utils::config_reader] Mappings File: mapping.yml m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter::utils::config_reader] Api Proxy File: /config/api-proxy.yml m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter] Server running: http://0.0.0.0:8901 m3u-filter | [2024-07-06T08:36:01Z INFO m3u_filter::api::main_api] Web root: "/web" m3u-filter | [2024-07-06T08:36:01Z INFO actix_server::builder] starting 8 workers m3u-filter | [2024-07-06T08:36:01Z INFO actix_server::server] Actix runtime found; starting in Actix runtime m3u-filter | [2024-07-06T08:36:05Z INFO m3u_filter::processing::playlist_processor] {"stats": {"name": http://***, "type": xtream, "errors": 0, "raw": {"groups": 377, "channels": 21284}, "processed": {"groups": 7, "channels": 473}}} m3u-filter | [2024-07-06T08:36:06Z INFO actix_web::middleware::logger] 192.168.1.58 "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" 0.000539 m3u-filter | [2024-07-06T08:36:06Z INFO actix_web::middleware::logger] 192.168.1.58 "GET /static/js/main.18a56688.js HTTP/1.1" 304 0 "http://192.168.1.99:8901/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" 0.000301 m3u-filter | [2024-07-06T08:36:06Z INFO actix_web::middleware::logger] 192.168.1.58 "POST /auth/token HTTP/1.1" 200 22 "http://192.168.1.99:8901/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" 0.000368 m3u-filter | [2024-07-06T08:36:06Z INFO actix_web::middleware::logger] 192.168.1.58 "GET /api/v1/file/download/info HTTP/1.1" 200 33 "http://192.168.1.99:8901/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" 0.000118 m3u-filter | [2024-07-06T08:36:06Z INFO actix_web::middleware::logger] 192.168.1.58 "GET /api/v1/config HTTP/1.1" 200 1179 "http://192.168.1.99:8901/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" 0.007383 m3u-filter | [2024-07-06T08:36:28Z INFO actix_web::middleware::logger] 192.168.1.58 "POST /api/v1/playlist/update HTTP/1.1" 200 0 "http://192.168.1.99:8901/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" 0.000239

[euzu]

@kruska7 hi, i think there is a misunderstanding. The webui tree view is not displaying any playlist from the configuration. It is to display a raw list of given sourrce to see what content tge provider delivers to define your rules. It is only a helper to see the original content from provider.

The processed entries are only accessible through the api. If you have xtream output then the xtream api, if you have m3u output than the m3u api should be used to access the processed playlists.

The user credentials for each output can be defined in the ui.

[kruska7]

I understand that, and I was expecting the raw list of content from the provider to show up in the web ui. But after selecting the source, nothing shows up, the view where the tree view should show is completely empty.

So disregard that I mentioned playlist_processor. The issue is that nothing from the provider shows up in the tree view.

[euzu]

@kruska7 if you have a xtream source you can only use the right drop down list which shows you the configured sources. You can only write a m3u playlist url in the edit field. Any xtream source can only be used through the dropdown list from the right.

[kruska7]

It is a xtream source, but as you see on my screenshot I've already selected the source and still no data shows up below where the tree view should be presented.

[euzu]

I can see a http link on your screenshot, i mean the source in my Screenshot

[jamesarbrown]

Hi, I dont know if the same, but I am also unable to get the list.

You can choose the input provider, but I get a "Fail to Download PlayList" when selecting the download button.

The logs are showing a 401 Auth error, I believe the UI is failing to try and auth properly on Xtream input.

Note The m3u file is being created ok via Xtream in the background in /data. So the Auth creds are right.

[euzu]

@jamesarbrown i dont know why it fails, but the ui sends the download request to the backend , the backend uses the same function to download the list and not something else. There must be another reason which i dont understand. I dont have this problem. And i cant figure it out only looking at the source code. If you can debug the applicaiton it would be helpful . You can use Intellij Rover.

[jamesarbrown]

Suspect then its something to do with that i have it running in docker from a release, rather than source on baremetal.

If i can not resolve will open a new issue, but for now wont be able to setup debugging environment. Will try to increase log levels

[euzu]

@jamesarbrown i am running it in docker too, and it is working.

[kruska7]

@euzu Yes, I've selected my http xtream source from the list (it is the only source I have). Am I suppose to press the download button after I've selected the source in order for the tree view to be displayed? I tried that, but I get the error

ERROR m3u_filter::utils::request_utils] cant download input url: http://xxx => Request failed with status 401 Unauthorized

[jamesarbrown]

Same problem as mine then.

I increased log levels and the 401 goes missing, but here are two versions. I noted that the request headeras are not present in the failing version, but not sure if that has anything to do with the issue

Background Scrape (Works)

[2024-07-07T08:24:32Z DEBUG reqwest::connect] starting new connection: http://REDACTED/
[2024-07-07T08:24:32Z DEBUG hyper_util::client::legacy::connect::dns] resolving host="REDACTED"
[2024-07-07T08:24:32Z DEBUG hyper_util::client::legacy::connect::http] connecting to REDACTED:80
[2024-07-07T08:24:32Z DEBUG hyper_util::client::legacy::connect::http] connected to REDACTED:80
[2024-07-07T08:24:32Z DEBUG m3u_filter::utils::request_utils] downloading json content from http://REDACTED:80/player_api.php?username=XYZ&password=XYZ&action=get_series
[2024-07-07T08:24:32Z DEBUG m3u_filter::utils::request_utils] Request headers {"accept-encoding": "gzip", "accept": "application/json, application/xml", "user-agent": "Mozilla/5.0 (Linux; Tizen 2.3) AppleWebKit/538.1 (KHTML, like Gecko)Version/2.3 TV Safari/538.1"}

UI Scrape by Clicking Cloud Download (Fails)

[2024-07-07T08:30:00Z DEBUG m3u_filter::utils::request_utils] getting input text content working_dir: /data, url: http://REDACTED:80
[2024-07-07T08:30:00Z DEBUG m3u_filter::utils::request_utils] Request headers {}
[2024-07-07T08:30:00Z DEBUG reqwest::connect] starting new connection: http://REDACTED/
[2024-07-07T08:30:00Z DEBUG hyper_util::client::legacy::connect::dns] resolving host="REDACTED"
[2024-07-07T08:30:00Z DEBUG hyper_util::client::legacy::connect::http] connecting to REDACTED:80
[2024-07-07T08:30:00Z DEBUG hyper_util::client::legacy::connect::http] connected to REDACTED:80
[2024-07-07T08:30:00Z INFO  actix_web::middleware::logger] 192.168.1.241 "POST /api/v1/playlist HTTP/1.1" 400 12 "http://10.8.5.3:8901/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:127.0) Gecko/20100101 Firefox/127.0" 0.153284

[euzu]

@jamesarbrown @kruska7 i will check the request headers , it could be that your provider block browser.

[euzu]

@jamesarbrown @kruska7 to send Request-Header you need to define them for the input. As an example:

sources:
- inputs:
  - name: simple_xt
    type: xtream
    username: test
    password: secret
    url: http://iptv-test.tv
    epg_url: http://iptv-test.tv/xmltv.php?username=test&password=secret
    headers:
       User-Agent: "Mozilla/5.0 (AppleTV; U; CPU OS 14_2 like Mac OS X; en-us) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.1 Safari/605.1.15"
       Accept: application/json
       Accept-Encoding: gzip
  targets:
  - name: simplext
    output:
      - type: xtream
     ...
     ...

[jamesarbrown]

If you look at https://github.com/euzu/m3u-filter/issues/65#issuecomment-2212371345, it was enabled (as you can see in the logging), but in UI its not using it

Both logs are from same container with no changes in between

[euzu]

@jamesarbrown as you can see in the wireshark log the headers are sent to the provider, i checked it twice:

I have used the download button from the UI, the headers are only send when you select a defined input source from the right dropdown menu

[jamesarbrown]

Sorry, dont know, its a docker container, I cant change the code, so something is different. Its 3 providers, 2 for me tested, 1 for @kruska7

All I was looking at was the logging Request headers {"accept-encoding... vs Request headers {}

And thought that may help point to an issue.

If I get time, I will try and hook up to the ethernet and also run wireshark so I can see NAS traffic.

[euzu]

@jamesarbrown i am running the m3u filter inside a docker container on a proxmox vm, and it is working without any problems.

[euzu]

@jamesbrown there is soemthing odd in your log at this line :

[2024-07-07T08:30:00Z INFO  actix_web::middleware::logger] 192.168.1.241 "POST /api/v1/playlist HTTP/1.1" 400 12 "http://10.8.5.3:8901/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:127.0) Gecko/20100101 Firefox/127.0" 0.153284

Your browser sends the request to http://10.8.5.3:8901/api/v1/playlist and gets a 400 Bad Request. Can you open the Developer Tools for the browser and look at the network tab to see what is the content of the request?

[jamesarbrown]

This?

[euzu]

@jamesarbrown exactly what i wrote, this is not the selected source, this is a http url, and without a input selection you cant send the headers. You need to select the source like in my previous comment:

Click the right arrow, select the desired input, press the download cloud icon

[jamesarbrown]

Ok, let me look at that, I am getting this

[jamesarbrown]

Have to go out now, will look again this evening. This is my config

sources:
  - inputs:
      - enabled: true
        type: xtream
        headers:
          User-Agent: "Mozilla/5.0 (Linux; Tizen 2.3) AppleWebKit/538.1 (KHTML, like Gecko)Version/2
          Accept: application/json, application/xml
          Accept-Encoding: gzip
        url: 'http://xxxx:80'
        username: xxx
        password: xxx

[euzu]

@jamesarbrown ok now i understand, the name property is missing, pleae add name and test it again, yes name is optional, and i should fix this issue whit the ui, please add a name attribute and your problem should be fixed

sources:
- inputs:
  - name: my_xtream  <------ any name 
    type: xtream
    username: test
    password: secret

[euzu]

@kruska7 please add a unique name to each input and your problem should be fixed

[kruska7]

Issue solved by adding a unique name to input as suggested.

[jamesarbrown]

Confirmed also.

BTW I do not think "name" is in the readme.

https://github.com/euzu/m3u-filter#221-inputs

[euzu]

@jamesarbrown i have added name to README nd WIKI, should be updated with the next commit.