search

E2OpenPlugins / e2openplugin-OpenWebif

GPL Web Interface for e2stabs
GNU General Public License v3.0
205 stars 226 forks source link

Openwebif picon #1202

Closed bobika99 closed 3 years ago

bobika99 commented 3 years ago

Hello, my VU+ Zero 4K not displayed picon in Openwebif. They are displayed correctly in the receiver and on my second receiver VU+ Zero they also display fine on the Openweif. Picon saved on NAS in dir \picon. OpenAtv 6.4. I described it here: https://www.opena.tv/english-section/54561-openwebif-picon-post460369.html#post460369

Thank you for the advice.

jbleyel commented 3 years ago

What's the correct path of the picon folder on the 1. and 2. receiver. The path must be different.

bobika99 commented 3 years ago

The path is the same for both receivers on NAS http://192.168.88.114/picon/. network4k

jbleyel commented 3 years ago

http://192.168.88.114/picon/ is not a valid picon path. The valid picon paths are:

    /media/cf/picon
    /media/mmc/picon
    /media/usb/picon
    /media/hdd/picon
    /usr/share/enigma2/picon
    /picon

You need a symbolic link to the picon folder on your nas.

bobika99 commented 3 years ago

Sorry, this path is /media/hdd/picon picon

jbleyel commented 3 years ago

/media/hdd/picon is a valid path and should work.

Please provide the output of "ls -l /media/hdd/picon" via telnet/ssh from both receivers. Not the complete output, i need only a few lines!

bobika99 commented 3 years ago

Both receivers have a directory available:

Vu+ Zero: List_Zero

Vu+ Zero 4K: List_Zero4K

jbleyel commented 3 years ago

Do you have one of these folders?

/media/cf/picon /media/mmc/picon /media/usb/picon

/media/cf/owipicon /media/mmc/owipicon /media/usb/owipicon

jbleyel commented 3 years ago

I have checked the code again. Error 404 for http://IP/picon means the global var PICON_PATH is None. This can only happen here -> https://github.com/E2OpenPlugins/e2openplugin-OpenWebif/blob/master/plugin/controllers/defaults.py#L59

I don't know why. Maybe there is a crash.

I need the enigma debug log from the box start.

bobika99 commented 3 years ago

This is dir /media/... on Zero 4K. DIR on the Zero receiver are the same media

This is log after enable log and restart receiver and reload OpenWefif: Enigma2-debug-20201231_16-57-45.log

jbleyel commented 3 years ago

I have checked the log. This log line is missing. "Current Picon Path : ..."

Please copy this file https://gist.github.com/jbleyel/0ced97ae586e109174783462e0dc8093 to -> /usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/

Then send the enigma log again.

bobika99 commented 3 years ago

Enigma2-debug-20210102_09-34-02.log

jbleyel commented 3 years ago

Please open telnet or ssh.

Then call python -c "import os;print os.path.isdir('/media/hdd/')" and python -c "import os;print os.path.isdir('/media/hdd/picon/')"

jbleyel commented 3 years ago

And call 

cd /media/hdd
ls -l
bobika99 commented 3 years ago

putty4K

jbleyel commented 3 years ago

Very strange.

The result of this: python -c "import os;print os.path.isdir('/media/hdd/picon/')" is True and this is correct.

BUT the same function in OWF says False.

The log of line 88 shows: DEBUG Current Picon Path current : /media/hdd/picon/ -> https://gist.github.com/jbleyel/0ced97ae586e109174783462e0dc8093#file-defaults-py-L88 and the next line is: if os.path.isdir(current): -> https://gist.github.com/jbleyel/0ced97ae586e109174783462e0dc8093#file-defaults-py-L89 The result is False and this is wrong.

I have no idea why.

s3n0 commented 3 years ago

@bobika99

Try this:

  1. Download the following file to your PC: (file "defaults.txt" was removed - the corrected file can be found below, in the next post)
  2. Rename the file extension from .txt to .py (due to attachments restriction here, unfortunately!)
  3. Copy and overwrite the file to -> /usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/
  4. Reboot the set-top box
  5. Try to start OFW
  6. Send the debug-log again

I tried to edit one line, which creates a path to the directory directly using os.path.join() and then its existence is tested in the if condition.

I also added a lengthy but effective directory listing to the debug-log records (to check the existence of the "picon" subdirectory).

The os.path.join() method is no longer recognized and unnecessary in Python, but this method actually just merges multiple strings into one and checks for slashes between directories. At the end of the path, however, there is never a slash, and it should not be correct on Linux systems. This probably won't help ... but I would be personally interested in listing the contents of directories, which I also added to the debugging source code (only temporarily).

bobika99 commented 3 years ago

Enigma2-debug-20210102_21-58-12.log OpenWebif failed to load...

s3n0 commented 3 years ago

Sorry, my fault... here is it: defaults.txt

jbleyel commented 3 years ago

But why only on the Zero 4K and not on the Zero. Same OWF and same Image version.

bobika99 commented 3 years ago

Sorry, my fault... here is it: defaults.txt

Enigma2-debug-20210102_22-16-45.log

jbleyel commented 3 years ago

It looks like that the hdd is not mounted at the time of starting of the OWF.

bobika99 commented 3 years ago

It looks like that the hdd is not mounted at the time of starting of the OWF.

But hdd in NAS runs nonstop without hibernation...

s3n0 commented 3 years ago

@jbleyel

This defaults.py algorithm is supposed to run only once, by importing it - into the plugin.py source code - i.e. during Enigma2 startup. Right ?

The /media/hdd directory is really empty - when the defaults.py algorithm tries to find some available directories (for picons).

@bobika99

Apparently this is an external HDD that is not mounted in the operating system on this Vu+ Zero 4K device at the moment when Enigma2 / OpenATV 6.4 is started (during Enigma2 startup). If the HDD is mounted via Python (Enigma2 algorithm), this problem is very likely.

However, if an external HDD is mounted over a Linux system, then it should be already pre-mounted (before start the Enigma) and then the problem should no longer occur.

jbleyel commented 3 years ago

We can do this: Do not call this PICON_PATH = getPiconPath() at load time of the plugin. Do call this at the fist usage of PICON_PATH.

I will try to do this.

jbleyel commented 3 years ago

The first usage of PICON_PATH is in RootController init. There is no easy way to change this.

s3n0 commented 3 years ago

That probably won't help. If a user sets the external HDD to sleep on idle, then when you open OpenWebif, you will wait again for 5 seconds, each time the external network drive spins up and picos are read from it.

I think this is some issue in Enigma. After all, external network drives must be mounted before Enigma can start.

It also depends on the setting options in the specific Enigma, for the new mount point. There must to be a possiblity in the mount point configuration, also a choice, to use the operating system (Linux) or use the Python / Enigma2 algorithm - to mount an external network drive.

jbleyel commented 3 years ago

That's why i put my picons on a USB stick.

s3n0 commented 3 years ago

Me too. I always used a USB flash drive ("/media/hdd" or "/media/usb") or a Micro-SD card ("/media/mmc").

However, since I use synchronized picons through my own plugin to update Chocholousek's picons, I save space on the internal flash drive - so now I keep the picons in the internal flash drive again. :) There are fewer picons and I save space in the internal flash drive.

s3n0 commented 3 years ago

I looked again at the LOG file from your Enigma and it is clear that it will mount the network drive (look for the IP address of your NAS or the directory you want to mount, for example) only after OpenWebif tries to find the path to the picos. This also means that Enigma / Python takes care of mounting the disk and not the Linux system itself. This is obviously bad from the point of view of OpenWebif.

In some Enigma distributions, it is possible to switch the method of mounting a network drive in the "mount-points manager". So, you set there whether the network drive will be mounted via the Linux system (when the Linux system boots) or whether the network drive will be mounted via Enigma / Python (when Enigma starts). I don't use network disk mounting via GUI / Enigma, so I can't say how it is with OpenATV.

So, if possible, switch the disk mount process to a Linux system, instead of the Enigma / Python mount process.

Enigma2-debug-20210102_22-16-45.log :

Searching for a directory with picons in the OpenWebif plugin algorithm :

22:16:58.5892 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:16:58+0100 [-] DEBUG Current Picon Path prefix : /media/hdd/
22:16:58.5898 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:16:58+0100 [-] DEBUG Current Picon Path prefix isfolder : /media/hdd/
22:16:58.5903 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:16:58+0100 [-] DEBUG Current Picon Path prefix listdir : []

Mounting a network drive (Enigma2 / OpenATV 6.4 algorithm) :

22:17:05.0085 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:05+0100 [-] [Console] finished: mount //192.168.88.114/vuzero
22:17:05.0093 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:05+0100 [-] [eBatch] retval=0, cmds left=0, data:

The difference is about 4-5 seconds, between searching the OpenWebif directory with picons and Enigma2 mounting the network drive :). Unfortunately to the detriment of OpenWebif.

However, it is still a mystery that this problem does not manifest itself in the case of VuZero. The problem only occurs with VuZero4K.

According to debug-log, it looks like Enigma is basically "remounting" the /media/hdd directory to an external HDD ... and there is even a waiting time of 2 secs: (follow mainly the time sequence of events - i.e. the time trace at the beginning of the debug log lines)

...
22:16:46.9015 {   } /usr/lib/enigma2/python/Components/Harddisk.py:871 enumerateNetworkMounts new Network Mount being used as HDD replacement -> /media/hdd/
...
22:16:55.6586 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:16:55+0100 [-] command ['umount -fl /media/hdd', 'sleep 2', 'mount //192.168.88.114/vuzero']
22:16:55.6593 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:16:55+0100 [-] [Console] command: umount -fl /media/hdd
...
22:17:02.2114 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:02+0100 [-] [Console] finished: umount -fl /media/hdd
22:17:02.2120 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:02+0100 [-] [eBatch] retval=0, cmds left=2, data:
22:17:02.2124 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:02+0100 [-] 
22:17:02.2130 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:02+0100 [-] [Console] command: sleep 2
...
22:17:04.2307 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:04+0100 [-] [Console] finished: sleep 2
22:17:04.2313 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:04+0100 [-] [eBatch] retval=0, cmds left=1, data:
22:17:04.2318 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:04+0100 [-] 
22:17:04.2323 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:04+0100 [-] [Console] command: mount //192.168.88.114/vuzero
...
22:17:05.0085 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:05+0100 [-] [Console] finished: mount //192.168.88.114/vuzero
22:17:05.0093 {   } /usr/lib/python2.7/site-packages/twisted/python/util.py:815 untilConcludes 2021-01-02 22:17:05+0100 [-] [eBatch] retval=0, cmds left=0, data:
...

The strange thing is that the algorithm in Enigma for rendering picons, managed to find the directory correctly:

22:16:50.8019 {   } /usr/lib/enigma2/python/Components/Renderer/Picon.py:30 onMountpointAdded [Picon] adding path: /media/hdd/picon/

Maybe this module /Components/Renderer/Picon.py could be used in OpenWebif to find out the picon directory, but I'm not sure if it is used in all Enigma distributions to keep OpenWebif compatible with all Enigma distributions.

jbleyel commented 3 years ago

Hi @s3n0 , we can try to implement the onPartitionChange event like Picon.py in OWF.

s3n0 commented 3 years ago

Hi.

Yes, that sounds good.

But in case of better compatibility, it would probably be appropriate to copy or take inspiration from the algorithm (the method of searching the directory for picons) according to the source code in Picon.py file: OpenATV: https://github.com/openatv/enigma2/blob/6.4/lib/python/Components/Renderer/Picon.py OpenPLi: https://github.com/OpenPLi/enigma2/blob/develop/lib/python/Components/Renderer/Picon.py Disconnecting and connecting external disks (mount points) is already handled in this source code.

Bobika99 sent me a debug-log file Enigma2-debug-20210103_21-42-04.log from working VuZero device. The difference compared to the problematic VuZero4K is really only in a few seconds :). And of course again the Picon.py algorithm, even in this case, was able to find the picon directory correctly.

The VuZero device is "lucky" in that there is a delayed process of re-mounting the external network drive for a few seconds.

I think the main role there plays this code:

from Components.Harddisk import harddiskmanager
.......
harddiskmanager.on_partition_list_change.append(onPartitionChange)

This is important to control the disconnection and connection of the external drive. Each time you disconnect or reconnect in Enigma (via the Python algorithm and not via the OS), the directory search with the icons starts.

However, the question is whether this module Components.Harddisk is also included in all Enigma distributions (due to OpenWebif compatibility for all Enigmas). Unfortunately, I don't know if this module is in every Enigma. If not, then it is necessary to implement this "hard disk manager" into OpenWebif directly. And it will be more complicated, I think :).

The second way could be to add a time cycle - for example lasting 5 seconds, ie. that the whole picon directory recognition algorithm in OpenWebif would run twice in a row with a time interval of 5 seconds (with the comment that some Enigma distributions perform re-mounting of external disks at startup and therefore need to be repeated 5 seconds later, retry scan picon directories). If this algorithm is run only once - when initializing the OpenWebif plugin, then it should not interfere. However, if you run this code to recognize picon directories over and over again - then this is the wrong way.

jbleyel commented 3 years ago

This is not so easy. We can use Picon.py to find single picons. But we cannot use this for the twisted folder link. -> https://github.com/E2OpenPlugins/e2openplugin-OpenWebif/blob/master/plugin/controllers/root.py#L74

The twisted link needs a path and this is lastPiconPath of Picon.py. The lastPiconPath is None and will be set at the fist usage of findPicon.

This means we need to call these 2 lines on every change of lastPiconPath:```

    if PICON_PATH:
        self.putChild2("picon", static.File(six.ensure_binary(PICON_PATH)))
s3n0 commented 3 years ago

I did not study this source code in depth at all. Neither the Picon.py source code (in Enigma) nor the defaults.py source code (in OpenWebif).

So I can't comment on what would be best to do. It must be done by someone who knows the OpenWebif code very well, but I'm not. It would take me days or weeks to study and test the OpenWebif code. Unfortunately, I am a lazy person :).

wedebe commented 3 years ago

I plan to (eventually) cache picons in the browser to reduce network and disk requests on the enigma2 box.

jbleyel commented 3 years ago

Cached picons sounds good but will not help here. We need to catch onPartitionChange and call getPiconPath() again.

jbleyel commented 3 years ago

Please get the lastest version and test again.