Closed obert0 closed 1 month ago
7wells
commented
4 months ago The problem that sometimes replugging is necessary and its reason was discussed over there, if I remember correctly:
https://community.home-assistant.io/t/allow-loading-firmware-files-on-boot/632031/19
dfigus
commented
4 months ago @7wells thanks, I found this discussion as well. So it's seems an underlying HA OS issue home-assistant/operating-system#1157 which does not provide the firmware files during the early boot stage and hence those can't be loaded. After boot those FW files are available the the replugging triggers the whole device initialization again.
Probably can't solve in the add-on, but let's keep the issue open here and link to the HA OS one.
I'll do some further investigation and testing on my end, but there is probably no workaround available.
obert0
commented
4 months ago Thank you @7wells, I will try to create a simlink to the normal firmware folder see if it make it accessible. I think part of the problem is the home assistant files are not accessible at boot too. The firmware is in the share folder of the supervisor. But this is just part of the problem because even with a replugging it did not work. I should unplug, remove the firmware, replug, and copy back the firmware. Then it work well. I think this problem may be caused by tvheadend. I have succeed to repeat the problem with all addon I tried where the dvb adapter show up. I try too automate my workaround to make to start it from HA. I succeed to move the firmware and reset the usb through a named pipe. But I can't make the named pipe listener scipt to start on host boot. If anyone have a idea!
dfigus
commented
4 months ago @obert0 I did some research on the USB adapter reset topic and found https://unix.stackexchange.com/questions/525532/usb-disable-and-reenable-device-on-linux/525533#525533 and netinvent/usb_resetter. Can you try whether this works? You probably can't run this inside the add-on now as it's missing the special privileged authorizations to do this.
obert0
commented
4 months ago Actually this is exactly what I did to reset the usb through named pipe. Actually the command I use is echo 1.1—1 >/sys/bus/drivers/usb/unbind or bind to reverse. This work really well, except I should be in shh on port 22222 to make this work as I cannot access the file either way. If in the addon config "usb: true" is activated I can see the path within the addon but without acces to add the command. I eill try the solution to interact with port, but at think I will find the same issue. And I dont think it is possible to start a shell script on host at boot. The haos host is pretty close and I dont think I will be able to Install something. My last hope is to break a usb extension, cut the power wire and pass it through a relay. I will try to drive the relay with esphome to automate the replugging. I let you know if it works!
dfigus
commented
4 months ago @obert0 the add-on needs elevated permissions to do this, see privileged (or full_access as the last line of defense) here. I guess we would need the SYS_ADMIN privilege here. If you can overwrite the config.yaml locally, you might try this and report back.
I will receive a USB adapter shortly and would try this out as well.
obert0
commented
4 months ago @dfigus, I will try with privileges to replug the tuner. If this work I can start the named pipe listener with the init commands.
By the way I update your addon and the server didn't startup. Both release and edge. I did not investigate alot since I wanted to focus on my tuner problem I switch to my local addon.
dfigus
commented
4 months ago @obert0 I updated to the latest release version earlier today and it started as usual. What I experienced is that the config backup before TVH config migration (triggered by a new TVH "release") could easily take 30+ minutes due to wg+ installed in there and in this timeframe TVH is not fully up. In case it's still happening on your end, can you create a new issue with the logs attached?
obert0
commented
4 months ago Yes I will try again. The weird thing is I start the addon and then try to enter in the container file system with ssh port 22222, but I got a message saying the container isn't open. I will retry and look at the logs.
EDIT: You were right and it was the first start up that take longer than usual. And I made a mistake on the container name so I coudn't get in.... My bad :)
obert0
commented
4 months ago I tried with SYS_ADMIN Privileged and full_acces without succes. I get this message:
root@local-tvheadend-local:/sys/bus/usb/drivers/usb$ echo "1-1.1" > unbind
sh: can't create unbind: Read-only file system
I try the "Advanced SSH & Web Terminal" addon with the same result. The only way to force usb unbind/bind is with ssh on port 22222. The only way I see to succeed without hardware approch is to make a command line switch and push a ssh command on the port 22222. But I think this is security issue. It is not recommanded to let the api key permently in HA.
dfigus
commented
4 months ago You were right and it was the first start up that take longer than usual. And I made a mistake on the container name so I coudn't get in.... My bad :)
You can skip the backup part with command line argument --nobackup, I use it as well. See https://github.com/dfigus/addon-tvheadend/blob/main/tvheadend/DOCS.md#option-args. Furthermore you don't need to ssh to the host (port 22222) but rather can use the advanced ssh addon with disabled protection mode to get into the addon container.
I tried with SYS_ADMIN Privileged and full_acces without succes
Did you disable the protection mode for the addon? With full_access the addon should be able to do nearly anything you can do on the host: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
obert0
commented
4 months ago Yes I disable the protection mode on both advance ssh addon and my local addon. It don't seam to be the same level of permissiom than with ssh on port 22222. I can enter in the container which need high permission.
When I first try to make the firmware load I reach the end of permissions allowed by the advanced ssh addon so I needed to use ssh.
7wells
commented
4 months ago AFAIK, logging in via ssh port 22222 is the only way to get into the "developer" mode (well, they wrote that this is for developers, only, over in the HA forum) with all needed privileges. If there is anything I can help you with (i.e. testing), please let me know. I have a DVB-T2 USB stick here attached to an RPi4 and most of the time, the tvheadend addon both in HA and in Kodi works. Sometimes I have to replug the DVB-T2 USB stick to the RPi4. I am capable of besic Linux stuff but don't know much about programming.
dfigus
commented
4 months ago If you ssh into 22222, you will land in the OS of the host machine (HA OS), otherwise you end up in a docker container with restricted privileges (depending on how the container was started, e.g. can be influenced via the config.yaml). However a privileged container (full_access for the addon) should grant you nearly all permissions of the host. I just received a USB DVB-S2 tuner yesterday and I hope I find some time for testing on the weekend.
dfigus
commented
3 months ago @obert0 There is an alpine package linux-firmware-other that provides the required firmware. I added it via #143 to the addon and released v5.1.2. I don't whether this helps, but let's give it a try. Also as this firmware can be redistributed via linux, there is a buildroot switch to include it into HAOS (refer to #143). So that would be the best solution and I can create a HAOS PR for this. But let's give it in the addon a first try.
obert0
commented
3 months ago @dfigus Thank you very much for looking into that. I update the addon, but the tuner still isn't visible inside tvheadend. I try to reboot the addon and HAOS without success. I will try more test tomorrow.
dfigus
commented
3 months ago Could you also share the kernel messages from dmesg from the addon once you test that?
obert0
commented
3 months ago Yes. But since the HAOS (maybe there's something that broke in the update) I have strange behavior with the tuner.
I think I will wait for a HAOS update before retry because I have really the impression of turning around. Maybe I should reinstall to get a fresh install... I will try to find the time in the next weeks.
7wells
commented
3 months ago @obert0 Does your device get enough power via USB?
obert0
commented
3 months ago Yes, I have a powered usb hub because I have TVHeadend on a Raspberry Pi already and I face this problem when I setuo this server. Before HAOS 12 I didn't have any issue with usb port.
dfigus
commented
3 months ago @obert0 I created PR home-assistant/operating-system/pull/3236 in HA OS to directly include the FW files.
dfigus
commented
3 months ago @obert0 there is a new HA OS 12.1 released which contains some fixes for USB, however the issues seem to be present already with HA OS 11.2:
Maybe you can give this a try.
If it's not working anymore, maybe you should report it on the HA OS repo as an issue. In the meantime you could downgrade to a working version via
ha os update --version <version that worked>@obert0 I created PR home-assistant/operating-system/pull/3236 in HA OS to directly include the FW files.
I had asked in October 2022 already to add firmware files and learnt (later) that it is not so easy. Now they confirmed this: https://github.com/home-assistant/operating-system/pull/3236#issuecomment-1994583170
To my understanding, this means that there is little we can do from the addon side, and they (at the moment) can do little from the HA OS side.
Placing the correct firmware files seems to remain a manual task for the time being.
@dfigus Would it be feasible that your addon supports this process? E.g., on initiation check if firmware files are available and if the tuner is recongised - and if not, offer the users a box where they could upload the files for the addon to retry initiation. Since I am no programmer, I have no idea if this is possible at all and if you want to try this. It's just a thought.
dfigus
commented
3 months ago @7wells: The ask from the PR is a little different to what you asked for. Some firmware files are already included in the linux kernel and can be freely distributed. To include it into HA OS you just need to enable the config option for it so it's baked into the image. The HA OS colleagues rejected this inclusions as they are running low on space in the rootfs partition.
However the firmware you requested is not part of the linux kernel and hence to include it into HA OS they would need to create a custom buildroot package for this. That's a very big difference.
What I can do is to symlink /share/firmware so that you can drop in there firmware files. Based on my research the firmware should be loaded on demand (when the tuner is accessed the first time) and hence not directly upon boot. Hence I would have thought including the XCx000 firmware in the addon would have solved the issue for @obert0 already - unless there is something that tries to load the firmware before it's accessed the first time when TVH starts in my addon... But as I don't have such a DVB tuner it's impossible to test it on my end. If including it in the addon does not work, then also symlink'ing /share/firmware would not solve it.
obert0
commented
3 months ago Hi @dfigus and @7wells,
I don't think it is possible from a addon perspective. The only way the firmware can be manually proposed to the kernel is by adding a line in the kernel boot (cmdline.txt) to consider a custom firmware file path. I don't think a simlink can work either because we can't add or modify the os filesystem to add the simlink even in ssh port 22222.
I try again without luck with the haos new update. Thank you for the downgrade command. I will try to load the last release before 12.0.
At least before the update a have a systematic way to get the firmware load with manual step. I still need to try a hardware methode to automatically replug the tuner.
Thank you guys for your time! I will keep you in touch.
dfigus
commented
3 months ago Thanks for the feedback. I just ordered a WinTV-HVR-935HD which should also require additional firmware based on linuxtv. Let's see whether I can get this running 😀
obert0
commented
3 months ago Oh wow! Thank you very much! I am very excited to see if you will face rhe same struggles. Are running haos on pi?
dfigus
commented
3 months ago @obert0 I'm running on RPI 4 and I got the WinTV-HVR-935HD running with the addon. I had to copy the firmware files to the /mnt/data/supervisor/firmware/ and during the reboot if fails initially, but it retries it until it succeeds:
[ 4.738730] cx25840 23-0044: Direct firmware load for v4l-cx231xx-avcore-01.fw failed with error -2
[ 4.738751] cx25840 23-0044: unable to open firmware v4l-cx231xx-avcore-01.fw
[ 5.129902] si2168 27-0064: firmware version: B 4.0.2
[ 142.135407] cx25840 23-0044: loaded v4l-cx231xx-avcore-01.fw firmware (16382 bytes)
[ 142.412149] si2168 27-0064: firmware version: B 4.0.2
[ 161.263968] si2168 27-0064: downloading firmware from file 'dvb-demod-si2168-b40-01.fw'
[ 162.008298] si2168 27-0064: firmware version: B 4.0.11
[ 162.017205] si2157 27-0060: Using ROM firmware.
[ 162.066901] si2157 27-0060: firmware version: 3.0.5The same retry also works when I plugged it in and had no fw files at this location. It kept retrying and once the files were there I picked them up and the adapter was working. I was downloading the files from https://github.com/OpenELEC/dvb-firmware/
I will investigate whether I can do something from the addon.
obert0
commented
3 months ago Hi @dfigus,
It was a easy process. It appears that it wasn't the same driver used for your tuner. Did you add the firmware path to cmdline.txt?
dfigus
commented
3 months ago No changes to cmdline.txt. The path /mnt/data/supervisor/firmware/ is symlimk'ed to /lib/firmware/updates which is a default search location for firmwares in linux.
obert0
commented
3 months ago Ok I will try to put it there instead. Did the folder firmware was already there?
github-actions[bot]
commented
2 months ago There hasn't been any activity on this issue recently, so we clean up some of the older and inactive issues. Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thanks!
dfigus
commented
2 months ago @7wells & @obert0 someone shared a workaround outside of github with me which seems to be persistent across HA OS upgrades:
/mnt/boot/ e.g. /mnt/boot/firmware and store the firmware files there./mnt/boot/cmdline.txt the firmware_class.path=/mnt/boot/firmware/ (or whatever you chose as the location).
You need to do this via ssh into the HA OS directly, port 222227wells
commented
2 months ago Yes, that must have been myself. 🤣
https://community.home-assistant.io/t/allow-loading-firmware-files-on-boot/632031/18
It had taken me a long time to understand that I had to ssh to the underlying system, hence port 22222.
See also there: https://github.com/home-assistant/operating-system/issues/1157#issuecomment-1374671958
PS: Your updated version 5.1.7 runs very nicely - thank you so much, Daniel! ❤️
obert0
commented
2 months ago Hi @dfigus, in my initial post I explain this solution. After adding the firmware with this solution I succeed to use the tuner with tvheadend. But still I had to make some manipulation to make the driver load the firmware.
For the moment I switch to another project, but soon I will try again to make this work.
dfigus
commented
2 months ago @obert0 That is not exactly what you wrote in your first post:
I enter in the HAOS in SSH to the port 22222 and add "firmware_class.path=/mnt/data/supervisor/share/firmware" in "/mnt/boot/cmdline.txt". I download the firmware and put it in "/share/firmware" with the ssh addon. Now on HAOS startup, the firmware is in place and TvHeadend start on boot. After boot I can't see the tuner in TvHeadend. This is the Kernel logs:
The important part here is that /mnt/data/ is not available during the early stage of the boot, however /mnt/boot/ is. Did you try to put the firmware into /mnt/boot/?
obert0
commented
2 months ago Hi @dfigus , I put the firmware into /mnt/boot/firmware and change cmdline.txt. But no success to get the tuner visible in TvHeadend... I use the last version 5.1.9 of your addon. But with this new position I can't move the firmware from HA to use my last fix.
I will get more time to make this work. My next step is to downgrade HAOS to version <12. I keep you inform. I also consider to just change my tuner. Did you recommend the one that you buy?
Thank you!
obert0
commented
2 months ago I roll back to HAOS 11.5 and I can reproduce the same behaviour from my original post. I think version 12+ change something in the usb that doesn't match my setup. I will keep try new update if it work, but I will stay on 11.5 until I found a fix. I found today that haos as dual boot for safe update that ease this step!
dfigus
commented
1 month ago My TV tuner WinTV-HVR-935HD works, but I don't really use it. I still use SATIP with my FritzBox tuners. In generell you need to make your TV tuner work with HA OS, afterwards it will also work with the addon - that's the general rule. In case HA OS 12+ does not work anymore, I would propose to open an issue to HA OS. As I can't really do anything more from the addon, I would close this issue and convert it to a discussion. There you can report your progress for others to read.
Hi everyone, I try to fork the project of @GauthamVarmaK to get the DVB tuner to work and add COMSKIP and network drive mapping in this project. I will be pleased to work with @dfigus too get this addon too it's full potential! But for the moment I have some problem with my tuner doesn't show up in TvHeadend. I have a Hauppauge WINTV 950Q and I figured the firmware isn't in the Kernel. So as I see in GitHub and thanks to @7wells I enter in the HAOS in SSH to the port 22222 and add "firmware_class.path=/mnt/data/supervisor/share/firmware" in "/mnt/boot/cmdline.txt". I download the firmware and put it in "/share/firmware" with the ssh addon. Now on HAOS startup, the firmware is in place and TvHeadend start on boot. After boot I can't see the tuner in TvHeadend. This is the Kernel logs:
Now if I unplugged and replugged the tunner I can see it in TVHeadend, but it is not usable. When I try to use it more line
usb 2-1.3.1L Tuner is busy. Error -16appear. Here the Kernel logs:Now if I remove the firmware from the folder and unplugged/replugged the tunner I can still see it in TVHeadend, but it is not usable. When I try to use it more line:
usb 2-1.3.1: Direct firmware load for dvb-fe-xc5000-1.6.114.fw failed with error -2xc5000: Upload failed. rc -2appear. Here the Kernel logs:Finally if put back the firmware in the folder and use TvHeadend I got this line:
[ 1296.524396] xc5000: Firmware dvb-fe-xc5000-1.6.114.fw loaded and runningAnd now it work in TvHeadend... But on each startup I have to :
Anyone have a idea of the problem? Many thanks