andimik commented 1 year ago

In https://github.com/piratfm/eti-tools/issues/35 (please scroll down) a user (@mrwish7 , Adam) mentioned that his friend (whom he is helping via team viewer) uses your stid135 drivers and complained the errors he gets on Norwegian DAB Muxes on 0.8W (10720 V) which are in DVB-GSE.

I have the same card 6903-x but I'm using the TBS drivers and have no problems.

Let me summarize the discrepancies:

	deeptho driver	TBS driver
bb headers in bbfedi2edi	corrupt	ok
NRK DAB on 0.8W	fails	ok
Pseudo PID used for GSE	dec 280	dec 270
sample file	https://github.com/piratfm/eti-tools/files/10337381/10720V_TS_dump.zip	https://github.com/piratfm/eti-tools/files/10333539/nrk_23nov2020_copy.zip
missing packets in Transedit	yes	no
GSE length gets 0 (according to bbfedi2edi)	yes	no

So his friend needs to patch a tool in order to use PID 280 instead of PID 270 and its output is corrupt.

Any chance to fix that? I've compared your fork with the one from TBS but is a nightmare to find the root cause as the fork is too much behind of its master.

deeptho commented 1 year ago

I do not use programs like transedit, but I did find and fix a bug in the drivers. So please try the latest version and see if it solves the problem

mrwish7 commented 1 year ago

Thanks both @deeptho and @andimik

We updated the deeptho linux_media and tried again just now, unfortunately the GSE output still seems to be corrupt.

I modified pts2bbf to take into account PID 280 instead of PID 270. It seems to pick up GSE fragments but they are corrupt or incomplete, as shown in @andimik 's post above

I also tested German DAB in DVB-GSE on 23.5°E (12641V 1500 DVB-S2) before updating the drivers, TransEdit reports no missing packets on that one but the GSE data is still corrupt and undecodable according to bbfedi2eti.

1°W 10720V file is with new drivers, 23.5°E 12641V file is with old version DVB-GSE TS dumps.zip

Best regards,

Adam

deeptho commented 1 year ago

This does not work with the official drivers either:

/mnt/devel/ciplus/dvblast/dvblast -s 5400000 -v 13 -f 10720000 -m psk_8 -3 -a 0 -1 171 -S 1 -k 4 -u | (sleep 10; pts2bbf) | bbfedi2eti -dst-ip 239.199.2.8 -dst-port 1234 | dablin_gtk

Data is flowing but no sound. This is of course after editing pts2bbf to extract the correct pid.

The following does work:

cat /tmp/nrk_23nov2020_copy.ts | ~/bbframe-tools/pts2bbf | bbfedi2eti -dst-ip 239.199.2.8 -dst-port 1234 | dablin_gtk

So either the stream has changed, or you did not use the official drivers in one of the tests

andimik commented 1 year ago

I have never tested dvblast for GSE. I am afraid this works.

The stream is still the same.

Yesterday I have again compiled both, TBS and your driver.

With TBS drivers I get sound via PID 270.

On the other hand, your driver compiles in a few seconds, but doesn't provide adapter0 or adapter1. So I can't test your recent updates.

Please try tune-s2 and dvbstream. Of course you need appropriate hardware like DD or 6903x or 6909x. Other cards which can handle GSE are not tested so far.

deeptho commented 1 year ago

I have never tested dvblast for GSE. I am afraid this works.

It probably does not matter what program you use (dvblast/dvbstream/...), because all it does is extract the stream data. dvblast works on 9.0E for DAB (but that is a transport stream, not GSE). So does neumoDVB+dvbsnoop

The stream is still the same. That is good to know. Did you use the official drivers. These are the only official ones: https://github.com/tbsdtv/linux_media.git

Yesterday I have again compiled both, TBS and your driver.

With TBS drivers I get sound via PID 270.

On the other hand, your driver compiles in a few seconds, but doesn't provide adapter0 or adapter1. So I can't test your recent updates.

Then it means you are doing something wrong. It should take longer than a few seconds and no adapter should be missing. However, that is unrelated to the DAB problem. How did you test my drivers then?

Also, inspecting the code for pts2bbf: it seems that this does not take into account the adaptation field in ts packets. Not sure if this is the problem (you may be lucky)

Please try tune-s2 and dvbstream. Of course you need appropriate hardware like DD or 6903x or 6909x. Other cards which can handle GSE are not tested so far.

mrwish7 commented 1 year ago

This does not work with the official drivers either:

/mnt/devel/ciplus/dvblast/dvblast -s 5400000 -v 13 -f 10720000 -m psk_8 -3 -a 0 -1 171 -S 1 -k 4 -u | (sleep 10; pts2bbf) | bbfedi2eti -dst-ip 239.199.2.8 -dst-port 1234 | dablin_gtk

Data is flowing but no sound. This is of course after editing pts2bbf to extract the correct pid.

The sleep 10 part may not actually be needed, pts2bbf on it's own should probably work.

I actually noticed the same as @andimik with installing the most recently modified of your drivers - they compiled within a couple of seconds and then there seemed to be an error. I think because of this, after rebooting, the old version would have still been in use. Sorry about that oversight! I followed the process in your readme with the ./install.sh script.

I guess the manual build and install process as done for the TBSDTV drivers would work too (without using the sh script)? We can probably try again with that method to see more clearly if/where the error occurs. I will post my findings.

Thanks,

Adam

deeptho commented 1 year ago

This does not work with the official drivers either: /mnt/devel/ciplus/dvblast/dvblast -s 5400000 -v 13 -f 10720000 -m psk_8 -3 -a 0 -1 171 -S 1 -k 4 -u | (sleep 10; pts2bbf) | bbfedi2eti -dst-ip 239.199.2.8 -dst-port 1234 | dablin_gtk Data is flowing but no sound. This is of course after editing pts2bbf to extract the correct pid.

The sleep 10 part may not actually be needed, pts2bbf on it's own should probably work.

I actually noticed the same as @andimik with installing the most recently modified of your drivers - they compiled within a couple of seconds and then there seemed to be an error. I think because of this, after rebooting, the old version would have still been in use. Sorry about that oversight! I followed the process in your readme with the ./install.sh script.

That would explain it. There are many procedures for compiling. One of the problems in mixing them is that multiple versions of the drivers can be installed simulateneously and then the wrong one is used.

If there is an error during compiling, then nothing will be installed.

I guess the manual build and install process as done for the TBSDTV drivers would work too (without using the sh script)? We can probably try again with that method to see more clearly if/where the error occurs. I will post my findings.

The script should be fine. It is not my script, but crazycat's and it aims 1) to build less (and thus faster) 2) to avoid breaking some ubuntu graphics drivers, 3) to avoid leaving older modules which could override newer ones.

Not sure if it fully works. You have to check every step. So execute each line in the script one by one.

I do it as follows cd media_build make dir DIR=../media # (or whereever the code is installed) make distclean

Then execute the script lines one by one, checking for errors

make install

Then find /lib/modules/uname -r/ -name stid135.ko -ls to checkif indeed something was installed and if more than one module is installed

the one in .../updates/... takes precedence

Thanks,

Adam

andimik commented 1 year ago

cd media_build
make dir DIR=../media # (or whereever the code is installed)
make distclean

But then I start ./install.sh and as soon as the message appears, I enter the su password and reboot immediately.

Let me try in the next days with dvblast and I will try to install your driver again.

deeptho commented 1 year ago

Also check the installed version: cat /sys/module/dvb_core/info/version type = "neumo"; version = "1.5";

and sudo grep -A 2 git /var/log/messages|tail -3 an 7 00:58:40 streacomold kernel: [ 8.371715] Latest git patches (needed if you report a bug to linux-media@vger.kernel.org): Jan 7 00:58:40 streacomold kernel: [ 8.371715] 79700dede4bc84b3104556a50d643fc69a0798ae tas2101: set timing lock status Jan 7 00:58:40 streacomold kernel: [ 8.371715] 404bca27c34b21cedc40c99e5751e0b167e3e635 tas2101 - increase constellation size $ You can compare the long hex string to the output of "git log" and compare.

mrwish7 commented 1 year ago

The error seems to relate to patching failures -

make -C /home/sat/blindscan_kernel/media_build/v4l stagingconfig
make[1]: Entering directory '/home/sat/blindscan_kernel/media_build/v4l'
make[2]: Entering directory '/home/sat/blindscan_kernel/media_build/linux'
Syncing with dir ../../media
Sync'ing file drivers/media/mc/mc-device.c (orig = d9ea77c566186b2c3a683be7ec2632cd20f7d6f3, copy = ffb79478cb90245c3d252308ca4bc98345e09f1e, patched = )
Sync'ing file drivers/media/v4l2-core/v4l2-ioctl.c (orig = d96ad2cdb025b7ce696ad5bf5296db0581a36523, copy = 035d7b73af53fc48dd2f0594240f468c0758aa9d, patched = )
Sync'ing file drivers/media/cec/core/cec-api.c (orig = 8de7e5358535ed51b600a92034d6375ad5b32e26, copy = c6fa962fdee15d2d3e1438f1cd107b691d3130c3, patched = )
make[3]: Entering directory '/home/sat/blindscan_kernel/media_build/linux'
Unapplying patches
patch -s -f -R -p1 -i ../backports/api_version.patch
1 out of 1 hunk FAILED -- saving rejects to file drivers/media/cec/core/cec-api.c.rej
1 out of 1 hunk FAILED -- saving rejects to file drivers/media/mc/mc-device.c.rej
1 out of 1 hunk FAILED -- saving rejects to file drivers/media/v4l2-core/v4l2-ioctl.c.rej
make[3]: Leaving directory '/home/sat/blindscan_kernel/media_build/linux'
Applying patches for kernel 5.13.0-39-generic
patch -s -f -N -p1 -i ../backports/api_version.patch
patch -s -f -N -p1 -i ../backports/pr_fmt.patch
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-is-param.c b/drivers/media/platform/samsung/exynos4-is/fimc-is-param.c
|index 9c816ae3b3e5..c318a6bd5f55 100644
|--- a/drivers/media/platform/samsung/exynos4-is/fimc-is-param.c
|+++ b/drivers/media/platform/samsung/exynos4-is/fimc-is-param.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-is.c b/drivers/media/platform/samsung/exynos4-is/fimc-is.c
|index e55e411038f4..c19d2ca8d07c 100644
|--- a/drivers/media/platform/samsung/exynos4-is/fimc-is.c
|+++ b/drivers/media/platform/samsung/exynos4-is/fimc-is.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-isp.c b/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
|index 855235bea46d..3039e7326c6c 100644
|--- a/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
|+++ b/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/s3c-camif/camif-capture.c b/drivers/media/platform/samsung/s3c-camif/camif-capture.c
|index 140854ab4dd8..b8ba2dc65297 100644
|--- a/drivers/media/platform/samsung/s3c-camif/camif-capture.c
|+++ b/drivers/media/platform/samsung/s3c-camif/camif-capture.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/s3c-camif/camif-core.c b/drivers/media/platform/samsung/s3c-camif/camif-core.c
|index 32892ab359ee..56c9c7fe7a99 100644
|--- a/drivers/media/platform/samsung/s3c-camif/camif-core.c
|+++ b/drivers/media/platform/samsung/s3c-camif/camif-core.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/s3c-camif/camif-regs.c b/drivers/media/platform/samsung/s3c-camif/camif-regs.c
|index e80204f5720c..235a40337893 100644
|--- a/drivers/media/platform/samsung/s3c-camif/camif-regs.c
|+++ b/drivers/media/platform/samsung/s3c-camif/camif-regs.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
make[2]: *** [Makefile:133: apply_patches] Error 1
make[2]: Leaving directory '/home/sat/blindscan_kernel/media_build/linux'
make[1]: *** [Makefile:380: stagingconfig] Error 2
make[1]: Leaving directory '/home/sat/blindscan_kernel/media_build/v4l'
make: *** [Makefile:26: stagingconfig] Error 2

Is there a specific commit of media_build that we can try?

Thanks,

Adam

andimik commented 1 year ago

Let us check the paths.

deeptho commented 1 year ago

The error seems to relate to patching failures -

make -C /home/sat/blindscan_kernel/media_build/v4l stagingconfig
make[1]: Entering directory '/home/sat/blindscan_kernel/media_build/v4l'
make[2]: Entering directory '/home/sat/blindscan_kernel/media_build/linux'
Syncing with dir ../../media
Sync'ing file drivers/media/mc/mc-device.c (orig = d9ea77c566186b2c3a683be7ec2632cd20f7d6f3, copy = ffb79478cb90245c3d252308ca4bc98345e09f1e, patched = )
Sync'ing file drivers/media/v4l2-core/v4l2-ioctl.c (orig = d96ad2cdb025b7ce696ad5bf5296db0581a36523, copy = 035d7b73af53fc48dd2f0594240f468c0758aa9d, patched = )
Sync'ing file drivers/media/cec/core/cec-api.c (orig = 8de7e5358535ed51b600a92034d6375ad5b32e26, copy = c6fa962fdee15d2d3e1438f1cd107b691d3130c3, patched = )
make[3]: Entering directory '/home/sat/blindscan_kernel/media_build/linux'
Unapplying patches
patch -s -f -R -p1 -i ../backports/api_version.patch
1 out of 1 hunk FAILED -- saving rejects to file drivers/media/cec/core/cec-api.c.rej
1 out of 1 hunk FAILED -- saving rejects to file drivers/media/mc/mc-device.c.rej
1 out of 1 hunk FAILED -- saving rejects to file drivers/media/v4l2-core/v4l2-ioctl.c.rej
make[3]: Leaving directory '/home/sat/blindscan_kernel/media_build/linux'
Applying patches for kernel 5.13.0-39-generic
patch -s -f -N -p1 -i ../backports/api_version.patch
patch -s -f -N -p1 -i ../backports/pr_fmt.patch
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-is-param.c b/drivers/media/platform/samsung/exynos4-is/fimc-is-param.c
|index 9c816ae3b3e5..c318a6bd5f55 100644
|--- a/drivers/media/platform/samsung/exynos4-is/fimc-is-param.c
|+++ b/drivers/media/platform/samsung/exynos4-is/fimc-is-param.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-is.c b/drivers/media/platform/samsung/exynos4-is/fimc-is.c
|index e55e411038f4..c19d2ca8d07c 100644
|--- a/drivers/media/platform/samsung/exynos4-is/fimc-is.c
|+++ b/drivers/media/platform/samsung/exynos4-is/fimc-is.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-isp.c b/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
|index 855235bea46d..3039e7326c6c 100644
|--- a/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
|+++ b/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/s3c-camif/camif-capture.c b/drivers/media/platform/samsung/s3c-camif/camif-capture.c
|index 140854ab4dd8..b8ba2dc65297 100644
|--- a/drivers/media/platform/samsung/s3c-camif/camif-capture.c
|+++ b/drivers/media/platform/samsung/s3c-camif/camif-capture.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/s3c-camif/camif-core.c b/drivers/media/platform/samsung/s3c-camif/camif-core.c
|index 32892ab359ee..56c9c7fe7a99 100644
|--- a/drivers/media/platform/samsung/s3c-camif/camif-core.c
|+++ b/drivers/media/platform/samsung/s3c-camif/camif-core.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
The text leading up to this was:
--------------------------
|diff --git a/drivers/media/platform/samsung/s3c-camif/camif-regs.c b/drivers/media/platform/samsung/s3c-camif/camif-regs.c
|index e80204f5720c..235a40337893 100644
|--- a/drivers/media/platform/samsung/s3c-camif/camif-regs.c
|+++ b/drivers/media/platform/samsung/s3c-camif/camif-regs.c
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
make[2]: *** [Makefile:133: apply_patches] Error 1
make[2]: Leaving directory '/home/sat/blindscan_kernel/media_build/linux'
make[1]: *** [Makefile:380: stagingconfig] Error 2
make[1]: Leaving directory '/home/sat/blindscan_kernel/media_build/v4l'
make: *** [Makefile:26: stagingconfig] Error 2

Is there a specific commit of media_build that we can try?

Thanks,

Adam

There could be multiple reasons:

not having run "make distclean"
some mismatch between kernel version and media_build. The way to resolve this is to try mutliple media_build versions until one works (starting with the newest), or try an older kernel. Another way is to edit media_build/backports/backports.txt and try to remove the offending patches.

mrwish7 commented 1 year ago

I've just tried the version of media_build here in your repo - https://github.com/deeptho/media_build

The drivers have now successfully built, so we have been able to try your latest version. Unfortunately the same problem is encountered as with the previous version, the DVB-GSE data is corrupt and unable to be decoded with the DAB ETI tools.

One thing I noticed, in neumoDVB - it seems we have to set the the Multistream/ISI number in order to get any data via -

dvbsnoop -adapter 7 -s ts -tsraw -b or dvbstream -c 7 -o 8192 etc.

I think this differs from the TBSDTV drivers, and may be where part of the problem is coming from. For example, on 23.5°E 12641 V 1500 there are two streams - MIS1 with DR Deutschland and MIS2 with Antenne DE. I can't get any data via your drivers unless I set the MIS value (1 or 2). However, in the bbfedi2eti tool it's possible to filter the data by MIS with the -mis flag, which suggests that the tested DVB-GSE data via the TBSDTV drivers is arriving without any MIS filtering having taken place. This would probably also affect 1°W as MIS 171 is being used. Maybe the filtering removes some of the expected GS data that pts2bbf and bbfedi2eti need?

It seems like the last synchronised point in your version of media_build with TBSDTV is a commit on 19th October 2021 in the TBSDTV version

Disable VIDEO_ROCKCHIP_ISP1.

deeptho commented 1 year ago

I've just tried the version of media_build here in your repo - https://github.com/deeptho/media_build

The drivers have now successfully built, so we have been able to try your latest version. Unfortunately the same problem is encountered as with the previous version, the DVB-GSE data is corrupt and unable to be decoded with the DAB ETI tools.

Yes, and this problem is also present in the official drivers (linux and windows) of the tbs card. I tested this for NRK on 0.8W

One thing I noticed, in neumoDVB - it seems we have to set the the Multistream/ISI number in order to get any data via -

dvbsnoop -adapter 7 -s ts -tsraw -b or dvbstream -c 7 -o 8192 etc.

Indeed, you have to specify this, but in the tuning program and not in dvbsnoop and NOT for NRK (single stream).

I think this differs from the TBSDTV drivers, and may be where part of the problem is coming from. For example, on 23.5°E 12641 V 1500 there are two streams - MIS1 with DR Deutschland and MIS2 with Antenne DE. I can't get any data via your drivers unless I set the MIS value (1 or 2). However, in the bbfedi2eti tool it's possible to filter the data by MIS with the -mis flag, which suggests that the tested DVB-GSE data via the TBSDTV drivers is arriving without any MIS

Mis is a tuning parameter. It is not even possibble to send more than one MIS simultaneously over the same adapter. There are many tuning programs which allow setting the stream_id (e.g., dvblast, neumodvb, ...)

Im any case, this is not related to the problem, because the NRK mux is a single stream mux and does not work.

filtering having taken place. This would probably also affect 1°W as MIS 171 is being used. Maybe the filtering removes >some of the expected GS data that pts2bbf and bbfedi2eti need?

This is not a multistreambut a single stream with a nonconventional stream id of 171. WIth the latest neumodvb drivers "-1" should also work as it will then select whatever it finds first.

It seems like the last synchronised point in your version of media_build with TBSDTV is a commit on 19th October 2021 in the TBSDTV version

Please try the official tbs drivers. I have found that they don't work either. There has been no need to syncrhonize as the main changes are not relevant for stid135.

If you can find a version of the drivers that works, then let me know and I can investigate. As far as I can tell the NRK stream I receive is NOT corrupt (but there are some lost packets due to the dish being not large enough). All looks fine. It could be a difference in format compared to digital devices (but then why would people report that it works on tbs6903X?), or it could be something else, but I can only tell you that I cannot get the software to work on the official drivers. So it would be good to be really sure if it works for other people or not WITH UNMODIFIED DRIVERS. That includes being sure that not accidentally a wrong version was installed.

If you can make it work, tell me the driver version and the user space commands involved. I can then look for differences.

deeptho commented 1 year ago

And here is the reason why I suspect a format problem. According to what I read, I need a command like

dvbsnoop -adapter 1 -s ts -tsraw -b | pts2bbf | ts2na -s 12 -p 1063 | na2ni | ni2out --list The first part reads the data and sends it to tsout pts2bbf extracts the encapsulated stream. Based on its name, it expects that the output is "base band frame".

However, ts2na expects its input to be a transport stream, and the data that it receives clearly is not a transport stream, but a "GSE" stream.

So I don't really understand the logic of the tools. I received these types of command lines via a private message and also from data I found on the web.

mrwish7 commented 1 year ago

Thanks @deeptho I am very curious myself, as @andimik seems to be able to get uncorrupted data for 1°W and 23.5°E.

Thanks for your clarification. I have only decoded the 23.5°E streams myself using an SDR device before. The leandvb software can decode the raw BBFrames (including both MIS 1 and MIS 2 together) to a file, which can then also be read using bbfedi2eti with the -mis option to select the stream required. Maybe the required MIS filter for TBS cards cuts off the BBFrame headers which pts2bbf and bbfedi2eti are looking for.

And here is the reason why I suspect a format problem. According to what I read, I need a command like

dvbsnoop -adapter 1 -s ts -tsraw -b | pts2bbf | ts2na -s 12 -p 1063 | na2ni | ni2out --list The first part reads the data and sends it to tsout pts2bbf extracts the encapsulated stream. Based on its name, it expects that the output is "base band frame".

However, ts2na expects its input to be a transport stream, and the data that it receives clearly is not a transport stream, but a "GSE" stream.

So I don't really understand the logic of the tools. I received these types of command lines via a private message and also from data I found on the web.

Ah yes - pts2bbf is only useful in combination with the bbfedi2eti tool. The various ETI tools are used differently together depending on the format of the satellite feed in question.

The only known GSE format DAB feeds are the NRK on 1°W 10720V and the two German national muxes on 23.5°E 12641V. For Digital Devices cards and TBS 6903X plus TBS 6909X (if they can be got to work), this requires pts2bbf and then bbfedi2eti to extract the ETI-NI data that can be played with dablin.

ts2na -s 12 -p 1063 | na2ni | ni2out --list

That line refers to PID 1063 so would relate to one of the UK DAB feeds for SDL National on 9°E/28.2°E. These are in regular MPEG-TS format, encapsulated as ETI-NA inside a PID (1063 in this case). The dvbsnoop output of the transponder would be passed directly to ts2na (to extract ETI-NA from the PID, with the given byte offset), then na2ni to convert that to the required ETI-NI format -

dvbsnoop -adapter 1 -s ts -tsraw -b | ts2na -s 12 -p 1063 | na2ni | ni2out --list or

dvbstream -c 1 -o 1063 | ts2na -s 12 -p 1063 | na2ni | ni2out --list

ni2out --list just shows decodes the list of services and bitrates from the valid ETI-NI data. dablin/dablin_gtk can be substituted to listen to the services on the ensemble.

deeptho commented 1 year ago

Thanks @deeptho I am very curious myself, as @andimik seems to be able to get uncorrupted data for 1°W and 23.5°E.

Thanks for your clarification. I have only decoded the 23.5°E streams myself using an SDR device before. The leandvb software can decode the raw BBFrames (including both MIS 1 and MIS 2 together) to a file, which can then also be read using bbfedi2eti with the -mis option to select the stream required. Maybe the required MIS filter for TBS cards cuts off the BBFrame headers which pts2bbf and bbfedi2eti are looking for.

So that explains a lot. the commands for SDR are of course very different than for linux dvb. Specifically I think indeed the bbframe headers are missing (because not needed) by default. They could be re-added, but that seems unneeded (I expect all needed data to be in the payload, but perhaps that assumption is wrong) and could break other things.

For SDR, the headers are probably there and as there has been no filtering for mis, they are needed to (at least) extracvt the mis to perform the filtering.

s - pts2bbf is only useful in combination with the bbfedi2eti tool. The various ETI tools are used differently together depending on the format of the satellite feed in question.

I tried that as well, but without success, perhaps again because the bbheaders are missing.

The only known GSE format DAB feeds are the NRK on 1°W 10720V and the two German national muxes on 23.5°E 12641V. For Digital Devices cards and TBS 6903X plus TBS 6909X (if they can be got to work), this requires pts2bbf and then bbfedi2eti to extract the ETI-NI data that can be played with dablin.

ts2na -s 12 -p 1063 | na2ni | ni2out --list

That line refers to PID 1063 so would relate to one of the UK DAB feeds for SDL National on 9°E/28.2°E. These are in regular MPEG-TS format, encapsulated as ETI-NA inside a PID (1063 in this case). The dvbsnoop output of the transponder would be passed directly to ts2na (to extract ETI-NA from the PID, with the given byte offset), then na2ni to convert that to the required ETI-NI format -

Yes, I know. I tried various pids and cvan get ts2na produce an output, but it is not right

dvbsnoop -adapter 1 -s ts -tsraw -b | ts2na -s 12 -p 1063 | na2ni | ni2out --list or

dvbstream -c 1 -o 1063 | ts2na -s 12 -p 1063 | na2ni | ni2out --list

ni2out --list just shows decodes the list of services and bitrates from the valid ETI-NI data. dablin/dablin_gtk can be substituted to listen to the services on the ensemble.

I know. I have also tested dablin. ALl works fine on a regular transport stream.

So probably the tools could be simplified, or as a workaround the drivers could be reconfigured. The fact that @andimik gets a result could be explained by using some modified drivers?

andimik commented 1 year ago

Yes, I confirm, I get audio on 0.8W and 23.5. Let me find some files which I have uploaded here in github.com.

BTW, I have discovered most of the DAB feeds. NRK was already known long before, as some fragments can be seen in strings even with a TBS 5927, but I found the 8th stream (Riks).

deeptho commented 1 year ago

@andimik: what I need is to know exactly which drivers and commands are working for you (e.g. NRK or the German feed on 23.5 which is stronger for me).

I am currently inspecting the stid135 registers. It seems that bbframes are being output. So this is unexpected.

Which version of eti-tools should I use? This one (I am currently using it): https://github.com/piratfm/eti-tools or this one https://github.com/newspaperman/eti-tools.git

Currently I am using a command like dvbsnoop -adapter 1 -s ts -tsraw -b | pts2bbf | bbfedi2eti -mis 1 -dst-ip 239.199.2.8 -dst-port 1234 | dablin_gtk while tunes to 12641v at 23.5 east, stream 1. This produces the messages: EnsembleSource: reading ETI from 'stdin' EnsembleSource: EOF reached!

But no audio

mrwish7 commented 1 year ago

So that explains a lot. the commands for SDR are of course very different than for linux dvb. Specifically I think indeed the bbframe headers are missing (because not needed) by default. They could be re-added, but that seems unneeded (I expect all needed data to be in the payload, but perhaps that assumption is wrong) and could break other things.

For SDR, the headers are probably there and as there has been no filtering for mis, they are needed to (at least) extracvt the mis to perform the filtering.

Yes, with an SDR the software decoder tool gives the full BBFrame including header.

This post - https://github.com/piratfm/eti-tools/issues/35#issuecomment-730238085 on the eti-tools issue page by @newspaperman (who may be able to help us here) implies that some changes to the driver code he has discussed in the above messages on the page set the card registers (TBS6903X/6909X) to "BBFrame mode" - the byte data he posts seem to include the BBFrame header as the MIS is present in those first few bytes.

mrwish7 commented 1 year ago

Currently I am using a command like dvbsnoop -adapter 1 -s ts -tsraw -b | pts2bbf | bbfedi2eti -mis 1 -dst-ip 239.199.2.8 -dst-port 1234 | dablin_gtk while tunes to 12641v at 23.5 east, stream 1. This produces the messages: EnsembleSource: reading ETI from 'stdin' EnsembleSource: EOF reached!

But no audio

For 23.5°E, when I've successfully decoded (with full BBFrames via SDR), the following was sufficient, with just MIS specified -

bbfedi2eti -mis 1 | dablin_gtk or bbfedi2eti -mis 2 | dablin_gtk

But of course -dst-ip and -dst-port would be required instead for 1°W due to multiple IP addresses within the same stream.

andimik commented 1 year ago

There are no modifications necessary. The TBS driver works out of the box for GSE. I will later tell you exactly, what version I am using.

deeptho commented 1 year ago

I have just rechecked with the standard drivers. It seems the format output there is different. The stream did not play because of some user space issue: using --mis instead of -mis

The following command plays the German mux. dvbsnoop -adapter 0 -s ts -tsraw -b| pts2bbf |bbfedi2eti -mis 2 | dablin_gtk

I think neumodvb outputs a different format: GSE instead of bbframe So now I will have to look into that.

deeptho commented 1 year ago

I have made some changes to the driver to change the format to "raw bbframe" for the GSE muxes on 0.8W and the 2 muxes on 23.5E. Now they work (no patches needed to pts2bbf) usually as follows:

23.5E dvbsnoop -adapter 3 -s ts -tsraw -b| pts2bbf | bbfedi2eti -mis 1 | dablin_gtk

0.8W dvbsnoop -adapter 1 -s ts -tsraw -b| pts2bbf |bbfedi2eti -dst-ip 239.199.2.4 -dst-port 1234 | dablin_gtk

Sometimes they don't work and then the whole command needs to be restarted

Fixed in 19e872f01a1a5c77b92bd8e

mrwish7 commented 1 year ago

Thanks very much @deeptho

I will get in touch with my friend so we can try it out.

andimik commented 1 year ago

I could now - at least - compile your latest driver.

No chance to use git clone --depth=1 https://github.com/tbsdtv/media_build.git, I had to use git clone --depth=1 https://github.com/deeptho/media_build.git media_build_deeptho instead.

a) I needed to install sudo apt-get install libproc-processtable-perl . This is NOT required for the TBS driver! Only then ./install.sh was successfully. b) But I don't get any output from the card using tune-s2 and dvbstream?? Because my installed version of tune-s2 does NOT allow to specify the MIS!! But I need it for the DiSEqC rotor. I also tried yours in the sub-directory, but I cannot lock with it.

Attached please find a file recorded with the TBS driver under Ubuntu.

git clone --depth=1 https://github.com/tbsdtv/media_build.git

bumux_20230106_tbsdriver.zip

Success!

dvblast -s 1500000 -v 13 -f 12642000 -m qpsk -3 -a 0 -1 1 -u | pts2bbf | bbfedi2eti -mis 1 | ni2out --list

or

dvblast -s 1500000 -v 13 -f 12642000 -m qpsk -3 -a 0 -1 1 -u | pts2bbf | bbfedi2eti -mis 1 | dablin_gtk

grafik grafik

And the output is really good, see BBfength line (the bbfedi2edi debug messages have been uncommented), which is now higher than 0x14F0

GSE Header:72 d8 0
b8 42 2 a7 d0
BBlength:14fa
b8 42 1 79 d0
BBlength:f3a
GSE Header:70 f0 0
b8 42 2 a7 d0
BBlength:14fa
b8 42 1 a7 d0
BBlength:14fa
GSE Header:c4 c5 8
b8 42 2 a7 d0
BBlength:14fa
b8 42 1 a7 d0
BBlength:14fa
GSE Header:72 d8 0
b8 42 2 a7 d0
BBlength:14fa
b8 42 1 79 d0
BBlength:f3a
GSE Header:70 f0 0
b8 42 2 a7 d0
BBlength:14fa
b8 42 1 a7 d0
BBlength:14fa
GSE Header:c4 c5 8
b8 42 2 a7 d0
BBlength:14fa
b8 42 1 a7 d0
BBlength:14fa
GSE Header:72 d8 0
b8 42 2 a7 d0
BBlength:14fa
b8 42 1 79 d0
BBlength:f3a
GSE Header:70 f0 0
b8 42 2 a7 18
BBlength:14e3
b8 42 1 a7 d0
BBlength:14fa

bumux_20230106_deepthodriver.zip (Remark: the date was wrong, it's from today, after your new update!)

Sometimes they don't work and then the whole command needs to be restarted

Yes, indeed.

andimik commented 1 year ago

Update: I forgot to mention, the PID is now 0x270 like with the TBS driver.

grafik

Let me try with NRK later, then we may close this issue.

@deeptho many thanks!

mrwish7 commented 1 year ago

I can confirm, all works perfectly now for both 23.5°E 12641V 1500 and also NRK/Riks on 1°W 10720V 5400. I have just tested both with satesco and his TBS6903X.

Thanks very much @deeptho and also @andimik for your help in debugging and fixing this.

andimik commented 1 year ago

Yes, I also tried NRK and Riks (beside of Bundesmux 1 and 2) and confirm it!

Benedek012 commented 5 months ago

This is 23,5°E 12641V 1500 Enigma 2 Box working???And Telnet,???? wget?????LamedB???

andimik commented 5 months ago

I guess you have the wrong hardware for GSE. You need a PC (not a laptop) and a professional DVB-S2X PCIe card.

Other solutions will not work according to my knowledge. But if you found a new way out, please let us know.

andimik commented 2 months ago

@deeptho

Don't wanted to open a new issue.

Was there any major change in your driver recently? Seems it worked last year.

Had to reinstall Ubuntu and had no success in locking with tune-s2 to the GSE transponder on 23.5 (Bundesmux, see above) with your (latest) driver. But the blindscan can lock and identifies both streams. Very strange. I could lock the neighbor transponder, BTW.

Then I installed TBS driver and could lock without problems with tune-s2. But then blindscan does not work.

deeptho commented 2 months ago

I do not know what version you are using, and the drivers have a changelog, so you can check yourself.

deeptho / linux_media

GSE support (needed for Norwegian DAB) #6

Success!