Error: Manifest not found!

[janhenriko]

Recently NRK have changed how the video-source is presented to the browser. Earlier the file "manifest.f4m" was a part of the source-code of the webpage showing the recording. Now this is revealed as a part of the query-string when clicking on "Play", so the script will not see the manifest-file, resulting in Error: Manifest not found!

[liangqi]

This is just a clone of original nrkopptak project. Do you have some idea to fix it in script?

[janhenriko]

No, I believe that the script need to change the way it extracts the manifest-information.

[Oevind]

Feil: Fann ikkje manifestet! 0 minutt og 1 sekund

Dessverre har jeg ingen anelse.

[bmbob]

Tried to make a "work for me"-patch, effd7adc0221ff429a520074cbc03da30d32548c. Please give feedback if there is any issues.

[janhenriko]

It worked very vell! Thank you.

They have changed subtitles, now available inn TTML-format under a different URL. Does not matter for me.

[bmbob]

Could you try with ef2acbbbffb76e02687fd28ee4ddca7803c774a6? Have only done some limited testing on tv.nrk.no, so may cause problems elsewhere.

[janhenriko]

It works like a charm! Thank you again!

[Oevind]

alt i orden igjen!

[petterreinholdtsen]

There are two different approaches for a fix available from https://github.com/bmbob/nrkopptak and https://github.com/kaistian/nrkopptak/ . Perhaps those should be merged into this version to have a common base?

[bmbob]

It's probably better to use https://github.com/rg3/youtube-dl anyway.

[petterreinholdtsen]

[bmbob]

It's probably better to use https://github.com/rg3/youtube-dl anyway.

Not in my experience, as this tool combines the subtitles and the video into one MKV file, while youtube-dl give you a video file without the subtitles. My video players make it a lot easier to show subtitles using the MKV file produced by this tool.

Happy hacking Petter Reinholdtsen

[bmbob]

Have you tested the fix(es) ? I can make a pull request if someone does some testing.

[petterreinholdtsen]

[bmbob]

Have you tested the fix(es) ? I can make a pull request if someone does some testing.

I'm using commit ef2acbbbffb76e02687fd28ee4ddca7803c774a6 as the fix myself, on top of commit b5a352f76b10db1c4572665f63e69e5cfe3a9d65. And this version work just fine. :)

Happy hacking Petter Reinholdtsen

[tnfno]

I have installed latest version, checked all dependencies but still get "Fann ikkje manifestet!" . Does it work for anyone else or has NRK changed something again? They switched to https on 1. June, could that be the reason?

Example; `./nrk-opptak fvd https://tv.nrk.no/serie/ut-i-naturen/DVNA92000314/20-11-2014 nrk-opptak 0.1.3c (229) nn (GPLv3) falkeklubben

PHP: /usr/bin/php ADOBEHDS: ./AdobeHDS.php FFMPEG: /usr/local/bin/avconv (mov_text) MKVMERGE: /usr/bin/mkvmerge MP4BOX: /usr/bin/MP4Box MEDIAINFO: /usr/bin/mediainfo

Kommandoar og val: fvd som betyr: debug flv flv_file verbose video

URL: https://tv.nrk.no/serie/ut-i-naturen/DVNA92000314/20-11-2014

Ukjend NRK TV

Feil: Fann ikkje manifestet!

Ingen filer vart lagra. `

[bmbob]

Got no error on that video here. Did you add the URL::Escape module too? Which OS do you use? The debug option would ideally have been more informative :)

$ nrk-opptak d https://tv.nrk.no/serie/ut-i-naturen/DVNA92000314/20-11-2014
nrk-opptak 0.1.3c (229) nn (GPLv3) falkeklubben

PHP:       /usr/bin/php
ADOBEHDS:  /usr/bin/AdobeHDS.php
FFMPEG:    /usr/bin/avconv (mov_text)
MKVMERGE:  /usr/bin/mkvmerge
MP4BOX:    /usr/bin/MP4Box
MEDIAINFO: /usr/bin/mediainfo

Kommandoar og val: d
        som betyr: debug

URL:        https://tv.nrk.no/serie/ut-i-naturen/DVNA92000314/20-11-2014

Korte dyrehistorier - 20.11.2014
Kun tilgjengeleg i Norge, til 2079-06-06.
Tre korte dyrehistorier: Grankorsnebb, bever, lappugle.
NRK TV, Natur, 9 minutter

0 minutt og 1 sekund

[tnfno]

Hi, I use Fedora 23, I used the debug option so I believe I got the same output as you as far as I can see? What is the URL::Escape module, tried to find a PHP or Linux module that matches, but the only thing I can find is a Ruby module, I believe that is not it.

[tnfno]

I am not doing Perl programing, but did some simple debug and have found that the $url seems to be passed OK, but get_content($url) returns nothing, so I guess the problem is in the get_content() function. I do not know Perl well enough to debug myself, but here are the variables in the function:

$ua = "LWP::UserAgent=HASH(0x864a5a8)" $response = "HTTP::Response=HASH(0x8f8cb80)"

[bmbob]

The package is named perl-URI-Encode for fedora, or install using : cpan URI::Escape (sorry for the typo in the above post)

[tnfno]

Thank you for the reply, but URI::Escape where already installed so still no luck. Have tried to find anything in the log files, but it does not look like any errors are logged. Any debugging I can do to find out more? I am a programmer, but my Perl knowledge is very limited.

[petterreinholdtsen]

[tnfno]

Thank you for the reply, but URI::Escape where already installed so still no luck. Have tried to find anything in the log files, but it does not look like any errors are logged. Any debugging I can do to find out more, as mentioned are my Perl knowledge very limited.

Are you using the two commits I mentioned in <URL: https://github.com/liangqi/nrkopptak/issues/4#issuecomment-230168070 >?

They are not applied to the liangqi/nrkopptak repo yet, unfortunately. No idea why.

Happy hacking Petter Reinholdtsen

[tnfno]

Yes, since it reports version 0.1.3c I guess that should be OK. I got the files from here https://github.com/liangqi/nrkopptak/tree/ef2acbbbffb76e02687fd28ee4ddca7803c774a6

[bmbob]

Can you modify to log the html and upload nrk.html to e.g. pastebin?

@@ -1114,6 +1114,9 @@

   unless ($url) {
     _warn('Feil: Fann ikkje manifestet!');
+    open(my $htmllog, '>', 'nrk.html');
+    print $htmllog $html;
+    close $htmllog;
     return undef;
   }
 }

[liangqi]

On Mac, got nothing for nrk.html...

And tried Ubuntu 16.04 tonight, it works. Just failed when converting .flv to .mp4.

[tnfno]

Tried to add bmbob's log , but the file comes out empty. So it looks like the URL is ok, but it is not able to load the HTML. Tried to wget the URL that the script reports and it works, so it is not a connection issue.

[bmbob]

Tried the script in a new Fedora 23 VM and got no errors, so I have no idea why it fails for you. Haven't got a Mac to test it. I think it's an external tool that causes the conversion to fail on some videos.

[tnfno]

Ok, I'll do some more testing and see if I find a solution.

[marhoy]

Noen som har fått dette til å funke under macOS? Jeg har 10.12.2, har klonet master og lagt til https://github.com/liangqi/nrkopptak/pull/5/commits Men jeg får fremdeles manifest-feilmeldingen:

marhoy@MBP: ~ $ nrk-opptak d https://tv.nrk.no/serie/ut-i-naturen/DVNA92000314/20-11-2014 nrk-opptak 0.1.3c (229) nn (GPLv3) falkeklubben

PHP: /usr/bin/php ADOBEHDS: /Users/marhoy/GitHub/nrkopptak-marhoy/bin/AdobeHDS.php FFMPEG: /Users/marhoy/bin/ffmpeg (mov_text) MKVMERGE: /Users/marhoy/bin/mkvmerge MP4BOX: /Users/marhoy/bin/MP4Box MEDIAINFO:

Kommandoar og val: d som betyr: debug

URL: https://tv.nrk.no/serie/ut-i-naturen/DVNA92000314/20-11-2014

Ukjend NRK TV

Feil: Fann ikkje manifestet! 0 minutt og 0 sekund

[janhenriko]

Ja, jeg kjører dette under MacOS 10.12.1. Nå skal det også sies at jeg kjører Homebrew for installasjon av øvrige komponenter. Anbefales. Hva angår feilmeldingen, kan du sjekke hvilke Perl-komponenter du har installert. Mener å huske at man ved overgangen til SSL (https) måtte legge til Perl SSL-moduler også.

Fyr opp Terminal, og kjør kommandoen instmodsh og velg l - List all installed modules. Se etter Net::HTTP og Net::SSLeay. Jeg ser også at jeg har både IO::HTML og IO::Socket::SSL

Alternativt kan du selvfølgelig dra i gang Wireshark og se på loggen fra nettverks-interface't ditt og se hva som faktisk traverserer av requests og respons. Men dette kan være (tid-/) krevende om du ikke kjenner til det fra før.

[liangqi]

@janhenriko, so you managed to run it well on macOS? Could you list full steps about the requirements? Thanks.

[petterreinholdtsen]

Something have changed on tv.nrk.no, and the ef2acbbbffb76e02687fd28ee4ddca7803c774a6 commit is no longer enough. I managed to get the script working again by applying this patch:

diff --git a/bin/nrk-opptak b/bin/nrk-opptak index 206ff74..959fdf5 100755 --- a/bin/nrk-opptak +++ b/bin/nrk-opptak @@ -451,7 +451,7 @@ sub process { my $json; my $progid;

• if ($html =~ /programid."(.?)"/) {
• if ($html =~ /programid."(.?)"/i) { $progid = $1; $json = get_content("https://psapi-we.nrk.no/mediaelement/$progid"); }

The HTML changed, programid is now programId. -- Happy hacking Petter Reinholdtsen

[petterreinholdtsen]

For the record, I just discovered that https://github.com/kaistian/nrkopptak had fixed this issue in his master branch. His fix for the latest problem was to change the string instead of making it case insensitive, which work too.

[liangqi]

@kaistian 's code works fine on Linux, but still failed with "Error: Manifest not found!" on macOS for me.

[janhenriko]

@liangqi , I'm not sure about what makes my setup work, but lately I got "Manifest not found" on my other Macbook. What I did was this.

1. Upgrade Homebrew
2. Upgrade CPAN cpan upgrade
3. Clone this repository https://github.com/kaistian/nrkopptak

My installed Perl modules are: Crypt::SSLeay Encode::Locale File::Listing HTML::Parser HTML::Tagset HTTP::Cookies HTTP::Daemon HTTP::Date HTTP::Message HTTP::Negotiate IO::HTML IO::Socket::SSL LWP LWP::MediaTypes LWP::Protocol::https Module::Build Mozilla::CA Net::HTTP Net::SSLeay Path::Class Perl Try::Tiny URI WWW::RobotRules XML::NamespaceSupport XML::Parser XML::SAX XML::SAX::Base XML::SAX::Expat XML::Simple

I can't remember which ones who are necessary and not. Good luck!

[kaistian]

@liangqi my patch did not introduce any new requirement to Perl, since I don't have access to macOS it's hard for me to investigate the issue.