search

svt / encore

Transcode media files in an epic manner
European Union Public License 1.2
273 stars 25 forks source link

Job execution failed: Error parsing ProbeResult from output #15

Closed deedos closed 2 years ago

deedos commented 3 years ago

Describe the bug Getting: "Job execution failed: Error parsing ProbeResult from output: '' after creating a job

To Reproduce Steps to reproduce the behavior: 1 - Creating a job using the swagger "try out" with the sample provided by the docs:

  {
    "profile": "program",
    "outputFolder": "/tmp/output",
    "baseName": "quicktest_",
    "inputs": [{
        "uri": "/tmp/input/test.mp4",
        "useFirstAudioStreams": 2,
        "type": "AudioVideo"
    }]
  }

Expected behavior A job to be created with success

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information): Docker compose provided by the docs

Additional context I can run "ffprobe" from the OS a get success with /tmp/input/test.mp4

Here the log:

encore_1  | 2021-12-01 17:29:01.568  INFO 1 --- [nio-8080-exec-7] s.s.o.e.h.EncoreJobHandler               : Adding job to queue.. EncoreJob(id=f77786d3-ff42-447b-b5c5-d658b7d70a3d, externalId=null, profile=program, outputFolder=/tmp/output, baseName=quicktest_, createdDate=2021-12-01T17:29:01.530894Z, progressCallbackUri=null, priority=0, message=null, progress=0, speed=null, startedDate=null, completedDate=null, debugOverlay=false, logContext={}, seekTo=null, duration=null, thumbnailTime=null, inputs=[AudioVideoInput(uri=/tmp/input/test.mp4, videoLabel=main, audioLabel=main, params={}, dar=null, useFirstAudioStreams=2, cropTo=null, padTo=null, videoFilters=[], audioFilters=[], analyzed=null, videoStream=null, audioStream=null, probeInterlaced=true)])
encore_1  | 2021-12-01 17:29:01.583  INFO 1 --- [nio-8080-exec-7] s.s.o.e.h.EncoreJobHandler               : Added job to queue
encore_1  | 2021-12-01 17:29:02.288  INFO 1 --- [   scheduling-3] s.s.o.e.s.p.JobPoller                    : Handling job f77786d3-ff42-447b-b5c5-d658b7d70a3d
encore_1  | 2021-12-01 17:29:02.301  INFO 1 --- [   scheduling-3] s.s.o.e.s.p.JobPoller                    : Running job
encore_1  | 2021-12-01 17:29:02.698 ERROR 1 --- [   scheduling-3] s.s.o.e.s.EncoreService                  : Job execution failed: Error parsing ProbeResult from output: ''
encore_1  | 
encore_1  | java.lang.RuntimeException: Error parsing ProbeResult from output: ''
encore_1  |     at se.svt.oss.mediaanalyzer.ffprobe.FfprobeAnalyzer.analyze(FfprobeAnalyzer.kt:72) ~[media-analyzer-1.0.3.jar!/:?]
encore_1  |     at se.svt.oss.mediaanalyzer.MediaAnalyzer.analyze(MediaAnalyzer.kt:60) ~[media-analyzer-1.0.3.jar!/:?]
encore_1  |     at se.svt.oss.encore.service.mediaanalyzer.MediaAnalyzerService.analyzeInput(MediaAnalyzerService.kt:28) ~[classes!/:?]
encore_1  |     at se.svt.oss.encore.service.EncoreService.encode(EncoreService.kt:72) ~[classes!/:?]
encore_1  |     at se.svt.oss.encore.service.poll.JobPoller.handleJob(JobPoller.kt:77) ~[classes!/:?]
encore_1  |     at se.svt.oss.encore.service.poll.JobPoller.init$lambda-2$lambda-1(JobPoller.kt:47) ~[classes!/:?]
encore_1  |     at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.3.9.jar!/:5.3.9]
encore_1  |     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
encore_1  |     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
encore_1  |     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
encore_1  |     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
encore_1  |     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
encore_1  |     at java.lang.Thread.run(Thread.java:829) [?:?]
encore_1  | Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
encore_1  |  at [Source: (String)""; line: 1, column: 0]
encore_1  |     at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |     at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4688) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |     at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4586) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |     at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3548) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |     at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3531) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |     at se.svt.oss.mediaanalyzer.ffprobe.FfprobeAnalyzer.analyze(FfprobeAnalyzer.kt:69) ~[media-analyzer-1.0.3.jar!/:?]
encore_1  |     ... 12 more
janderssonse commented 3 years ago

What os are you using? There are some differences for docker for mac, (a bit messier due to it's docker implementation), see the docs. For Linux (only tested this on Ubuntu) verify that the folders under tmp (input/output) also have broad permission and not only root or alike. I got this error when ffprobe in the docker-image did not have permission to read my host platform folders.

deedos commented 3 years ago

Hi,

I 'm using Linux (Debian 10). Input and output folders have 777 permission (drwxrwxrwx) and owned by root. The docker container has permission to write to both folders (Tested with sudo docker exec -it encore_encore_1 bash and could write files there).

Trying ffprobe inside the docker, got a core dumped

avtools@1f40e0d44c81:/tmp/input$ ffprobe -i test.mp4 
Illegal instruction (core dumped)

What else could I try ? Thanks in advance

deedos commented 2 years ago

Updating the issue: Did the very same deployment with my local machine running Ubuntu 20.04 and worked as expected. The other deployment was made on a Virtual Machine inside a RHEV (Red Hat Virtualization) Host. Being so, Could it be related to hw architecture ? Could I debug that somehow ?

Many thanks in advance

janderssonse commented 2 years ago

Have not had the time to test yet, but it could certainly be an architecture issue. I have not tested it in a virtual Machine, but now that you mention it, I'll have to just because I'm curious:). The Docker Compose file is not something we recommend for production use, but as for doing a quick test run it is certainly nice.

janderssonse commented 2 years ago

Describe the bug Getting: "Job execution failed: Error parsing ProbeResult from output: '' after creating a job

To Reproduce Steps to reproduce the behavior: 1 - Creating a job using the swagger "try out" with the sample provided by the docs:

  {
      "profile": "program",
      "outputFolder": "/tmp/output",
      "baseName": "quicktest_",
      "inputs": [{
          "uri": "/tmp/input/test.mp4",
          "useFirstAudioStreams": 2,
          "type": "AudioVideo"
      }]
  }

Expected behavior A job to be created with success

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information): Docker compose provided by the docs

Additional context I can run "ffprobe" from the OS a get success with /tmp/input/test.mp4

Here the log:

encore_1  | 2021-12-01 17:29:01.568  INFO 1 --- [nio-8080-exec-7] s.s.o.e.h.EncoreJobHandler               : Adding job to queue.. EncoreJob(id=f77786d3-ff42-447b-b5c5-d658b7d70a3d, externalId=null, profile=program, outputFolder=/tmp/output, baseName=quicktest_, createdDate=2021-12-01T17:29:01.530894Z, progressCallbackUri=null, priority=0, message=null, progress=0, speed=null, startedDate=null, completedDate=null, debugOverlay=false, logContext={}, seekTo=null, duration=null, thumbnailTime=null, inputs=[AudioVideoInput(uri=/tmp/input/test.mp4, videoLabel=main, audioLabel=main, params={}, dar=null, useFirstAudioStreams=2, cropTo=null, padTo=null, videoFilters=[], audioFilters=[], analyzed=null, videoStream=null, audioStream=null, probeInterlaced=true)])
encore_1  | 2021-12-01 17:29:01.583  INFO 1 --- [nio-8080-exec-7] s.s.o.e.h.EncoreJobHandler               : Added job to queue
encore_1  | 2021-12-01 17:29:02.288  INFO 1 --- [   scheduling-3] s.s.o.e.s.p.JobPoller                    : Handling job f77786d3-ff42-447b-b5c5-d658b7d70a3d
encore_1  | 2021-12-01 17:29:02.301  INFO 1 --- [   scheduling-3] s.s.o.e.s.p.JobPoller                    : Running job
encore_1  | 2021-12-01 17:29:02.698 ERROR 1 --- [   scheduling-3] s.s.o.e.s.EncoreService                  : Job execution failed: Error parsing ProbeResult from output: ''
encore_1  | 
encore_1  | java.lang.RuntimeException: Error parsing ProbeResult from output: ''
encore_1  |   at se.svt.oss.mediaanalyzer.ffprobe.FfprobeAnalyzer.analyze(FfprobeAnalyzer.kt:72) ~[media-analyzer-1.0.3.jar!/:?]
encore_1  |   at se.svt.oss.mediaanalyzer.MediaAnalyzer.analyze(MediaAnalyzer.kt:60) ~[media-analyzer-1.0.3.jar!/:?]
encore_1  |   at se.svt.oss.encore.service.mediaanalyzer.MediaAnalyzerService.analyzeInput(MediaAnalyzerService.kt:28) ~[classes!/:?]
encore_1  |   at se.svt.oss.encore.service.EncoreService.encode(EncoreService.kt:72) ~[classes!/:?]
encore_1  |   at se.svt.oss.encore.service.poll.JobPoller.handleJob(JobPoller.kt:77) ~[classes!/:?]
encore_1  |   at se.svt.oss.encore.service.poll.JobPoller.init$lambda-2$lambda-1(JobPoller.kt:47) ~[classes!/:?]
encore_1  |   at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.3.9.jar!/:5.3.9]
encore_1  |   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
encore_1  |   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
encore_1  |   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
encore_1  |   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
encore_1  |   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
encore_1  |   at java.lang.Thread.run(Thread.java:829) [?:?]
encore_1  | Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
encore_1  |  at [Source: (String)""; line: 1, column: 0]
encore_1  |   at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |   at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4688) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |   at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4586) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |   at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3548) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |   at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3531) ~[jackson-databind-2.12.4.jar!/:2.12.4]
encore_1  |   at se.svt.oss.mediaanalyzer.ffprobe.FfprobeAnalyzer.analyze(FfprobeAnalyzer.kt:69) ~[media-analyzer-1.0.3.jar!/:?]
encore_1  |   ... 12 more

For the first error, it is really hard to know what went wrong, due to https://github.com/svt/media-analyzer/issues/8. When that is fixed, It will be easier to pinpoint what went wrong.

janderssonse commented 2 years ago

Updating the issue: Did the very same deployment with my local machine running Ubuntu 20.04 and worked as expected. The other deployment was made on a Virtual Machine inside a RHEV (Red Hat Virtualization) Host. Being so, Could it be related to hw architecture ? Could I debug that somehow ?

Many thanks in advance Sadly I have no possible to try this configuration at the moment. If you have the time to dig into this and come up with a solution please do.

janderssonse commented 2 years ago

Hi,

I 'm using Linux (Debian 10). Input and output folders have 777 permission (drwxrwxrwx) and owned by root. The docker container has permission to write to both folders (Tested with sudo docker exec -it encore_encore_1 bash and could write files there).

Trying ffprobe inside the docker, got a core dumped

avtools@1f40e0d44c81:/tmp/input$ ffprobe -i test.mp4 
Illegal instruction (core dumped)

What else could I try ? Thanks in advance

I tried running this in Debian 11 (not in kvm, but in a virtualbox) , amd64-architecture, it encoded happily after the given example. Altough I had to set permissions, after that it worked fine. So, I guess something with the RedHat setup or arch, or possible debian 10 (but that would be odd, as docker-compose is only running on top of that). Anyway, as it run here on Debian 11, and the bug in media-analyzer has been posted and needs to be fixed to evaluate correct error codes from that, and we cant reproduce on redhat I'm closing this for now.