search

wpoa / open-access-media-importer

A tool for harvesting media files from Open Access articles for upload into Wikimedia Commons
http://commons.wikimedia.org/wiki/User:Open_Access_Media_Importer_Bot
23 stars 8 forks source link

ERROR: GStreamer encountered a general stream error. #21

Open Daniel-Mietchen opened 12 years ago

Daniel-Mietchen commented 12 years ago

Converting “/home/daniel/.cache/open-access-media-importer/media/raw/pmc_doi/pone.0042990.s006.mov”, saving into “/home/daniel/.cache/open-access-media-importer/media/refined/pmc_doi/pone.0042990.s006.mov.ogv” … ERROR: GStreamer encountered a general stream error.

Affected DOIs: 10.1371/journal.pone.0042990 also video 1 of 10.1186/1472-6793-11-13

Daniel-Mietchen commented 12 years ago

A variant: Converting “/home/daniel/.cache/open-access-media-importer/media/raw/pmc_doi/pone.0011915.s005.avi”, saving into “/home/daniel/.cache/open-access-media-importer/media/refined/pmc_doi/pone.0011915.s005.avi.ogv” …

\ (oa-cache:22497): WARNING **: ffmpegcsp0: size 798728 is not a multiple of unit size 798720 ERROR: GStreamer encountered a general stream error.

Affected DOI: 10.1371/journal.pone.0011915

erlehmann commented 12 years ago

File from second comment is hosted at http://www.plosone.org/article/fetchSingleRepresentation.action?uri=info:doi/10.1371/journal.pone.0011915.s005. It works with mplayer and ffplay, but not with Totem (GStreamer-based).

erlehmann commented 12 years ago 
gst-launch-0.10 playbin uri=file:///tmp/journal.pone.0011915.s005.avi
geht auch nicht. 
** (gst-launch-0.10:8893): WARNING **: vconv: size 798728 is not a multiple of unit size 798720
FEHLER: Von Element /GstPlayBin:playbin0/GstDecodeBin:decodebin0/GstAviDemux:avidemux0: Interner Datenstromfehler.
Zusätzliche Fehlerdiagnoseinformation:
gstavidemux.c(5212): gst_avi_demux_loop (): /GstPlayBin:playbin0/GstDecodeBin:decodebin0/GstAviDemux:avidemux0:
streaming stopped, reason error
erlehmann commented 12 years ago

Reported as GStreamer Bug 686550: https://bugzilla.gnome.org/show_bug.cgi?id=686550

Daniel-Mietchen commented 12 years ago

OK, knowing that, we should now just make sure the program does not crash at that point.

erlehmann commented 12 years ago

GStreamer claims to have it fixed. https://bugzilla.gnome.org/show_bug.cgi?id=686550

Daniel-Mietchen commented 12 years ago

Just reran these three DOIs - doesn't look like a fix to me: For the first one, only two of 15 videos are converted and uploaded (in fact, they were already uploaded when I filed the bug), for the second, one video is not even attempted to be converted, and for the third DOI, the converter stalls. Full log below.

daniel@oami-host:~/open-access-media-importer$ echo 10.1371/journal.pone.0042990 | ./oami_pmc_doi_import Removing “/home/daniel/.local/share/open-access-media-importer/pmc_doi.sqlite” … done. Input DOIs, delimited by whitespace: Getting PubMed Central IDs for given DOIs … found: 3416812 Downloading “http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pmc&id=3416812”, saving into directory “/home/daniel/.cache/open-access-media-importer/metadata/raw/pmc_doi” … 100% |#########################################################################| /usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:463: SAWarning: Unicode type received non-unicode bind param value. param.append(processorskey) “A Single Protofilament Is Sufficient to Support Unidirectional Walking of Dynein and Kinesin”: 8 × video/quicktime 2 × video/x-msvideo 4 × image/tiff 2 × application/msword

Checking MIME types … 100% |#########################################################################| Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s006.mov. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s007.mov. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s008.mov. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s009.mov. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s010.mov. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s011.mov. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s012.mov. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s013.mov. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s014.avi. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416812/bin/pone.0042990.s015.avi. “/home/daniel/.cache/open-access-media-importer/media/refined/pmc_doi/pone.0042990.s014.avi.ogv” uploaded to http://commons.wikimedia.org/w/api.php. “/home/daniel/.cache/open-access-media-importer/media/refined/pmc_doi/pone.0042990.s015.avi.ogv” uploaded to http://commons.wikimedia.org/w/api.php. daniel@oami-host:~/open-access-media-importer$ echo 10.1186/1472-6793-11-13 | ./oami_pmc_doi_import Removing “/home/daniel/.local/share/open-access-media-importer/pmc_doi.sqlite” … done. Input DOIs, delimited by whitespace: Getting PubMed Central IDs for given DOIs … found: 3177774 Downloading “http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pmc&id=3177774”, saving into directory “/home/daniel/.cache/open-access-media-importer/metadata/raw/pmc_doi” … 100% |#########################################################################| /usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:463: SAWarning: Unicode type received non-unicode bind param value. param.append(processorskey) “Monitoring the wild black bear's reaction to human and environmental stressors”: 4 × video/x-msvideo

Checking MIME types … 100% |#########################################################################| Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3177774/bin/1472-6793-11-13-S1.AVI. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3177774/bin/1472-6793-11-13-S2.AVI. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3177774/bin/1472-6793-11-13-S3.AVI. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3177774/bin/1472-6793-11-13-S4.AVI. Skipping “/home/daniel/.cache/open-access-media-importer/media/refined/pmc_doi/1472-6793-11-13-S2.AVI.ogv”, already exists at http://commons.wikimedia.org/w/api.php. Skipping “/home/daniel/.cache/open-access-media-importer/media/refined/pmc_doi/1472-6793-11-13-S3.AVI.ogv”, already exists at http://commons.wikimedia.org/w/api.php. Skipping “/home/daniel/.cache/open-access-media-importer/media/refined/pmc_doi/1472-6793-11-13-S4.AVI.ogv”, already exists at http://commons.wikimedia.org/w/api.php. daniel@oami-host:~/open-access-media-importer$ echo 10.1371/journal.pone.0011915 | ./oami_pmc_doi_import Removing “/home/daniel/.local/share/open-access-media-importer/pmc_doi.sqlite” … done. Input DOIs, delimited by whitespace: Getting PubMed Central IDs for given DOIs … found: 2912860 Downloading “http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pmc&id=2912860”, saving into directory “/home/daniel/.cache/open-access-media-importer/metadata/raw/pmc_doi” … 100% |#########################################################################| /usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py:463: SAWarning: Unicode type received non-unicode bind param value. param.append(processorskey) “Turning the Table: Plants Consume Microbes as a Source of Nutrients”: 4 × image/tiff 5 × video/x-msvideo

Checking MIME types … 100% |#########################################################################| Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2912860/bin/pone.0011915.s005.avi. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2912860/bin/pone.0011915.s006.avi. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2912860/bin/pone.0011915.s007.avi. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2912860/bin/pone.0011915.s008.avi. Skipping http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2912860/bin/pone.0011915.s009.avi. Converting “/home/daniel/.cache/open-access-media-importer/media/raw/pmc_doi/pone.0011915.s007.avi”, saving into “/home/daniel/.cache/open-access-media-importer/media/refined/pmc_doi/pone.0011915.s007.avi.ogv” … \ (oa-cache:6751): WARNING **: ffmpegcsp0: size 798728 is not a multiple of unit size 798720

erlehmann commented 12 years ago

You will have to install the fix for it to work – that means installing GStreamer from yesterday.

Daniel-Mietchen commented 11 years ago

Another example: 10.1371/journal.pone.0010872

Converting “/home/danielmietchen/.cache/open-access-media-importer/media/raw/pmc_doi/pone.0010872.s005.avi”, saving into “/home/danielmietchen/.cache/open-access-media-importer/media/refined/pmc_doi/pone.0010872.s005.avi.ogv” … \ (oa-cache:28607): WARNING **: ffmpegcsp3: size 212168 is not a multiple of unit size 212160 ERROR: GStreamer hat einen allgemeinen Datenstromfehler festgestellt.

Daniel-Mietchen commented 11 years ago

Another one:

Converting “/home/danielmietchen/.cache/open-access-media-importer/media/raw/pmc_doi/pone.0011915.s005.avi”, saving into “/home/danielmietchen/.cache/open-access-media-importer/media/refined/pmc_doi/pone.0011915.s005.avi.ogv” … \ (oa-cache:25415): WARNING **: ffmpegcsp0: size 798728 is not a multiple of unit size 798720 ERROR: GStreamer hat einen allgemeinen Datenstromfehler festgestellt.

RaphaelWimmer commented 11 years ago

pone.0011915.s005.avi pone.0010872.s005.avi

both AVI with 'rawbgr24flip' stream (raw, uncompressed video) - works with mplayer

Daniel-Mietchen commented 11 years ago

Next one: 10.1371/journal.pcbi.1003014

Converting “/home/danielmietchen/.cache/open-access-media-importer/media/raw/pmc_pmcid/pcbi.1003014.s003.avi”, saving into “/home/danielmietchen/.cache/open-access-media-importer/media/refined/pmc_pmcid/pcbi.1003014.s003.avi.ogv” … ERROR: GStreamer hat einen allgemeinen Datenstromfehler festgestellt. ERROR: GStreamer hat einen allgemeinen Datenstromfehler festgestellt.

Daniel-Mietchen commented 11 years ago

One more: 10.1371/journal.pgen.1000972

Daniel-Mietchen commented 11 years ago

One more: 10.1186/1475-925X-12-54

Daniel-Mietchen commented 11 years ago

One more: 10.1371/journal.pone.0076721

Daniel-Mietchen commented 11 years ago

Some more: PMC3832380 (10.1371/journal.pone.0080500):

Converting “/home/danielmietchen/.cache/open-access-media-importer/media/raw/pmc_doi/http%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC3832380%2Fbin%2Fpone.0080500.s006.avi”, saving into “/home/danielmietchen/.cache/open-access-media-importer/media/refined/pmc_doi/http%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC3832380%2Fbin%2Fpone.0080500.s006.avi.ogg” … ERROR: GStreamer hat einen allgemeinen Datenstromfehler festgestellt.
done.
Converting “/home/danielmietchen/.cache/open-access-media-importer/media/raw/pmc_doi/http%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC3832380%2Fbin%2Fpone.0080500.s007.mpeg”, saving into “/home/danielmietchen/.cache/open-access-media-importer/media/refined/pmc_doi/http%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC3832380%2Fbin%2Fpone.0080500.s007.mpeg.ogg” … ERROR: Die Art des Datenstroms konnte nicht ermittelt werden.
ERROR:root:Unknown audio/video stream combination.: Skipping conversion of “/home/danielmietchen/.cache/open-access-media-importer/media/raw/pmc_doi/http%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC3832380%2Fbin%2Fpone.0080500.s007.mpeg”.