Add option to convert Ugoira to Webm

[ghost]

I saw that there is an option to convert them to .gif or .apng but would it be possible to have an option to convert them into webm since they are basically just a video of a few frames (plus it will make it easier to watch/manipulate and will be lighter).

[Nandaka]

so far I cannot find webm lib for python without executing 3rd party executables (e.g. https://pypi.python.org/pypi/webm require you to install the dependencies separately).

[ghost]

if it's possible to call a command-line software through python you could just make a call to ffmpeg I think.

Edit : I have found that https://pypi.python.org/pypi/ffmpy maybe it's what you were seeking.

[Nandaka]

hmm, you still need to download the ffmpeg.

technically the current lib that I use (imageio) can output to mp4/mkv using h264 codec (ffmpeg requried)

[awiebe]

The question is how to provide the input to the command line software. Right now pixiv util just makes a zip of jpeg and a .js as I understand it, but ffmpeg wants a directory of images and a frame rate.

Ugoira is a zip of frames and a .js that describes the playback speed of those images.
So you would have to

1. Choose a frame rate
2. Copy zip contents to temporary directory
3. Clone the missing frames since ugoira is programatically timed
4. ffmpeg frames together into a video

I think PixivUtil should probably have the option to call a tool which does this, but I'm not sure the logic for doing this belongs in PixivUtil.

One of these two could probably be used as an extension to pixivutil, or you could just copy bits of them in wholesale if their licenses allow. https://github.com/tsudoko/ugoira-tools/blob/master/ugoira2webm/ugoira2webm.py https://github.com/mikf/ugoira-conv/blob/master/ugoira_conv/ugoira.py

[AvocadoFlour]

The option to have it this way should be implemented in any case, as WEBM wouldn't eat disk space nearly as much as .gif does.

[Nandaka]

tried some of the sample script, it will require you to manually download the ffmpeg.exe manually and it will take some times to convert the images to webm. Need to modify it as it target python 3.

It does generate smaller file size. Is there any image viewer that support webm?

[ghost]

Webm are basically videos so you can use a video player to see them. Otherwise there is Irfanview for Windows or Gthumb image viewer for Linux that are mainly used as image viewer that can also view .webm

[Evaia]

WebM are typically encoded with VP8, VP9 or H.264 codecs, for all intents and purposes you can think of it as a .mp4 file.

[momoe]

I too support converting ugoira to mp4 or webm format. Scanning briefly I found al ittle bit of python code, that can hopefully be used to further this endevor. https://github.com/senko/python-video-converter I really hope this gets implemented as it would greatly help save a lot of disk space.

[Nandaka]

Try https://github.com/Nandaka/PixivUtil2/releases/tag/v20180224-beta2

you need to download FFmpeg and put it into the application folder (or configure the path in config.ini)

[momoe]

Currently testing beta2 in Linux and have ffmeg v1:3.4.2-1 installed. I tried pointing the config.ini towards the directory and tested a URL giving me the following "Missing or No Directory"

I'm going to try and see if I can compile a local ffmpeg in the beta directory....

config.ini settings: createugoira = True createwebm = True

[FFmpeg] ffmpeg = /usr/bin/ffmpeg ffmpegcodec = libvpx-vp9 ffmpegparam = -lossless 1

results:

....ugoira 
Traceback (most recent call last):
  File "PixivUtil2.py", line 812, in process_image
    __config__.ffmpegParam)
  File "/home/momoe/PixivUtil2-20180224-beta2/PixivHelper.py", line 809, in ugoira2webm
    p = subprocess.Popen(cmd, stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 390, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1025, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
Error at process_image(): (<type 'exceptions.OSError'>, OSError(2, 'No such file or directory'), <traceback object at 0x7f5e0802f518>) 
Dumping html to: Error medium page for image 66406775.html 
Traceback (most recent call last):
  File "PixivUtil2.py", line 2033, in main
    np_is_valid, op_is_valid, selection = main_loop(ewd, op_is_valid, selection, np_is_valid, args)
  File "PixivUtil2.py", line 1793, in main_loop
    menu_download_by_image_id(op_is_valid, args)
  File "PixivUtil2.py", line 1463, in menu_download_by_image_id
    process_image(None, int(image_id))
  File "PixivUtil2.py", line 812, in process_image
    __config__.ffmpegParam)
  File "/home/momoe/PixivUtil2-20180224-beta2/PixivHelper.py", line 809, in ugoira2webm
    p = subprocess.Popen(cmd, stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 390, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1025, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

follow up compiled and pasted the files into the beta folder resulted in the same error.

~/PixivUtil2-20180224-beta2/
    ffmpeg_g
    ffplay_g
    ffplay
    ffprobe_g
    ffprobe
    ffmpeg

[Nandaka]

if possible, try to modify the source code so it print the cmd? https://github.com/Nandaka/PixivUtil2/commit/3de601307758294cb30ee0e96d8873c513996bbc#diff-0ba8e05dd3dea6faaf75541a3ed46a24R808

[Nandaka]

also try to set ffmpeg = ./usr/bin/ffmpeg or ffmpeg = ./ffmpeg ?

[momoe]

I don't know how to modify the source code to print the cmd, but I have attempted ffmpeg = ./ffmpeg since I've downloaded the git clone into the subdir and compiled it there. It looks like it's skipping over the process and not reporting any success /failure. No webm or video file is created.

Side note my system has ffmpeg installed in the root directory /usr/bin/ to call upon, I'm not sure how to explicitly tell pixiv to look there if it doesn't already call upon it with the regular ffmpeg command

Update I've tried both of those directories and still the childprocess ugorial2webm isn't creating any webm file. Not erorrs reported just doesn't seem to process anything.

[Nandaka]

Updated the python script to shows the command when checking ffmpeg binary. It should be visible on when the script is started, and also logged.

Example:

PixivDownloader2 version 20180224-beta2
https://github.com/Nandaka/PixivUtil2/releases
Donate at https://bit.ly/PixivUtilDonation
Reading D:\PixivUtil\config.ini ...
done.
Using default DB Path: D:\PixivUtil\db.sqlite
Creating database... done.
Importing list... Found 3522 items done.
Updated 3522 items.
################################################################################
Failed to load ffmpeg, createWebm disabled: [Error 2] The system cannot find the file specified
Command used: ffmpeg -encoders.
Please download ffmpeg with libvpx-vp9 encoder enabled.
################################################################################

[Nandaka]

If you compile your own binary, ensure it support libvpx-vp9.

Here is the output from executing ffmpeg -encoders

D:\PixivUtil> .\ffmpeg.exe -encoders | grep libvpx-vp9
ffmpeg version N-90077-g56f77b0f67 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
  libavutil      56.  7.101 / 56.  7.101
  libavcodec     58. 11.101 / 58. 11.101
  libavformat    58.  9.100 / 58.  9.100
  libavdevice    58.  1.100 / 58.  1.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
 V..... libvpx-vp9           libvpx VP9 (codec vp9)

[momoe]

Okay will test, and give feedback

Update I have compiled ffmpeg --enable-libvpx and have placed the ffmpeg executible in /home/momoe/PixivUtil2-20180227-beta3. I have also reset ffmpeg = ffmpeg. The resulting python response is:

Failed to load ffmpeg, createWebm disabled: [Errno 2] No such file or directory 
Command used: ffmpeg -encoders. 
Please download ffmpeg with libvpx-vp9 encoder enabled. 
################################################################################ 

[WhiteSp1rit]

On Windows, the ffmpeg file would be located in a bin directory by default. Do I need to include it in the path, so that it is would be a direct path to an .exe, or do I write the path just to a ffmpeg directory?

[Nandaka]

if the ffmpeg is not the PATH variable, then you need to supply the full path.

usually I just put the executable to the same folder with the pixivutil.

@momoe: try to set ffmpeg = .\ffmpeg ? The script will try to get the list of encoders using ffmpeg -encoders

[momoe]

@Nandaka Yes, I have attempted to provide the full path (starting with the linux root directory '/bin/') to the ffmpeg offically installed in my linux system which is located in /bin/

I can confirm it's location with the following terminal command...

[momoe@momoe-pc ~]$ /bin/ffmpeg
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-shared --enable-version3 --enable-omx
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

However setting ffmpeg = /bin/ffmpeg results in a failure...

PixivDownloader2 version 20180227-beta3
https://github.com/Nandaka/PixivUtil2/releases
Donate at https://bit.ly/PixivUtilDonation
Reading /home/momoe/PixivUtil2-20180227-beta3/config.ini ...
done.
Using default DB Path: /home/momoe/PixivUtil2-20180227-beta3/db.sqlite 
Creating database... done.
Only process members where the last update is >= 7 days ago 
################################################################################ 
Failed to load ffmpeg, createWebm disabled: [Errno 2] No such file or directory 
Command used: /bin/ffmpeg -encoders. 
Please download ffmpeg with libvpx-vp9 encoder enabled. 
################################################################################  ```

[Nandaka]

Might be related: https://stackoverflow.com/a/29775100, https://stackoverflow.com/a/901992

Can you modify https://github.com/Nandaka/PixivUtil2/blob/master/PixivUtil2.py#L1992

import shlex
cmd = u"{0} -encoders".format(__config__.ffmpeg)
ffmpeg_args = shlex.split(cmd)

and https://github.com/Nandaka/PixivUtil2/blob/master/PixivUtil2.py#L1995

p = subprocess.Popen(ffmpeg_args, stderr=subprocess.PIPE, stdout=subprocess.PIPE)

and https://github.com/Nandaka/PixivUtil2/blob/master/PixivHelper.py#L809

import shlex
cmd = cmd.format(ffmpeg, d, codec, param, tempname)
ffmpeg_args = shlex.split(cmd)
p = subprocess.Popen(ffmpeg_args, stderr=subprocess.PIPE)

[vampiricwulf]

This is great and works perfectly for me, I just wish there's a way for this to go through my list.txt and replace all ugoira apng with webm instead of having it "redownload" basically everything to check. 😢

[momoe]

I will try this and report back.

[momoe@momoe-pc PixivUtil2-20180227-beta3]$ python2 PixivUtil2.py 
PixivDownloader2 version 20180227-beta3
https://github.com/Nandaka/PixivUtil2/releases
Donate at https://bit.ly/PixivUtilDonation
Reading /home/momoe/PixivUtil2-20180227-beta3/config.ini ...
done.
Using default DB Path: /home/momoe/PixivUtil2-20180227-beta3/db.sqlite 
Creating database... done.
Only process members where the last update is >= 7 days ago 
################################################################################ 
Failed to load ffmpeg, createWebm disabled: global name 'ffmpeg_args' is not defined 
Command used: /bin/ffmpeg -encoders. 
Please download ffmpeg with libvpx-vp9 encoder enabled. 
################################################################################ 

[Nandaka]

Get the latest commit and try again? I've update the script.

@vampiricwulf I think if overwrite is disabled but alwaysCheckFileSize is enabled, it will try to convert existing .ugoira file to webm (assuming createWebm is enabled).

[momoe]

Tested and success! Current config.ini is

writeugoirainfo = True
createugoira = True
createwebm = True
[FFmpeg]
ffmpeg = ffmpeg
ffmpegcodec = libvpx-vp9
ffmpegparam = -lossless 1

Thank you for the the time and effort with this. Will be comitting to a donation soon!

Followup Is it fine to set deleteugoira = True to cleanup the leftover file?

[Nandaka]

Yes, as the image id should be recorded to the DB (assuming alwaysCheckFileSize = False).

I'm not sure how to handle alwaysCheckFileSize = True if the original ugoira file is deleted, as I cannot compare the server .zip file size with the webm file size. https://github.com/Nandaka/PixivUtil2/blob/master/PixivUtil2.py#L140

[Evaia]

Option was working, has stopped working recently: using PixivUtil20180426-beta1 Download of Ugoria succeeds, but when it goes to call ffmpeg it fails with:

  File "PixivUtil2.py", line 794, in process_image
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Error at process_image(): (<type 'exceptions.TypeError'>, TypeError("unsupported operand type(s) for +: 'NoneType' and 'str'",), <traceback object at 0x06124B20>)
Dumping html to: Error medium page for image 68268490.html
Stuff happened, trying again after 2 second ( 1 )
Traceback (most recent call last):
  File "PixivUtil2.py", line 511, in process_member
  File "PixivUtil2.py", line 794, in process_image
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Processing Image Id: 68268490

config file entry is:

ffmpeg = "C:\Program Files\ffmpeg\bin\ffmpeg.exe"
ffmpegcodec = libvpx-vp9
ffmpegparam = -lossless 1

[Nandaka]

Weird, it is working on my side:

Input: 2
Image ids: 68268490
Processing Image Id: 68268490
Member Name  : ザンクロー
Member Avatar: https://i.pximg.net/user-profile/img/2016/08/03/22/28/14/11292225_6dc8c08cad2c77231cd51dfb111ebfa1.jpg
Member Token : zankuro
Title: 動く子エッダ
Tags : R-18, うごイラ, 騎乗位, 乳揺れ, オリジナル, 主観視点
Date : 2018-04-16 15:45:47+00:00
Mode : ugoira_view
Image URL : https://i.pximg.net/img-zip-ugoira/img/2018/04/17/00/45/47/68268490_ugoira1920x1080.zip
Filename  : D:\Temp\Pixiv2\7964 zankuro\R-1820180416 68268490_ugoira1920x1080 - 動く子エッダ.zip
Start downloading... Using Referer: https://www.pixiv.net/member_illust.php?mode=medium&illust_id=68268490
Creating directory: D:\Temp\Pixiv2\7964 zankuro
 [||||||||||||||||||||] 414.69 KiB of 414.69 KiB  Completed in 1.871s (221.64 KiB/s)
 Image verified.

Creating ugoira archive => D:\Temp\Pixiv2\7964 zankuro\R-1820180416 68268490_ugoira1920x1080 - 動く子エッダ.ugoira
Deleting zip file => D:\Temp\Pixiv2\7964 zankuro\R-1820180416 68268490_ugoira1920x1080 - 動く子エッダ.zip
Start encoding D:\Temp\Pixiv2\7964 zankuro\R-1820180416 68268490_ugoira1920x1080 - 動く子エッダ.webm
frame=    5 fps=1.5 q=0.0 Lsize=    1099kB time=00:00:00.32 bitrate=28054.8kbits/s speed=0.0993x
 done with status= 0

can you upload the Error medium page for image 68268490.html file?

[Evaia]

Error medium page for image 68268490.zip Attached.

[Nandaka]

I see you still using the old ui, try the previous stable version https://github.com/Nandaka/PixivUtil2/releases/tag/v20180414

You can use the beta releases when Pixiv move you to the new ui (#355)

[Evaia]

Same issue with 20180414

  File "PixivUtil2.py", line 792, in process_image
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Error at process_image(): (<type 'exceptions.TypeError'>, TypeError("unsupported operand type(s) for +: 'NoneType' and 'str'",), <traceback object at 0x06039B48>)
Dumping html to: Error medium page for image 68268490.html
Traceback (most recent call last):
  File "PixivUtil2.py", line 2063, in main
  File "PixivUtil2.py", line 1816, in main_loop
  File "PixivUtil2.py", line 1466, in menu_download_by_image_id
  File "PixivUtil2.py", line 792, in process_image
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Error medium page for image 68268490.zip

edit: not the exact same issue, but still non-functional.

[Nandaka]

hmm, weird, it shows the error when it try to concatenate the filename: https://github.com/Nandaka/PixivUtil2/blob/v20180414/PixivUtil2.py#L792

What is the generated filename? e.g. Filename : D:\Temp\Pixiv2\7964 zankuro\R-1820180416 68268490_ugoira1920x1080 - 動く子エッダ.zip

[Evaia]

D:\StorageArchive\Temp\Pixiv\ザンクロー (7964)\68268490_ugoira1920x1080 - 動く子エッダ.zip

[Evaia]

I pointed it to D:\Temp\ and it works now. Perhaps it was having an issue with the number of characters? Edit: Spoke too soon, it worked once then broke again.

[Nandaka]

can you delete the pixivutil.log and replicate the issue, then upload the log file here?

[Evaia]

pixivutil.log

[Nandaka]

if you are running from source, can you try the new commit bd2b31a?

[Evaia]

I am not running from source.

[Nandaka]

try https://github.com/Nandaka/PixivUtil2/releases/tag/v20180503-beta2

[Evaia]

Processing Image Id: 68268490
Title: 動く子エッダ
Tags : R-18, うごイラ, 騎乗位, 乳揺れ, オリジナル, 主観視点
Date : 2018-04-17 00:45:00
Mode : ugoira_view
Image URL : https://i.pximg.net/img-zip-ugoira/img/2018/04/17/00/45/47/68268490_ugoira1920x1080.zip
Filename  : D:\StorageArchive\Temp\Pixiv\(7964) ザンクロー\68268490_ugoira1920x1080 - 動く子エッダ.zip
Getting remote filesize...
Using Referer: https://www.pixiv.net/member_illust.php?mode=medium&illust_id=68268490
Remote filesize = 414.69 KiB (424645 Bytes)
        File exist! (Identical Size)

Traceback (most recent call last):
  File "PixivUtil2.py", line 797, in process_image
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Error at process_image(): 68268490
Exception: (<type 'exceptions.TypeError'>, TypeError("unsupported operand type(s) for +: 'NoneType' and 'str'",), <traceback object at 0x0630A918>)
Dumping html to: Error medium page for image 68268490.html
Stuff happened, trying again after 2 second ( 1 )
Traceback (most recent call last):
  File "PixivUtil2.py", line 515, in process_member
  File "PixivUtil2.py", line 797, in process_image
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Error medium page for image 68268490.zip pixivutil.log

[Nandaka]

got the root cause, for now try to set writeUgoiraInfo=False in the config.ini for the time being. I'll upload the new version later.

[Evaia]

I can confirm, that fixes it for me. Much appreciated, Nandaka.

[whinette]

I have two problems: pixivutil can't write in /tmp but the user it is launched with can. I did chmod and chgrp to the current user and it still can't.

Input: 2
Image ids: 49739905
Processing Image Id: 49739905
Member Name  : 古い
Member Avatar: https://i.pximg.net/user-profile/img/2015/11/11/17/41/03/10103374_7d63643853f2e296dcc57c61774179d8_170.png
Member Token : hurui91
Title: 無題
Tags : うごイラ, 東方, MMD, 古明地こいし, 眼鏡, 東方うごイラ
Date : 2015-04-08 15:12:50+00:00
Mode : ugoira_view
Image URL : https://i.pximg.net/img-zip-ugoira/img/2015/04/09/00/12/50/49739905_ugoira1920x1080.zip
Filename  : /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.zip
Start downloading... Using Referer: https://www.pixiv.net/member_illust.php?mode=medium&illust_id=49739905
 [||||||||||||||||||||] 7.86 MiB of 7.86 MiB  Completed in 3.572906s (2.20 MiB/s)
 done.

Creating ugoira archive => /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.ugoira
Deleting zip file => /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.zip
Start encoding /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.webm
Traceback (most recent call last):
  File "PixivUtil2.py", line 822, in process_image
    __config__.ffmpegParam)
  File "/mnt/c/PixivUtil2_manual/PixivHelper.py", line 835, in ugoira2webm
    shutil.move(tempname, exportname)
  File "/usr/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/tmp/ugoira2webmLCAdcY/temp.webm'
Error at process_image(): 49739905
Exception: (<type 'exceptions.IOError'>, IOError(2, 'No such file or directory'), <traceback object at 0x7fadf0db8f80>)
Dumping html to: Error medium page for image 49739905.html
Traceback (most recent call last):
  File "PixivUtil2.py", line 2068, in main
    np_is_valid, op_is_valid, selection = main_loop(ewd, op_is_valid, selection, np_is_valid, args)
  File "PixivUtil2.py", line 1822, in main_loop
    menu_download_by_image_id(op_is_valid, args)
  File "PixivUtil2.py", line 1468, in menu_download_by_image_id
    process_image(None, int(image_id))
  File "PixivUtil2.py", line 822, in process_image
    __config__.ffmpegParam)
  File "/mnt/c/PixivUtil2_manual/PixivHelper.py", line 835, in ugoira2webm
    shutil.move(tempname, exportname)
  File "/usr/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/tmp/ugoira2webmLCAdcY/temp.webm'
press enter to exit.

The second problem is on redownload of already present ugoira. It will loop without being able to download the zip. When I delete the .ugoira it "works" (back to problem one). This seems to be the same problem as the first one but the stacktrace is bizarre (Error at download_image(): (<type 'exceptions.IOError'>, IOError(2, 'No such file or directory'), <traceback object at 0x7fe34eb19e60>) at https://i.pximg.net/img-zip-ugoira/img/2015/04/09/00/12/50/49739905_ugoira1920x1080.zip (9001))

Input: 2
Image ids: 49739905
Processing Image Id: 49739905
Member Name  : 古い
Member Avatar: https://i.pximg.net/user-profile/img/2015/11/11/17/41/03/10103374_7d63643853f2e296dcc57c61774179d8_170.png
Member Token : hurui91
Title: 無題
Tags : うごイラ, 東方, MMD, 古明地こいし, 眼鏡, 東方うごイラ
Date : 2015-04-08 15:12:50+00:00
Mode : ugoira_view
Image URL : https://i.pximg.net/img-zip-ugoira/img/2015/04/09/00/12/50/49739905_ugoira1920x1080.zip
Filename  : /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.zip
Getting remote filesize...
Using Referer: https://www.pixiv.net/member_illust.php?mode=medium&illust_id=49739905
Remote filesize = 7.86 MiB (8238694 Bytes)
        File exist! (Identical Size)
Start encoding /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.webm
Traceback (most recent call last):
  File "PixivUtil2.py", line 176, in download_image
    __config__.ffmpegParam)
  File "/mnt/c/PixivUtil2_manual/PixivHelper.py", line 835, in ugoira2webm
    shutil.move(tempname, exportname)
  File "/usr/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/tmp/ugoira2webm5k4Ci_/temp.webm'
Error at download_image(): (<type 'exceptions.IOError'>, IOError(2, 'No such file or directory'), <traceback object at 0x7fe34eb19e60>) at https://i.pximg.net/img-zip-ugoira/img/2015/04/09/00/12/50/49739905_ugoira1920x1080.zip (9001)
Retrying [1]... 1 2 3 4 5 6 7 8 9
Getting remote filesize...
Using Referer: https://www.pixiv.net/member_illust.php?mode=medium&illust_id=49739905
Remote filesize = 7.86 MiB (8238694 Bytes)
        File exist! (Identical Size)
Start encoding /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.webm
Traceback (most recent call last):
  File "PixivUtil2.py", line 176, in download_image
    __config__.ffmpegParam)
  File "/mnt/c/PixivUtil2_manual/PixivHelper.py", line 835, in ugoira2webm
    shutil.move(tempname, exportname)
  File "/usr/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/tmp/ugoira2webmQ_MF7l/temp.webm'
Error at download_image(): (<type 'exceptions.IOError'>, IOError(2, 'No such file or directory'), <traceback object at 0x7fe34ecd0710>) at https://i.pximg.net/img-zip-ugoira/img/2015/04/09/00/12/50/49739905_ugoira1920x1080.zip (9001)

Related settings are as bellow:

writeugoirainfo = False
createugoira = True
deletezipfile = True
enableinfiniteloop = False
verifyimage = False
writeurlindescription = True

creategif = False
useblacklistmembers = False
createapng = False
deleteugoira = True
createwebm = True

[FFmpeg]
ffmpeg = /mnt/c/ffmpeg-4.0-win64-static/bin/ffmpeg.exe
ffmpegcodec = libvpx-vp9
ffmpegparam = -lossless 1

Thanks in advance @Nandaka

[Nandaka]

can you check if the /tmp/ugoira2webm<random>/ folder is created after zip/ugoira file is downloaded?

also try to modify: shutil.move(tempname, exportname) in https://github.com/Nandaka/PixivUtil2/blob/master/PixivHelper.py#L835

to shutil.move(os.path.abspath(tempname), exportname) ?

[whinette]

It is not.

[Nandaka]

can you remove/comment out this line https://github.com/Nandaka/PixivUtil2/blob/master/PixivHelper.py#L844 https://github.com/Nandaka/PixivUtil2/blob/master/PixivHelper.py#L845

and check if the tmp folder is generated?

[whinette]

Folder are created with rights 700. There are jpg, animation.json and i.ffconcat inside them with rights 666.

I've put some debug, in the loop while p.strderr: It enters the first if but never in chatter.find("frame=") > 0:

Input: 2
Image ids: 49739905
Processing Image Id: 49739905
Member Name  : 古い
Member Avatar: https://i.pximg.net/user-profile/img/2015/11/11/17/41/03/10103374_7d63643853f2e296dcc57c61774179d8_170.png
Member Token : hurui91
Title: 無題
Tags : うごイラ, 東方, MMD, 古明地こいし, 眼鏡, 東方うごイラ
Date : 2015-04-08 15:12:50+00:00
Mode : ugoira_view
Image URL : https://i.pximg.net/img-zip-ugoira/img/2015/04/09/00/12/50/49739905_ugoira1920x1080.zip
Filename  : /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.zip
Getting remote filesize...
Using Referer: https://www.pixiv.net/member_illust.php?mode=medium&illust_id=49739905
Remote filesize = 7.86 MiB (8238694 Bytes)
        File exist! (Identical Size)
--------------- /tmp/ugoira2webmOl48u6
Start encoding /mnt/z/Pixiv/13575474 hurui91/古い _/49739905_ugoira1920x1080 - 無題.webm
print chatter:
ffmpeg version 4.0 Copyright (c) 2000-2018 the FFmpeg developers
print chatter:

  built with gcc 7.3.0 (GCC)
print chatter:

  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
print chatter:

  libavutil      56. 14.100 / 56. 14.100
print chatter:

  libavcodec     58. 18.100 / 58. 18.100
print chatter:

  libavformat    58. 12.100 / 58. 12.100
print chatter:

  libavdevice    58.  3.100 / 58.  3.100
print chatter:

  libavfilter     7. 16.100 /  7. 16.100
print chatter:

  libswscale      5.  1.100 /  5.  1.100
print chatter:

  libswresample   3.  1.100 /  3.  1.100
print chatter:

  libpostproc    55.  1.100 / 55.  1.100
print chatter:

/tmp/ugoira2webmOl48u6/i.ffconcat: No such file or directory
Traceback (most recent call last):
  File "PixivUtil2.py", line 176, in download_image
    __config__.ffmpegParam)
  File "/mnt/c/PixivUtil2_manual/PixivHelper.py", line 839, in ugoira2webm
    shutil.move(tempname, exportname)
  File "/usr/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/usr/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/tmp/ugoira2webmOl48u6/temp.webm'
Error at download_image(): (<type 'exceptions.IOError'>, IOError(2, 'No such file or directory'), <traceback object at 0x7fc32218a3f8>) at https://i.pximg.net/img-zip-ugoira/img/2015/04/09/00/12/50/49739905_ugoira1920x1080.zip (9001)

whinette@orin:/tmp/ugoira2webmOl48u6$ ls -l /tmp/ugoira2webmOl48u6/i.ffconcat
-rw-rw-rw- 1 whinette whinette 3771 Jun 19 09:12 /tmp/ugoira2webmOl48u6/i.ffconcat

Maybe a race condition? @ subprocess

[Nandaka]

try to comment out line https://github.com/Nandaka/PixivUtil2/blob/master/PixivHelper.py#L822 to https://github.com/Nandaka/PixivUtil2/blob/master/PixivHelper.py#L832 and see the result?

by right it should wait until the encoding is finished. https://github.com/Nandaka/PixivUtil2/blob/master/PixivHelper.py#L834

also, can you do ffmpeg.exe -encoders | grep libvpx-vp9 and see if the codec is supported?

[whinette]

ffmpeg.exe -encoders did not produced any output ... I reinstalled the binary and it works as expected. Sorry and thanks for the help. :)

[Ridolf]

Traceback (most recent call last): File "PixivUtil2.py", line 176, in download_image File "PixivHelper.pyc", line 850, in ugoira2webm File "shutil.pyc", line 302, in move File "shutil.pyc", line 130, in copy2 File "shutil.pyc", line 82, in copyfile IOError: [Errno 2] No such file or directory: 'c:/users/x/appdata/local/temp/ugoira2webmv4xu3i/temp.webm' Error at download_image(): (<type 'exceptions.IOError'>, IOError(2, 'No such file or directory'), <traceback object at 0x03E3C490>) at https://i.pximg.net/img-zip-ugoira/img/2018/09/19/01/45/31/70766086_ugoira1920x1080.zip (9001)

I need help