Getting PhantomJS not found "Error while fetching metadata!"

[ifsandbuts]

Hey, thank you for your efforts with this tool, I've added a decent list of subscriptions to ph and was going well 1st few days but recently getting this error and am lucky to get 1 video downloaded for every new 100+ video sub, the rest give this error.

Is this a throttling issue where I'm getting detected as a bot now?

ERROR: [PornHub] ph60223d895152b: PhantomJS not found, Please download it from https://phantomjs.org/download.html ERROR:project_functions:Error while fetching metadata! - Type Error: object of type 'NoneType' has no len() ERROR:project_functions:Error while fetching metadata! - Skip item First Time With Thick Girlfriend - Electra Rayne - Perfect Girlfriend DEBUG:project_functions:Scheme file for pornhub found DEBUG:project_functions:Check if provided url pornhub is valid for scheme

[ifsandbuts]

Also previously I've noticed an issue with it timing out and skipping some fragments of some videos, when I run a "validate" command it doesn't seem to be checking for these missing fragments, it says to "check log" where can I find? I'd like to be able to do a hashcheck of all my subscription videos downloaded so far to make sure no missing fragments/videos and where I can see the error log?. Thank you again

[ifsandbuts]

1 video of 386 was downloaded for this recent sub

ERROR: [PornHub] 1856521823: PhantomJS not found, Please download it from https://phantomjs.org/download.html ERROR:project_functions:Error while fetching metadata! - Type Error: object of type 'NoneType' has no len() ERROR:project_functions:Error while fetching metadata! - Skip item Charlee Chase Step Mom Needs More DEBUG:root:Key downloaded_content_count is a str DEBUG:database_manager:Prepared Query: UPDATE subscriptions SET downloaded_content_count= ? WHERE subscription_name="alex-adams"; DEBUG:database_manager:Data ['1'] ERROR:project_functions:Failed while downloading file for subscription alex-adams ERROR:project_functions:Subscription: alex-adams

[ifsandbuts]

[download] Got error: [SSL: SSLV3_ALERT_BAD_RECORD_MAC] sslv3 alert bad record mac (_ssl.c:2559) [download] fragment not found; Skipping fragment 23 ...

[download] Got error: HTTPSConnectionPool(host='ev-h.phncdn.com', port=443): Read timed out. (read timeout=20.0) [download] fragment not found; Skipping fragment 16 ... [download] Got error: HTTPSConnectionPool(host='ev-h.phncdn.com', port=443): Read timed out. (read timeout=20.0) [download] fragment not found; Skipping fragment 58 ...

[download] Got error: HTTP Error 502: Bad Gateway TA 00:48 (frag 74/204)ERROR: [download] fragment not found; Skipping fragment 75 ... [download] 47.2% of ~ 323.50MiB at 4.88MiB/s ETA 00:35 (frag 96/204)

[j54j6]

Hey,

thank you for the provided details. First things first -> Currently all logs are sended to standard out (Console). I'm planning on adding another config entry to set a file but at the time it is really hard to update this...

Regarding your other errors: PhantomJS is a part/extension lib for yt-dlp (Youtube-DL).YT-DLP is handling all the downloading stuff. You can influence it by modifying the yt_dls configuration function. My script only adds the registry and checking functionality alongside with the correct paths to save the files. The fragment errors are caused by different things. There are some ways to fix this e.g. extending the timeout time but you need to modify the code for thos (dont know if you can).

I will take a closer look on this ASAP :)

[ifsandbuts]

thank you for your reply, the timeout error maybe happening more frequently to me because of the pc i've been using I think freezes up once in awhile and doesn't manage to unfreeze itself and resume the frag its on within 20 seconds and then times out moves to next. This has happened more then I'd like unfortunately I hoped the script was keeping track of which videos I maybe missing fragments for and "validate/recheck".

This tool would be next level really amazing if you recreated the html paths for an offline UI view where you can how more control over your youtube/pornhub subscriptions.

Is there a way I can put a batch list of subscriptions/channels like I do for subscriptions but to export the metadata for example models on pornhub if I want to extract just their page bios ie age sex orientation size etc and channel stats like how many videos views & likes per video original upload date etc. basically all the metadata for the channel bio itself as well as videos but without downloading the videos. Thanks again

[ifsandbuts]

Quick update, it seems that the errors are related to the pornstars, the script is getting confused with the upload vs tagged maybe? I started a new folder/file added batch of pornstar pages and kept getting errors, when checking pages for total videos it was finding 700+ videos on stars who had no more then 200 even combining uploaded and tagged.

I just added a batch of models and seems to be running smoothly again.

Main priority Metadata Export/UI aside, would be ability to do hashcheck/validate frags for downloaded videos. I have 300+ model pages so far but many of those videos are likely missing frags I'd like to know which ones. Thanks

[j54j6]

Hey,

in fact it is planned that a web interface is added and also a standard web player. Mainly the UI will be used to edit metadata iof all files. Regarding your fragments. I need to check if this is possible since there are some problems getting the "initial" hash correct if you missed fragments. YT-DLP doesn't return an error if there are missing fragments as far as I can see it. Maybe I overlooked something but if I can find a way to implement a check from the source (fetch the fragments and compare it against something for example) than I will definitely add it :)

[ifsandbuts]

This applies for youtube channels also, I think if you could scrape the user profile/videos list with all its metadata including original upload date etc and can view your subscriptions in a personal offline html view where you can then do your own ratings notes etc.

I would like to download entire channel/user/model validate a hashcheck of each video then from there I can "lock" that subscription on a chosen date so future scans will only check/download new never before downloaded (when confirmed all past videos have been fully downloaded & validated). Then I can delete or chop up those videos leave them in their original folder/name just reducing the available offline length (offline html page still shows any deleted or chopped videos and their original stats/length but instead of any missing video coming from my offline folder it will link to its original video url).

Being able to manage/edit/chop up your own subscriptions offline would be great and save alot of time/data for people especially on subs/channels where there is a lot of duplicates or low quality content/chapters. Quick example on models for example lets say a model has 300 videos but in reality 90% of those videos are low quality not worth ever watching a 2nd time, I want to delete/crop the best parts of the videos to retain offline copy but it refers to the original url/length for any parts I have deleted. This way I don't need to waste $ storing 200gb of videos which I really only want 20gb worth of the minutes.

[j54j6]

Hey,

the metadata are already in your DB ;) - A copy of all metadata are saved into the db during the initial download :) - This includes like count and all other stuff ^^

For the other stuff I will try to implement these features ASAP. Could you create a little list with all your whishes. I will add them as new tasks ^^

[j54j6]

Hey,

I set the requirement to a higer version of yt dlp with some fixes. It works for me now (had similar issues) - Please check ^^