search

c477y / adulttime_dl

Bulk download porn videos
The Unlicense
8 stars 0 forks source link

ArgumentError - wrong number of arguments (given 1, expected 2) #2

Open complex-user opened 11 months ago

complex-user commented 11 months ago

Hello, I am trying to resolve this issue where I'm only getting this error:

/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime
[23:36:16] [INFO ] youtube-dl installed with version 2023.11.30
[23:36:16] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[23:36:16] [INFO ] [PROCESS START]
[23:36:16] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/pornstar/view/Leana-Lovings/91441
[23:36:16] [FATAL] ArgumentError - wrong number of arguments (given 1, expected 2)
/app/adulttime_dl #

My config file:

# [List] Skip studios based on name
# If any scene belongs to that studio, the scene will be skipped
skip_studios: []
# [List] Skip performers based on name
skip_performers: []
# [Boolean] Skip downloading scenes with only female actors
# This only works with certain sites that provide gender information for performers
skip_lesbian: false
# [String] Path to a cookie file. Used only for sites that require membership
# information. Use a browser extension to get a cookie file
# If you are on mozilla, you can use https://addons.mozilla.org/en-GB/firefox/addon/cookies-txt/
cookie_file: cookies.txt
# [String] Name of file that tracks list of downloaded scenes
store: adt_download_status.store
# [String] Name of downloader tool to use. Currently supports youtube-dl
# and yt-dlp
# Ensure that the downloader is available in your $PATH
downloader: youtube-dl
# [String] Directory to download scenes. '.' means current directory
download_dir: "."
# [String] Resolution of scene to download. Accepts one of 'sd', 'hd', and 'fhd'
quality: hd
# [Number] Number of parallel download jobs
parallel: 1
# [Boolean] Enable/Disable verbose logging
verbose: true
# [Boolean] Just log to console, doesn't download anything
dry_run: false
# [String] Arguments to pass to external downloader
# Check the documentation of youtube-dl or yt-dlp for details
# eg. --external-downloader aria2c --external-downloader-args
# "-j 8 -s 8 -x 8 -k 5M"
downloader_flags: ''
urls:
  # [List] URLs of performer names that will be downloaded
  performers: [https://members.adulttime.com/en/pornstar/view/Leana-Lovings/91441]
  # [List] URLs of movie names that will be downloaded
  movies: [https://members.adulttime.com/en/video/howwomenorgasm/Leana-Loves-To-Be-Stimulated/233394]
  # [List] URLs of scene names that will be downloaded
  scenes: []
# [OPTIONAL] These are site specific configuration only needed for some sites
site_config:
  blowpass:
    # Get Algolia credentials from Web Console
    algolia_application_id:
    algolia_api_key:

I'm not sure where I would supply additional arguments?

c477y commented 11 months ago

This probably looks like a bug. Can you run the command again with ‘—verbose’ flag and show me the stack trace of the error?

I can’t test the code myself because I don’t have an active subscription anymore.

complex-user commented 11 months ago 
/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime
[22:41:00] [INFO ] youtube-dl installed with version 2023.11.30
[22:41:00] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[22:41:00] [INFO ] [PROCESS START]
[22:41:00] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/pornstar/view/Leana-Lovings/91441
[22:41:00] [FATAL] ArgumentError - wrong number of arguments (given 1, expected 2)
/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime --verbose
[22:41:09] [INFO ] youtube-dl installed with version 2023.11.30
[22:41:09] [DEBUG] {:site=>"adulttime", :download_filters=>{:skip_studios=>#<Set: {}>, :skip_performers=>#<Set: {}>, :skip_lesbian=>false}, :cookie_file=>"cookies/cookies.txt", :store=>"adt_download_status.store", :downloader=>"youtube-dl", :download_dir=>".", :quality=>"hd", :parallel=>1, :verbose=>true, :dry_run=>false, :downloader_flags=>"", :urls=>{:performers=>"1 items", :movies=>"0 items", :scenes=>"0 items"}, :site_config=>{:blowpass=>{:algolia_application_id=>nil, :algolia_api_key=>nil}}}
[22:41:09] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[22:41:09] [INFO ] [PROCESS START]
[22:41:09] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/pornstar/view/Leana-Lovings/91441
[22:41:09] [FATAL] ArgumentError - wrong number of arguments (given 1, expected 2)
[22:41:09] [DEBUG]      /app/adulttime_dl/lib/adulttime_dl/net/adulttime_index.rb:8:in `search_by_actor'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:34:in `block in process_performer'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:32:in `map'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:32:in `process_performer'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:15:in `start!'
        /app/adulttime_dl/lib/adulttime_dl/cli.rb:34:in `download'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/command.rb:27:in `run'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/invocation.rb:127:in `invoke_command'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor.rb:392:in `dispatch'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/base.rb:485:in `start'
        ./exe/./adulttime_dl:10:in `<main>'
/app/adulttime_dl #
c477y commented 11 months ago

Sorry I've not been able to look into this in depth, but I have a hunch what might be happening. I've pushed a fix in this commit, can you pull the latest main and run the command again? Do send me the stacktrace if you get an error again. I might just need to get a trial account set up to find the issue myself.

c477y commented 11 months ago

Ah this is actually okay, so you need to delete the config file and generate it again (which should be done by running the command without the config file present). Add the URLs back and try again. I need to figure out a way to ensure old configs are still supported.

josecrosa commented 11 months ago

Here's the new trace

/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime --verbose
[19:44:13] [INFO ] youtube-dl installed with version 2023.11.14
[19:44:13] [DEBUG] {:site=>"adulttime", :download_filters=>{:skip_studios=>#<Set: {}>, :skip_performers=>#<Set: {}>, :skip_lesbian=>false, :skip_keywords=>[]}, :cookie_file=>"cookies/cookies.txt", :store=>"adt_download_status.store", :downloader=>"youtube-dl", :download_dir=>".", :quality=>"hd", :parallel=>1, :verbose=>true, :dry_run=>false, :downloader_flags=>"", :urls=>{:performers=>"1 items", :movies=>"1 items", :scenes=>"1 items"}, :site_config=>{:blowpass=>{:algolia_application_id=>nil, :algolia_api_key=>nil}}}
[19:44:13] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[19:44:13] [INFO ] [PROCESS START]
[19:44:13] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/pornstar/view/Leana-Lovings/
[19:44:13] [WARN ] AdultTimeDL::Net::AdultTimeIndex does not implement actor_name
[19:44:13] [FATAL] AdultTimeDL::FatalError - AdultTimeDL::Net::AdultTimeIndex does not implement scenes_index
[19:44:13] [FATAL]      /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:61:in `scenes_index'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:21:in `block in search_by_actor'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:45:in `with_retry'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:19:in `search_by_actor'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:42:in `block in process_performer'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:33:in `map'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:33:in `process_performer'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:15:in `start!'
        /app/adulttime_dl/lib/adulttime_dl/cli.rb:34:in `download'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/command.rb:27:in `run'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/invocation.rb:127:in `invoke_command'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor.rb:392:in `dispatch'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/base.rb:485:in `start'
        ./exe/./adulttime_dl:10:in `<main>'
c477y commented 11 months ago

This is also okay, so some sites support batch downloading based on either list of movies, actors or scene urls. AdultTime doesn't support scenes right now, which is why you are seeing this error. Can you change the scenes list to an empty array and try again? Downloading by actors and movie URLs should work all right (source).

complex-user commented 11 months ago

I don't think we're there yet


/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime --verbose
[22:13:31] [INFO ] youtube-dl installed with version 2023.11.14
[22:13:31] [DEBUG] {:site=>"adulttime", :download_filters=>{:skip_studios=>#<Set: {}>, :skip_performers=>#<Set: {}>, :skip_lesbian=>false, :skip_keywords=>[]}, :cookie_file=>"cookies/cookies.txt", :store=>"adt_download_status.store", :downloader=>"youtube-dl", :download_dir=>".", :quality=>"hd", :parallel=>1, :verbose=>true, :dry_run=>false, :downloader_flags=>"", :urls=>{:performers=>"1 items", :movies=>"1 items", :scenes=>"0 items"}, :site_config=>{:blowpass=>{:algolia_application_id=>nil, :algolia_api_key=>nil}}}
[22:13:31] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[22:13:31] [INFO ] [PROCESS START]
[22:13:31] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/pornstar/view/Leana-Lovings/
[22:13:31] [WARN ] AdultTimeDL::Net::AdultTimeIndex does not implement actor_name
[22:13:31] [FATAL] AdultTimeDL::FatalError - AdultTimeDL::Net::AdultTimeIndex does not implement scenes_index
[22:13:31] [FATAL]      /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:61:in `scenes_index'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:21:in `block in search_by_actor'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:45:in `with_retry'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:19:in `search_by_actor'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:42:in `block in process_performer'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:33:in `map'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:33:in `process_performer'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:15:in `start!'
        /app/adulttime_dl/lib/adulttime_dl/cli.rb:34:in `download'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/command.rb:27:in `run'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/invocation.rb:127:in `invoke_command'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor.rb:392:in `dispatch'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/base.rb:485:in `start'
        ./exe/./adulttime_dl:10:in `<main>'
/app/adulttime_dl # cat config/config.yaml 
# [List] Skip studios based on name
# If any scene belongs to that studio, the scene will be skipped
skip_studios: []
# [List] Skip performers based on name
skip_performers: []
# [Boolean] Skip downloading scenes with only female actors
# This only works with certain sites that provide gender information for performers
skip_lesbian: false
skip_keywords: []
# [String] Path to a cookie file. Used only for sites that require membership
# information. Use a browser extension to get a cookie file
# If you are on mozilla, you can use https://addons.mozilla.org/en-GB/firefox/addon/cookies-txt/
cookie_file: cookies.txt
# [String] Name of file that tracks list of downloaded scenes
store: adt_download_status.store
# [String] Name of downloader tool to use. Currently supports youtube-dl
# and yt-dlp
# Ensure that the downloader is available in your $PATH
downloader: youtube-dl
# [String] Directory to download scenes. '.' means current directory
download_dir: "."
# [String] Resolution of scene to download. Accepts one of 'sd', 'hd', and 'fhd'
quality: hd
# [Number] Number of parallel download jobs
parallel: 1
# [Boolean] Enable/Disable verbose logging
verbose: true
# [Boolean] Just log to console, doesn't download anything
dry_run: false
# [String] Arguments to pass to external downloader
# Check the documentation of youtube-dl or yt-dlp for details
# eg. --external-downloader aria2c --external-downloader-args
# "-j 8 -s 8 -x 8 -k 5M"
downloader_flags: ''
urls:
  # [List] URLs of performer names that will be downloaded
  performers: [https://members.adulttime.com/en/pornstar/view/Leana-Lovings/]
  # [List] URLs of movie names that will be downloaded
  movies: [https://members.adulttime.com/en/video/getupclose/Up-Close-with-Leana-Lovings/233445]
  # [List] URLs of scene names that will be downloaded
  scenes: []
# [OPTIONAL] These are site specific configuration only needed for some sites
site_config:
  blowpass:
    # Get Algolia credentials from Web Console
    algolia_application_id:
    algolia_api_key:
/app/adulttime_dl #
complex-user commented 11 months ago

Bump

c477y commented 11 months ago

Since the last time I wrote the script, AdultTime has changed a lot of their underlying APIs (also I made a lot of changed :p). Nevertheless, I think we may finally have a working version. Can you pull in the latest master and try again please? Hopefully it should work this time. I also added support for downloading scenes via URLs.

complex-user commented 11 months ago

EDIT: My mistake, old code in docker container. Better results next post.

Hmmm same still. Some outputs:

user1@server1 adulttime-dl % docker run -it  -v $(pwd)/config:/app/adulttime_dl/config   -v $(pwd)/cookies:/app/adulttime_dl/cookies -- adulttime-dl:test-7                                                           
/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime --verbose
[19:51:36] [INFO ] youtube-dl installed with version 2023.11.14
[19:51:36] [DEBUG] {:site=>"adulttime", :download_filters=>{:skip_studios=>#<Set: {}>, :skip_performers=>#<Set: {}>, :skip_lesbian=>false, :skip_keywords=>[]}, :cookie_file=>"cookies/cookies.txt", :store=>"adt_download_status.store", :downloader=>"youtube-dl", :download_dir=>".", :quality=>"hd", :parallel=>1, :verbose=>true, :dry_run=>false, :downloader_flags=>"", :urls=>{:performers=>"0 items", :movies=>"1 items", :scenes=>"0 items"}, :site_config=>{:blowpass=>{:algolia_application_id=>nil, :algolia_api_key=>nil}}}
[19:51:36] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[19:51:36] [INFO ] [PROCESS START]
[19:51:36] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/video/getupclose/Up-Close-with-Leana-Lovings/233445
[19:51:36] [FATAL] AdultTimeDL::FatalError - AdultTimeDL::Net::AdultTimeIndex does not implement scenes_index
[19:51:36] [FATAL]      /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:61:in `scenes_index'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:32:in `block in search_by_movie'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:45:in `with_retry'
        /app/adulttime_dl/lib/adulttime_dl/net/algolia_client.rb:30:in `search_by_movie'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:26:in `block in process_movies'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:24:in `map'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:24:in `process_movies'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:16:in `start!'
        /app/adulttime_dl/lib/adulttime_dl/cli.rb:34:in `download'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/command.rb:27:in `run'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/invocation.rb:127:in `invoke_command'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor.rb:392:in `dispatch'
        /usr/local/bundle/gems/thor-1.2.2/lib/thor/base.rb:485:in `start'
        ./exe/./adulttime_dl:10:in `<main>'
/app/adulttime_dl #

And verifying I'm on the right version:

user1@server1 adulttime_dl % git show
commit 0280863c4a6935f6ab237661d2132e0839138d6c (HEAD -> main, origin/main, origin/HEAD)
Author: c477y <c477y@pm.me>
Date:   Wed Dec 13 00:40:10 2023 +0530

    BUGFIX:
    * Fix implementation of AdultTime index

Building my image

user1@server1 adulttime-dl % docker build . -t adulttime-dl:test-7
[+] Building 0.4s (13/13) FINISHED                                                                                                                                                                                              
 => [internal] load build definition from Dockerfile                                                                                                                                                                       0.0s
 => => transferring dockerfile: 1.05kB                                                                                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/ruby:3.1-alpine                                                                                                                                                         0.0s
 => [1/9] FROM docker.io/library/ruby:3.1-alpine                                                                                                                                                                           0.0s
 => CACHED [2/9] WORKDIR /app                                                                                                                                                                                              0.0s
 => CACHED [3/9] RUN apk --no-cache add     git     curl  doas     build-base     linux-headers     libxml2-dev     libxslt-dev                                                                                            0.0s
 => CACHED [4/9] RUN git clone https://github.com/c477y/adulttime_dl.git                                                                                                                                                   0.0s
 => CACHED [5/9] WORKDIR /app/adulttime_dl                                                                                                                                                                                 0.0s
 => CACHED [6/9] RUN apk -U add yt-dlp                                                                                                                                                                                     0.0s
 => CACHED [7/9] RUN apk -U add yt-dlp                                                                                                                                                                                     0.0s
 => CACHED [8/9] RUN gem install bundler                                                                                                                                                                                   0.0s
 => CACHED [9/9] RUN bundle install                                                                                                                                                                                        0.0s
 => exporting to image                                                                                                                                                                                                     0.1s
 => => exporting layers                                                                                                                                                                                                    0.0s
 => => writing image sha256:ae1ffb98c8693364645d4a212fc885c76e9688e09dd169d3aa92418b98d3225d                                                                                                                               0.0s
 => => naming to docker.io/library/adulttime-dl:test-7                                                                                                                                                                     0.0s
user1@server1 adulttime-dl %
complex-user commented 11 months ago

Also, sorry if this was documented somewhere, but what's the difference between a movie and a scene? How do I get the URL for either?

complex-user commented 11 months ago

Oh, progress!! It turned out the CACHED git clone was messing me up, it was ok locally but on the docker image it was the old one. Fixed that. Now, I'm stuck here, two ways:

Using 

  # [List] URLs of performer names that will be downloaded
  performers: []
  # [List] URLs of movie names that will be downloaded
  movies: [https://members.adulttime.com/en/video/digitalsin/XXX-Big-Tit-Rubdown-Scene-1/209719]
  # [List] URLs of scene names that will be downloaded
  scenes: []

We get 

/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime --verbose
[20:12:46] [INFO ] [DOWNLOAD LOGS GENERATED TO] /app/adulttime_dl/downloader.log
[20:12:47] [INFO ] youtube-dl installed with version 2023.11.14
[20:12:47] [DEBUG] {:site=>"adulttime", :download_filters=>{:skip_studios=>#<Set: {}>, :skip_performers=>#<Set: {}>, :skip_lesbian=>false, :skip_keywords=>[]}, :cookie_file=>"/app/adulttime_dl/cookies/cookies.txt", :store=>"adt_download_status.store", :downloader=>"youtube-dl", :download_dir=>".", :quality=>"hd", :parallel=>1, :verbose=>true, :dry_run=>false, :downloader_flags=>"", :urls=>{:performers=>"0 items", :movies=>"1 items", :scenes=>"0 items"}, :site_config=>{:blowpass=>{:algolia_application_id=>nil, :algolia_api_key=>nil}}}
[20:12:47] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[20:12:47] [INFO ] [PROCESS START]
[20:12:47] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/video/digitalsin/XXX-Big-Tit-Rubdown-Scene-1/209719
[20:12:47] [DEBUG] [HTTParty] [2023-12-12 20:12:47 +0000] 302 "GET /" - 
[20:12:48] [DEBUG] [HTTParty] [2023-12-12 20:12:48 +0000] 200 "GET https://www.adulttime.com/en/login/index/aHR0cHMlM0ElMkYlMkZtZW1iZXJzLmFkdWx0dGltZS5jb20lMkY=" - 
[20:12:48] [ERROR] [EMPTY RESULT] XXX Big Tit Rubdown Scene 1
[20:12:48] [INFO ] [PROCESS COMPLETE]
/app/adulttime_dl #

Using 

  # [List] URLs of performer names that will be downloaded
  performers: [https://members.adulttime.com/en/pornstar/view/Angela-White/48667]
  # [List] URLs of movie names that will be downloaded
  movies: []
  # [List] URLs of scene names that will be downloaded
  scenes: []

We get 

<what appears to be a successful parsing of scenes by performer>
...
[20:14:58] [DEBUG] [HTTParty] [2023-12-12 20:14:58 +0000] 302 "GET /movieaction/download/227971/720p/mp4" - 
[20:14:58] [ERROR] [DIRECT DOWNLOAD FAIL] API Failure:
        URL: https://members.adulttime.com/movieaction/download/227971/720p/mp4
        RESPONSE CODE: 302
        ERROR MESSAGE: unexpected redirection to https://www.adulttime.com/en/login/index/aHR0cHMlM0ElMkYlMkZtZW1iZXJzLmFkdWx0dGltZS5jb20lMkZtb3ZpZWFjdGlvbiUyRmRvd25sb2FkJTJGMjI3OTcxJTJGNzIwcCUyRm1wNCUzRmFsdXAlM0RBRHBKUlBlbDg2a0JMZEtLREdkanAxTWR2blFiWGVLWHE3MHU5WVJtNWtuY01uSnVQYTYtQ2ZhRV80Y2w0Q014QXlhTHdUaTA5c1JLdFEtY0FtUUVOWmdVNEt1c3FPMWdwbnRtNDZvdGNHNEtLRnlBWFhxN0xaLUIxY3NRdFA3Rk8wbFNaVVppNEFURDZUNmhzZEpGVHdBZEljNkNKanNaelV0bllNdGt2ZXdRc29hYlIyYzdGaFFPM2dLdnR6VXBvN2U1OEVUM0k3V3hLbkhxdm9mU1dKSSUyQQ==
[20:14:59] [DEBUG] [HTTParty] [2023-12-12 20:14:59 +0000] 302 "GET /media/streamingUrls/227971" - 
[20:14:59] [ERROR] [DIRECT DOWNLOAD FAIL] API Failure:
        URL: https://members.adulttime.com/media/streamingUrls/227971
        RESPONSE CODE: 302
        ERROR MESSAGE: unexpected redirection to https://www.adulttime.com/en/login/index/aHR0cHMlM0ElMkYlMkZtZW1iZXJzLmFkdWx0dGltZS5jb20lMkZtZWRpYSUyRnN0cmVhbWluZ1VybHMlMkYyMjc5NzElM0ZhbHVwJTNEQUFwcUhja2MySjZsVFRFOFBhZkNWTHJ4dUoxaC1EQk9vM0xQdkFzS3h0WndoOW1lcUFwcmtzanlLTjlMX3BsNDF3M3dDM2Y4SEQ3RTZXd0R3ZklKOE5JTWdBaWhGRTN5Tk9qZkhuWC1fTEdVUldJVWdZRGFEVDk3ZUtmTjdiVzk0WVVwQzJielBGT3hJeGNGUXBYQWFseXhfa2N1emlLbFphdV9saU5LeEV3dTZ3MDJ6MGNobWV5MXZZZ2JkY0RGQVkzQ2pHMVZhNDBFM0kwOFB5bE81VkElMkE=
[20:14:59] [FATAL] AdultTimeDL::FatalError - Redirection suggests possible cookie/token expiry. Regenerate token and try again.
[20:14:59] [FATAL]      /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:108:in `rescue in download_using_stream'
        /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:93:in `download_using_stream'
        /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:34:in `download'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:44:in `block (3 levels) in process_performer'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:587:in `call_with_index'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:378:in `block (2 levels) in work_in_threads'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:597:in `with_instrumentation'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:377:in `block in work_in_threads'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:222:in `block (4 levels) in in_threads'
/app/adulttime_dl #

And btw the cookies were very freshly out of the oven.

c477y commented 11 months ago

So for issue 1, I think https://members.adulttime.com/en/video/digitalsin/XXX-Big-Tit-Rubdown-Scene-1/209719 is a link to an individual scene, not a movie (a movie is a collection of scenes). So in this case, you want to keep this link in the scenes list.

members.adulttime.com/en/dvds lists the movies and members.adulttime.com/en/videos lists the videos/scenes.

As for issue 2, I don't know what's happening 😕 . The error should only come if the request was made with an expired cookie OR the cookie file was empty. How are you generating the cookies.txt file? I updated the README with instructions to how to get the cookie file. Can you give that a try? I don't know what else might be happening. If this doesn't work, I can tell you some debugging steps so you can inspect the exact request made by the CLI.

complex-user commented 11 months ago

So after moving that url to scenes:

/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime --verbose
[21:41:07] [INFO ] [DOWNLOAD LOGS GENERATED TO] /app/adulttime_dl/downloader.log
[21:41:08] [INFO ] youtube-dl installed with version 2023.11.14
[21:41:08] [DEBUG] {:site=>"adulttime", :download_filters=>{:skip_studios=>#<Set: {}>, :skip_performers=>#<Set: {}>, :skip_lesbian=>false, :skip_keywords=>[]}, :cookie_file=>"/app/adulttime_dl/cookies/cookies.txt", :store=>"adt_download_status.store", :downloader=>"youtube-dl", :download_dir=>".", :quality=>"hd", :parallel=>1, :verbose=>true, :dry_run=>false, :downloader_flags=>"", :urls=>{:performers=>"0 items", :movies=>"0 items", :scenes=>"1 items"}, :site_config=>{:blowpass=>{:algolia_application_id=>nil, :algolia_api_key=>nil}}}
[21:41:08] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[21:41:08] [INFO ] [PROCESS START]
[21:41:08] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/video/digitalsin/XXX-Big-Tit-Rubdown-Scene-1/209719
/app/adulttime_dl/lib/adulttime_dl/net/base_index.rb:12:in `search_by_all_scenes': AdultTimeDL::Net::AdultTimeIndex does not implement search_by_all_scenes (NotImplementedError)
        from /app/adulttime_dl/lib/adulttime_dl/client.rb:62:in `block in process_scenes'
        from /app/adulttime_dl/lib/adulttime_dl/client.rb:60:in `map'
        from /app/adulttime_dl/lib/adulttime_dl/client.rb:60:in `process_scenes'
        from /app/adulttime_dl/lib/adulttime_dl/client.rb:17:in `start!'
        from /app/adulttime_dl/lib/adulttime_dl/cli.rb:38:in `download'
        from /usr/local/bundle/gems/thor-1.2.2/lib/thor/command.rb:27:in `run'
        from /usr/local/bundle/gems/thor-1.2.2/lib/thor/invocation.rb:127:in `invoke_command'
        from /usr/local/bundle/gems/thor-1.2.2/lib/thor.rb:392:in `dispatch'
        from /usr/local/bundle/gems/thor-1.2.2/lib/thor/base.rb:485:in `start'
        from ./exe/./adulttime_dl:10:in `<main>'
/app/adulttime_dl #

The cookies I'm generating using the Firefox Extension: https://github.com/rotemdan/ExportCookies I'm enabling the extension in a Private Browswing window Navigating to a video, and making sure it loads Then, on the extension menu I click export for all sites Save it in cookies/cookies.txt as is.

complex-user commented 11 months ago

Same results with the cookies.txt extension.

complex-user commented 11 months ago

I also tested:

Movie from https://members.adulttime.com/en/dvds

  # [List] URLs of performer names that will be downloaded
  performers: []
  # [List] URLs of movie names that will be downloaded
  movies: [https://members.adulttime.com/en/dvd/Cum-Swap-Cuties-03/99715]
  # [List] URLs of scene names that will be downloaded
  scenes: []

Output:

/app/adulttime_dl # ./exe/./adulttime_dl download --cookie-file=cookies/cookies.txt --config=config/config.yaml adulttime --verbose
[21:51:19] [INFO ] [DOWNLOAD LOGS GENERATED TO] /app/adulttime_dl/downloader.log
[21:51:19] [INFO ] youtube-dl installed with version 2023.11.14
[21:51:19] [DEBUG] {:site=>"adulttime", :download_filters=>{:skip_studios=>#<Set: {}>, :skip_performers=>#<Set: {}>, :skip_lesbian=>false, :skip_keywords=>[]}, :cookie_file=>"/app/adulttime_dl/cookies/cookies.txt", :store=>"adt_download_status.store", :downloader=>"youtube-dl", :download_dir=>".", :quality=>"hd", :parallel=>1, :verbose=>true, :dry_run=>false, :downloader_flags=>"", :urls=>{:performers=>"0 items", :movies=>"1 items", :scenes=>"0 items"}, :site_config=>{:blowpass=>{:algolia_application_id=>nil, :algolia_api_key=>nil}}}
[21:51:19] [INFO ] [DATABASE_INIT] /app/adulttime_dl/adt_download_status.store
[21:51:19] [INFO ] [PROCESS START]
[21:51:19] [INFO ] [PROCESSING URL] https://members.adulttime.com/en/dvd/Cum-Swap-Cuties-03/99715
[21:51:20] [DEBUG] [HTTParty] [2023-12-12 21:51:20 +0000] 302 "GET /" - 
[21:51:20] [DEBUG] [HTTParty] [2023-12-12 21:51:20 +0000] 200 "GET https://www.adulttime.com/en/login/index/aHR0cHMlM0ElMkYlMkZtZW1iZXJzLmFkdWx0dGltZS5jb20lMkY=" - 
[21:51:21] [DEBUG] [HTTParty] [2023-12-12 21:51:21 +0000] 302 "GET /movieaction/download/216286/720p/mp4" - 
[21:51:21] [ERROR] [DIRECT DOWNLOAD FAIL] API Failure:
        URL: https://members.adulttime.com/movieaction/download/216286/720p/mp4
        RESPONSE CODE: 302
        ERROR MESSAGE: unexpected redirection to https://www.adulttime.com/en/login/index/aHR0cHMlM0ElMkYlMkZtZW1iZXJzLmFkdWx0dGltZS5jb20lMkZtb3ZpZWFjdGlvbiUyRmRvd25sb2FkJTJGMjE2Mjg2JTJGNzIwcCUyRm1wNCUzRmFsdXAlM0RBQ2kybk9wMHJEMnVIUk91dzFNNFA4VVFNMk4wV3NxWjRVTS1vU3ZiZFQ4RG90NlhHVzFWLWNzbm1BSFlwLXBUYUExRkJhbEFRbGpHT0hXX3AxZ2VYRnFiMWVxbkVNSkZnc3g2a1JETk9Zdll2R3huN0dSU25zQmNZMEE3MTg2NUI3dTV3VEFyS29adDZrb3lYbEgwMzBkUTN4cFlPSVdKNFI0c195UE9oVm94Tl82VUZCXzAxdXAwRENoV0UzN0hlb0x1UWRUaE9jbG9rOVRSMG9qT0dKTSUyQQ==
[21:51:21] [DEBUG] [HTTParty] [2023-12-12 21:51:21 +0000] 302 "GET /media/streamingUrls/216286" - 
[21:51:21] [ERROR] [DIRECT DOWNLOAD FAIL] API Failure:
        URL: https://members.adulttime.com/media/streamingUrls/216286
        RESPONSE CODE: 302
        ERROR MESSAGE: unexpected redirection to https://www.adulttime.com/en/login/index/aHR0cHMlM0ElMkYlMkZtZW1iZXJzLmFkdWx0dGltZS5jb20lMkZtZWRpYSUyRnN0cmVhbWluZ1VybHMlMkYyMTYyODYlM0ZhbHVwJTNEQURNS2MxVEhKdTQySThlVS1ERWZnS3Z5RDh6bkNzX2hsNEpjQXcyMi14ZFgwZHFkVnl0SE04RnBPblRIWHh5ZlJwNGV6UXctN2RGYVR4VEpOVXhCWWQ3QVUtSUktVzIxTUVDemdBbkh3ejBLNDZWSWIzYmgtQWN0SFNaSjNEOFEwZ0JTY0ROdU84NmV3WnBCd0xvcmQydW5wWUNmVFNOUnc4ZllwYy1yUHJQQlhiSDAyNWhQRDFZcU1XR285MWFidlVZdHlXSTkxNkFiakg2aE5fcDdfWXMlMkE=
[21:51:21] [FATAL] AdultTimeDL::FatalError - Redirection suggests possible cookie/token expiry. Regenerate token and try again.
[21:51:21] [FATAL]      /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:108:in `rescue in download_using_stream'
        /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:93:in `download_using_stream'
        /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:34:in `download'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:27:in `block (2 levels) in process_movies'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:587:in `call_with_index'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:378:in `block (2 levels) in work_in_threads'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:597:in `with_instrumentation'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:377:in `block in work_in_threads'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:222:in `block (4 levels) in in_threads'
/app/adulttime_dl #
c477y commented 11 months ago

Hmm so what's happening is that the CLI is trying to call AdultTime APIs to get the link to stream a video. If the cookie provided is correct, it should return back a response with the link to the video. But for some reason, your API calls are not getting authenticated properly. I have a hunch that maybe the cookie file is not parsed properly (maybe because you are running it via Docker?), because I cannot reproduce this issue myself. Can you do run some debugging steps to see the exact request?

  1. Edit this file algolia_streaming_links.rb
  2. Add this line binding.pry anywhere before line 26. 
    ...
resp = handle_response!(self.class.get(path, headers: headers))
binding.pry
return nil if resp == []
...
  3. Run the CLI again, only this time the execution will halt when the interpreter hits the binding.pry line.
  4. Inspect the resp, headers and path variables (Just type them individually and press return to print them on console)
  5. Post the response back here.

I can understand RedirectionError to happen when trying to get the download links. This can happen if the membership plan does not allow downloads, so the API will redirect you to upgrade your plan. But streaming should be allowed on all plans, which leads me to think maybe the cookie is not parsed correctly.

complex-user commented 11 months ago

Right, I have a streaming only plan. I'll give this a go in a bit. If you have a safe place to drop creds, I don't mind sharing whatever is needed for troubleshooting.

complex-user commented 11 months ago 
[16] pry(#<AdultTimeDL::Net::AdultTimeStreamingLinks>)> resp
=> nil
[17] pry(#<AdultTimeDL::Net::AdultTimeStreamingLinks>)> 
[1] pry(#<AdultTimeDL::Net::AdultTimeStreamingLinks>)> headers
=> {"User-Agent"=>"Mozilla/5.0 (Windows NT x.y; rv:10.0) Gecko/20100101 Firefox/10.0",
 "Accept"=>"application/json",
 "DNT"=>"1",
 "X-Requested-With"=>"XMLHttpRequest",
 "Cookie"=>
  "_BEAMER_DATE_POOJBdFl6198=2023-11-30T16:40:26.000Z; _BEAMER_FIRST_VISIT_POOJBdFl6198=2023-11-28T22:20:44.604Z; _BEAMER_USER_ID_POOJBdFl6198=f8434498-b3a3-4398-84d3-b0c741d1a3d7; _BEAMER_USER_ID_POOJBdFl6198=f8434498-b3a3-4398-84d3-b0c741d1a3d7; __cfshdid=undefined; autologin_hash=bb8abde7f6e220a99eebc89f590c29ef40bd09c0fbc50f27dbf333672cd94344; autologin_userid=25100811; identityStatus=2bb6b986c5d6fb26dd9dd1054b545ce022371b0c; landingpage=%2Fen%2Fvideo%2Fhowwomenorgasm%2FLeana-Loves-To-Be-Stimulated%2F233394; mDateTime=2023-11-30%2015%3A25%3A17; mDateTime=2023-11-28%2015%3A18%3A00; mOffset=-7; mOffset=-7; origin=promo; previousImpressionUUID=115cbc9a-d44f-4c94-a6df-90082336435f; s_info=a%253A17%253A%257Bs%253A6%253A%2522adv_id%2522%253Bi%253A1000%253Bs%253A8%253A%2522campaign%2522%253Bs%253A0%253A%2522%2522%253Bs%253A12%253A%2522country_code%2522%253Bs%253A2%253A%2522US%2522%253Bs%253A6%253A%2522origin%2522%253Bs%253A5%253A%2522promo%2522%253Bs%253A10%253A%2522program_id%2522%253Bi%253A8%253Bs%253A11%253A%2522referer_url%2522%253Bs%253A0%253A%2522%2522%253Bs%253A13%253A%2522subprogram_id%2522%253Bi%253A1%253Bs%253A7%253A%2522site_id%2522%253Bi%253A247%253Bs%253A10%253A%2522gallery_id%2522%253Bi%253A0%253Bs%253A10%253A%2522user_agent%2522%253Bs%253A84%253A%2522Mozilla%252F5.0%2520%2528Macintosh%253B%2520Intel%2520Mac%2520OS%2520X%252010.15%253B%2520rv%253A109.0%2529%2520Gecko%252F20100101%2520Firefox%252F119.0%2522%253Bs%253A7%253A%2522section%2522%253Bs%253A3%253A%2522www%2522%253Bs%253A12%253A%2522adv_category%2522%253Bs%253A16%253A%2522internal%2520organic%2522%253Bs%253A4%253A%2522type%2522%253BN%253Bs%253A7%253A%2522skin_id%2522%253Bi%253A0%253Bs%253A5%253A%2522pg_id%2522%253Bs%253A88%253A%2522zGZKXnhgcNjiAFXUoCgHtuaJXIwXPMmPRqmDN7MlrHHTe0Gdu9XIxSsBchcocwqE5F1GT7Tp44xTRdRYIN39JA%253D%253D%2522%253Bs%253A2%253A%2522cp%2522%253Bi%253A961%253Bs%253A9%253A%2522timestamp%2522%253Bi%253A1701209796%253B%257D; shield_FPC=SCu43OHDgwVfKccuSBR38ekVrDLEznL3a6; ssale=a%253A17%253A%257Bs%253A6%253A%2522adv_id%2522%253Bi%253A1000%253Bs%253A8%253A%2522campaign%2522%253Bs%253A0%253A%2522%2522%253Bs%253A12%253A%2522country_code%2522%253Bs%253A2%253A%2522US%2522%253Bs%253A6%253A%2522origin%2522%253Bs%253A5%253A%2522promo%2522%253Bs%253A10%253A%2522program_id%2522%253Bi%253A8%253Bs%253A11%253A%2522referer_url%2522%253Bs%253A0%253A%2522%2522%253Bs%253A13%253A%2522subprogram_id%2522%253Bi%253A1%253Bs%253A7%253A%2522site_id%2522%253Bi%253A247%253Bs%253A10%253A%2522gallery_id%2522%253Bi%253A0%253Bs%253A10%253A%2522user_agent%2522%253Bs%253A84%253A%2522Mozilla%252F5.0%2520%2528Macintosh%253B%2520Intel%2520Mac%2520OS%2520X%252010.15%253B%2520rv%253A109.0%2529%2520Gecko%252F20100101%2520Firefox%252F119.0%2522%253Bs%253A7%253A%2522section%2522%253Bs%253A3%253A%2522www%2522%253Bs%253A12%253A%2522adv_category%2522%253Bs%253A16%253A%2522internal%2520organic%2522%253Bs%253A4%253A%2522type%2522%253BN%253Bs%253A7%253A%2522skin_id%2522%253Bi%253A0%253Bs%253A5%253A%2522pg_id%2522%253Bs%253A88%253A%2522zGZKXnhgcNjiAFXUoCgHtuaJXIwXPMmPRqmDN7MlrHHTe0Gdu9XIxSsBchcocwqE5F1GT7Tp44xTRdRYIN39JA%253D%253D%2522%253Bs%253A2%253A%2522cp%2522%253Bi%253A961%253Bs%253A9%253A%2522timestamp%2522%253Bi%253A1701209796%253B%257D; timestamp=1701209796"}
[2] pry(#<AdultTimeDL::Net::AdultTimeStreamingLinks>)> 
[2] pry(#<AdultTimeDL::Net::AdultTimeStreamingLinks>)> path
=> "/media/streamingUrls/216286"
[3] pry(#<AdultTimeDL::Net::AdultTimeStreamingLinks>)>

Then when allowed to continue

[23:11:16] [DEBUG] [HTTParty] [2023-12-13 23:11:16 +0000] 302 "GET /media/streamingUrls/216286" - 
[23:11:16] [ERROR] [DIRECT DOWNLOAD FAIL] API Failure:
        URL: https://members.adulttime.com/media/streamingUrls/216286
        RESPONSE CODE: 302
        ERROR MESSAGE: unexpected redirection to https://www.adulttime.com/en/login/index/aHR0cHMlM0ElMkYlMkZtZW1iZXJzLmFkdWx0dGltZS5jb20lMkZtZWRpYSUyRnN0cmVhbWluZ1VybHMlMkYyMTYyODYlM0ZhbHVwJTNEQUZTQjI4RkFtUk1kVzBVU3lXUkFWQnZkR3J1S3FRSk1UbVBkcDZPUVZZU1J0U2RSMG83Q3lxUmdjVFdEVlFwdloycEZOM0Riem1sT29TdWlXMTJibDA4X3lhNjJsYTRwcFNuWDVZbHB1aU5OdGVwYURCRnIybVpfTnNpcUNoUm51Q2thUXVXeVpkWTEtb3EtVWJYNHJSZkNkMERCdlk2NDZXVlhMS3p3TXlPb0J0UnNqNVBHNndDdGh6T0xZOEFBN3pDVFJPTjRFRWllX1E3QldqNmZQeWMlMkE=
[23:11:16] [FATAL] AdultTimeDL::FatalError - Redirection suggests possible cookie/token expiry. Regenerate token and try again.
[23:11:16] [FATAL]      /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:108:in `rescue in download_using_stream'
        /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:93:in `download_using_stream'
        /app/adulttime_dl/lib/adulttime_dl/downloader/download.rb:34:in `download'
        /app/adulttime_dl/lib/adulttime_dl/client.rb:27:in `block (2 levels) in process_movies'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:587:in `call_with_index'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:378:in `block (2 levels) in work_in_threads'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:597:in `with_instrumentation'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:377:in `block in work_in_threads'
        /usr/local/bundle/gems/parallel-1.22.1/lib/parallel.rb:222:in `block (4 levels) in in_threads'
/app/adulttime_dl #
c477y commented 11 months ago

I am not entirely sure what's happening here. Your cookie seems fine to me and other than a few different keys (compared with my credentials), the overall structure is similar.

My best (and arguably worst) idea is probably to test with your credentials directly (happy for you to do it as well, I can tell you the instructions). I would be curous to see if the website behaves differently with membership tiers. We have two ways forward:

  1. We test the APIs directly outside the CLI. This will involve calling the endpoints using some external tool like Postman to figure out why your cookies don't pass authentication on the APIs. I can't commit any time on this since I'm on a break over Christmas (apologies).
  2. The CLI also supports an alternate download strategy that leverages a headless browser. I don't prefer this method as it's slower than calling traditional APIs and is prone to breakage if the page layout changes for some reason. It's only meant to be used for sites that don't expose any APIs whatsover (SSR sites for example).

If you have a safe place to drop creds

You can drop me a mail at c477y@pm.me :)

complex-user commented 10 months ago

I don't see where the reply went, but I thought I saw seeing my credentials took 2FA? For reference, it's not turned on in settings. I used the gift card method of payment for the site, so I got a random user/pass generated.