justfoolingaround/animdl

^{A highly efficient, powerful and fast anime scraper.}

Overview

Installation
1. PIP Installation
2. Source Code Download
Support
Usage
Providers
Configurations
Contributing to the project
Project Disclaimer
In a nutshell
Code redistribution
From the author
Honourable mentions
- Similar Projects
- Internal Dependencies
Sponsoring the project

Installation

This project can be installed on to your device via different mechanisms, these mechanisms are listed below in the order of ease.

PIP Installs Packages aka PIP Installation
```
pip install animdl
```

Installation from source

First ensure Poetry is installed.

Then run the following command:

git clone https://www.github.com/justfoolingaround/animdl.git \
&& cd animdl \
&& poetry build \
&& pip install animdl \
&& cd ..

Support

You can contact the developer directly via this email. However, the most recommended way is to head to the discord server.

If you run into issues or want to request a new feature, you are encouraged to make a GitHub issue, won't bite you, trust me.

Usage

Usage: animdl [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  download  Download your favorite anime by query.
  grab      Stream the stream links to the stdout stream for external usage.
  schedule  Know which animes are going over the air when.
  search    Search for an anime in the provider.
  stream    Stream your favorite anime by query.

The stream option is disabled automatically if the project cannot find any of the supported streamers.

`stream` / `download` / `grab`

These commands are the main set of command in the project. All of them scrape the target site, the only difference is how it is used.

The stream option tosses the stream url to a player so that you can seamlessly binge your anime.
- Streaming supports Discord Rich Presence with pypresence.
The download option downloads the anime to your local machine.
- Downloading is done in the directory where you run the project.
- -d flag can be used to specify a download folder as well.
- Internet Download Manager is supported and can be used via --idm flag. This downloader cannot download HLS streams.
- The project cannot modify the content type. That means, videos in the ts format need to be converted to other formats externally post download.
- The downloading process cannot be controlled.
  - If download speed fluctuates, it is a server-side problem. The project cannot "fix" it.
  - If the download is slow, it is based on the server's upload speed. The project cannot "fix" it. Speedtests are not reliable and their results will not correspond to the download speeds obtained through the project.
- The project utilises the fastest and the most straight-forward way to download, there is no further optimisation, period.
The grab option simply streams the stream url to the stdout stream.
- This is useful for external usage and testing.

animdl stream "One Piece"

_{Providers can be specified by using provider prefix, 9anime:One Piece, will use the 9Anime provider.}

_{You can specify direct urls to the provider; the project will automatically detect the provider and continue scraping. This method ignores searching.}

`-r` / `--range` argument

This argument is shared by stream, download and grab, it can be used to hand over custom ranges for selecting episodes.

This argument constructs a check for the episodes, hence, will not throw error if the episode count does not meet the scraping count.
This argument can be used in reverse order, the constructer will automatically fix the order.
Range intersections will not cause issues.
1 will be treated as a singular range from 1 to 1.
1-2 will be treated as a range from 1 to 2.
1-2,230-340 will be treated as two different checks. The first check will be from 1 to 2, the second from 230 to 340.
You can use any separators, the project will automatically parse your range string.

`--index` argument

This argument is shared by stream, download and grab, it can be used to automatically select the search result. The default argument for index is 1, that is, the first stream.

--index 2 will automatically select the second search result without prompt.

`-s` / `--special` argument

This argument is shared by stream and download, it can be used to hand over the latest episode of the anime by using -s latest. Similarly, the latest 2 episodes can be selected via latest-2.

This argument is compatible with -r, you will get the last episode of the range.

`-q` / `--quality` argument

This argument is incredibly powerful and can be used to select streams from their attributes.

1080 will select the stream with the resolution of 1080.
1080/worst will do the above but will also select the worst quality stream if that stream is not available.
best[title] will select the best stream that has the title attribute.
best[title=r'^DUB'] will select the stream with the title attribute that matches the regular expression. If r is not used, the expression will be treated as a literal string.
The normal integers can be substituted with best and worst for special parsing.
You need not mention the quality in the argument if you just want an attribute.

You can find out what attributes are available for each stream by using the grab command.

[
  {
    "stream_url": "https://yqwym.vizcloud.digital/simple/EqPFI_kQBAro1HhYl67rC8UuoVwHubb7CkJ7rqk+wYMnU94US2El/br/list.m3u8#.mp4",
    "headers": {
      "referer": "https://vizcloud.digital/embed/83P7OX0N8PLE"
    }
  },
  {
    "stream_url": "https://yqwym.vidstream.pro/EqPVIPsMWl322yVezviuGdNz9wsVp_2ySFow5Od52MBlQ9QQG34s9aQ0yhbkTfyI+tzdG4991O3X4fVqACOikmeZRvMNGrBeQ5aivXxFIkYzNJElHAM1icyfowvCviiceQevRCxV9F7i7CIYt0hIz61716gsQxXskJ6eV4Gg4_OC/br/list.m3u8",
    "headers": {
      "referer": "https://vizcloud.digital/embed/83P7OX0N8PLE"
    }
  },
  {
    "stream_url": "https://yzqq.mcloud.to/12a3c523f910040ae8d4785897aeeb0bc52ea15c07b9b6fb0a427baea96ed0842f54d0184c6820e9f935c248a146eb8df28cc21a817ad2e8c0eefd680420a692659945f21618bd454698bcbf6e42394f3d4ee734180c3281ce9bb00fcaa2298e7913aa40036bbb0abaf07046a14442c2f32c9df66b1a/r/list.m3u8",
    "headers": {
      "referer": "https://mcloud.to/embed/k18xp6"
    }
  }
]

This is the prettified output of animdl grab "9anime:one piece", and the stream has headers and stream_url attributes.

If you feel that a particular stream is fast enough for you, -q [stream_url=r'.+mcloud\.to.+']_{(or equivalent, this is just for testing)} will select that stream.

Conclusion

This project posses powerful commands and arguments to aid them, there are many arguments that aren't specified here but are available in the project. This is done because these commands are advanced usage commands which may just cause confusion. Feel free to ask about them.

Providers

Please head to animdl's official provider benchmarks.

The images in that repository are automatically updated every few hours, hence, please be aware that the developers already know what's broken.

Some providers may not work due to DDoS protection services. We try our best to fix what's fixable. There are plenty of alternatives even if one goes down in the project.

The project contains providers that aren't mentioned here. This is intentional.

Configurations

Configuration files can be globally or locally be specified.

You can use the ANIMDL_CONFIG environment variable to specify a configuration file on a global level.

Else, a file with the name animdl_config.yml in the working directory will be used if available.

Futhermore, the configuration files can be globally placed at:

Windows:
- %USERPROFILE%/.animdl/config.yml
Anything else:
- $HOME/.config/animdl/config.yml

Only a singular configuration file in the above priority order is used, configurations aren't merged.

Setting up providers

You can specify a default provider using the default_provider configuration option.

default_provider: animixplay

This project uses a standardised url per provider. This makes the project capable of using different official proxies of the same provider.

This can be specified via site_urls by using the following configuration.

site_urls:
    animixplay: https://www.animixplay.to/

Quality selection

You can specify a default quality using the quality_string configuration option.

quality_string: best[subtitle]/best

Player selection

You can specify a default player using the default_player configuration option.

default_player: mpv

You can change player attributes, such as, what the executable is and what opts are to be passed during the player call.

players:
    mpv:
        executable: mpv
        opts: []

If the executable is found, the player will be eligible for use.

Currently supported players are:

mpv
vlc
iina
celluloid
ffplay
android (Android Player, only available when there aren't any headers for the stream)

ffmpeg set-up

You can make the project force ffmpeg for HLS downloading (awfully slow) and merging subtitles for downloads when external subtitles are available.

ffmpeg:
    executable: ffmpeg
    hls_download: false
    submerge: true

Discord Rich Presence set-up

This project supports RPC clients, this can be enabled only from the configuration. To use this, you must have pypresence installed via:

pip install pypresence

discord_presence: true

fzf set-up

You can force the project search prompts to use fzf.

This is an incredibly powerful tool and can enchance the user experience by a lot. For obvious reasons, you need to have it installed and on PATH (can be configured to use an executable path as well).

fzf:
    executable: fzf
    opts: []
    state: true

Users can benefit by using fzf and animdl stream twist: as this allows them to browse the entire Twist library with a heavenly interface.

The above screenshot has fzf's user theme configured.

Schedule set-up

The project can be used to view the schedule of the upcoming anime for the week with the user cherished time and date formats.

schedule:
    site_url: https://graphql.anilist.co/ # DO NOT CHANGE UNLESS YOU KNOW WHAT YOU'RE DOING
    date_format: "%b. %d, %A"
    time_format: "%X"

Please refer to Python datetime.strftime documentation for the available formats.

Contributing to the project

It is not recommended for any average Python developers to engage with the project codebase as the mechanisms used are extremely high level and requires greater understanding of the codebase and Python in general.

If you wish to, you can contribute to the project by submitting a pull request.

The best way to contribute would be to suggest the developer a provider or a feature, more better if you can show a logic of how that can be done, you'll be mentioned and thanked!

Project disclaimer

The disclaimer of the project can be found here.

In a nutshell

Abuses the developer's braincells in scraping to give about the fastest and most efficient toolset.
Brings about a highly powerful codebase which is just about appropriate for scraping.
Does not use any heavy dependencies by default.
Does not use selenium or JavaScript evaluators for scraping.
Integrates powerful internal tools such as a HLS downloader.
Is maintained, heavily as far as possible.

Code redistribution

Feel free to do so and take references from the codebase. You need not give credit to the project or justfoolingaround, however, please do a blame check and see if the contributor wants to be credited.

From the author

I just maintain this project for my gratification. I'd love to hear from you about your projects and problems (even unrelated to the project), so feel free to contact me.

I'm glad you're here!

Honourable mentions

Similar Projects

Shell
- ani-cli
Kotlin
- Cloudstream-3
- Saikou

These are actively maintained projects, each of which has its own unique features and functionality.

Internal Dependencies

Core
Optional
- junegunn/fzf
- qwertyquerty/pypresence

The project would definitely not be complete or even in a working state if it weren't for these dependencies.

Sponsoring the project

Usually sponsors means funding which consequently means money but for the project, it means stars. Feel free to star the project if you think it is worthy of one. Moreover, you can just talk, banter and argue with the developer as that way you'll be paying with your time.

^{You are an absolute legend, keep being awesome!}