mx-psi / fa-scraper

A FilmAffinity web scraper compatible with Letterboxd
GNU General Public License v3.0
19 stars 9 forks source link

Don't ignore TV films (TV) and shorts (S) by default #5

Closed sdecandelario closed 3 years ago

sdecandelario commented 4 years ago

At the moment we are skipping the tv films (TV) and shorts (S).

I think the best practice is to process all and a new option to specify what to ignore explicit.

Also I think can be a better improvement to remove the (TV) or (S) from the title to make the import more easy.

DiegoAsterio commented 4 years ago

@sdecandelario I am understanding your proposal as adding some argument (for instance --ignore) to the parser inside cli.py. The user will bind a list of categories they would like to be ignored to that argument. This list should affect the data retrieved so I guess it has to be passed as an argument to the function get_profile_data. Does this ignore list have to affect the tool when retrieving lists as well?

By looking at the source of fa_scrapper.py I notice you are skipping (TV) and (S) inside the function is_film. Should we

  1. modify it to take into account the list of ignored categories
  2. or add a new function (e.g. is_chosen_category) that substitutes is_film .

I personally prefer the latter option as the predicate is conceptually different.

mx-psi commented 4 years ago

@DiegoAsterio

I am understanding your proposal as adding some argument (for instance --ignore) to the parser inside cli.py.

Yes, we would add a new CLI option to set what to ignore and what not (your idea of how to go about it sounds good to me).

Does this ignore list have to affect the tool when retrieving lists as well?

Ideally yes, it would (it should be easy to do it so that it applies everywhere)

I personally prefer the latter option as the predicate is conceptually different.

I like option (2) more too!

sdecandelario commented 4 years ago

For me have more sense also the second option :)

sdecandelario commented 3 years ago

Also we need to update documentation?