Closed mxfilerelatedcache closed 3 months ago
From a quick glance, having the IS24-cookie or other crawler-specific settings read from an ENV variable is currently not possible without some code overhaul or refactoring.
Your configuration seems fine. Can you try changing the message format in the config.yaml to check whether the file gets read?
Hey @pomeloy, I did some trial & error and the solution seems to be the following: Once you define the urls inside the config.yaml
instead of thedocker-compose.yaml
, the script seems to look for the information in the config.yaml
, and then also used the cookie from there.
As the issue technically still persists, I would leave it open for now. However, with my fix, it is solvable.
I've added the option to override the IS24 cookie with an environment variable in #587 - you can now set FLATHUNTER_IS24_COOKIE
. In general it should be fine to combine the config file with the environment variables. If the environment variables are present, they will override the settings in the file. And if no config file is present, everything should be read from the environment.
I've added some tests for this. If you find that the functionality isn't working as expected with the latest code on main
, please let me know. If not, please close the ticket as resolved.
Thanks for the report!
Hey, thanks for your effort!
I just added FLATHUNTER_IS24_COOKIE
as an environment variable and tested whether it works using the urls from the config.yaml
, thus a combination of both config.yaml
and environment variables. Everything now works as expected.
I close the issue now.
I just set up the bot using docker-compose on my server and it works like a charm! Great work. However, when setting up the environment variables in the
docker-compose.yaml
, I realised some variables that would occur in theconfig.yaml
are missing and not yet implemented as environment variables. For thedurations
using Google Maps API that would be okay, but I feel that adding the IS24 Cookie seems like an important step.The readme mentions that both can be combined (environment variables &
config.yaml
), however is not quite clear as to how exactly this would work. Currently, I have theconfig.yaml
in my working directory (where it should be), yet it does not seem to be used by the script. Am I doing something wrong while mounting the volume in thedocker-compose.yaml
? (I doubt that because theprocessed_ids.db
seems to work fine) or am I missing something else?Here is my
docker-compose.yaml
:and this is my
config.yaml
: