funilrys / PyFunceble

The tool to check the availability or syntax of domain, IP or URL.
https://pyfunceble.github.io
Apache License 2.0
297 stars 44 forks source link

FEATURE: Preload/Continue like the CI workflow ... but without Git #329

Closed techdragon closed 1 day ago

techdragon commented 1 year ago

Description

The Preload/Continue functionality that's part of the CI workflow looks quite useful, but I'd like to be able to run it "offline"/"locally" without any Git repos or anything like that.

Possible Solution

Decouple the two parts of the CI workflow so that we can use the CI database and continue/preload functionality without needing to also use the CI automatic git repo commit writing.

Considered Alternative

I'm currently scripting around this by segmenting my (large, over 100,00,000 entries) list into multiple smaller lists and using shell scripts to dispatch the commands against these smaller sub-lists.

funilrys commented 1 year ago

Hi Samuel @techdragon,

Thank you for your contribution. I find your idea interesting but I'm not sure to understand it completely.


Let me start with a question: What is the difference between what you describe and the --preload along with the --continue argument?

Example:

pyfunceble --preload --continue -f https://raw.githubusercontent.com/PyFunceble/ci_test/master/test.list

As long as your input file doesn't change (hash changed), the preloader will load the content of your file in the DB you described (CSV, MySQL, MariaDB or PostgreSQL) before it starts the test. Once the content is loaded, the test will be triggered ...

To use that feature, there is no need to run within a CI/CD environment. In fact, if the CI Engine detection is disabled, it will run just fine 😄

So I'm not sure what you want to achieve or what the problem is ... Can you give me an example and more information about the environment (CI/CD engine, VMs, VPS, PC, Laptop, etc..) so that I can improve the process?

Maybe it's a problem in the wait I documented it ... Let me know so that we can fix any future misunderstanding!

I wish you all the best in this New Year!

@funilrys

techdragon commented 1 year ago

Ahh! It turns out this is a documentation issue.

From reading the documentation I was given the impression that both the continue and preload features required the use of the CI features. Should only need some small wording changes to point out the fact they work on their own without needing the CI features.

Thanks for clarifying!

spirillen commented 3 months ago

Ahh! It turns out this is a documentation issue.

From reading the documentation I was given the impression that both the continue and preload features required the use of the CI features. Should only need some small wording changes to point out the fact they work on their own without needing the CI features.

Thanks for clarifying!

Hi @techdragon

Is it possible I could have you make a issue about your commend for the docs, then I'll take a look into the wording.

Also, is that list something you have organized into categories? and one you would like to share?

spirillen commented 3 months ago

@techdragon was this like what you was requesting?? https://github.com/funilrys/PyFunceble/commit/baad5aedd06a28c26dd73842c006c9e7173bd5e3

funilrys commented 1 day ago

@techdragon @spirillen It is now possible to do this by setting the PYFUNCEBLE_STANDALONE_CI environment variable.

Once the environment variable is set, you can set/activate any CI related settings and it will work the same way but without any GIT commands.

@all-contributors please add @techdragon for ideas and bug.

allcontributors[bot] commented 1 day ago

@funilrys

We had trouble processing your request. Please try again later.