Zimfarm readyness - Githubissues

openzim / freecodecamp

FreeCodeCamp.org scraper (to ZIM)

GNU General Public License v3.0

4 stars 2 forks source link

Zimfarm readyness #14

Closed benoit74 closed 1 year ago

benoit74 commented 1 year ago

Fix #11 Fix #13

Changes : MANY (sorry)

rework CLI args to align with what is typically needed / available from other scrapers
rework Makefile target names
rework build and Docker image folder structures
migrate to hatch instead of setuptools
align with Python bootstrap conventions
removed devcontainer
fixed nodejs Docker image
optimize Dockerfile structure to better benefit from build cache

Some details:

I remove devcontainer because it did not worked on my machine and is hence really frustrating for a new contributor (+ I'm afraid we won't have resources to maintain it in the long run, at least for now)
I fixed the nodejs Docker image because devcontainer image is not meant for our build usage

I suggest that we release this as a 1.0.0 since it has not yet been released anyway.

~~NOTA: we have to enable Trusted Publishing for this repo.~~ ✅

benoit74 commented 1 year ago

Is there any way to tell CodeFactor to ignore a check on a specific line (like I did for Ruff on the specific line which raised an issue)?

rgaudin commented 1 year ago

Is there any way to tell CodeFactor to ignore a check on a specific line (like I did for Ruff on the specific line which raised an issue)?

Codefactor itself no but codefactor is just an integrated set of open tools (with private config). In this case it's a Bandit rule raising so silence it as such

kelson42 commented 1 year ago

Is there any way to tell CodeFactor to ignore a check on a specific line (like I did for Ruff on the specific line which raised an issue)?

Yes AFAIK with codefactor web ui

benoit74 commented 1 year ago

Is there any way to tell CodeFactor to ignore a check on a specific line (like I did for Ruff on the specific line which raised an issue)?

Yes AFAIK with codefactor web ui

Looks like I do not have proper access rights ... anyway, done through inline comment ok

rgaudin commented 1 year ago

Is there any way to tell CodeFactor to ignore a check on a specific line (like I did for Ruff on the specific line which raised an issue)?

Yes AFAIK with codefactor web ui

We obviously don't want to set exceptions on Codefactor UI!

rgaudin commented 1 year ago

Looks like I do not have proper access rights ... anyway, done through inline comment ok

You're admin on this repo.

benoit74 commented 1 year ago

Sorry about the files that are considered by git as "deleted/new" instead of been "moved", unfortunately there is little I can do on this.

Should I list in the first comment all files that are considered as "deleted/new" which have indeed only been "moved", so you can git diff them manually?

I agree this is a real pain.

For instance here:

openzim/fcctozim/build.py moved to scraper/src/fcc2zim/build.py
openzim/fcctozim/fetch.py moved to scraper/src/fcc2zim/fetch.py
openzim/fcctozim/__init__.py moved to scraper/src/fcc2zim/__init__.py (and emptied ...)
client/.eslintignore moved to zimui/.eslintignore

rgaudin commented 1 year ago

Should I list in the first comment all files that are considered as "deleted/new" which have indeed only been "moved", so you can git diff them manually?

I guess. We can try next time