[Question] Is this the intended behavior of brython-cli --modules?

rayluo commented 3 years ago

The installation documentation brieftly mentions that, if we pip install brython into local PC, we can run brython-cli --modules which will "creates an application-specific distribution, to replace brython_stdlib.js by a smaller file".

I would assume that, when I have multiple independent projects Foo and Bar on my PC, I can run cd ~/Foo; brython-cli --modules" and "cd ~/Bar; brython-cli --modules at EACH project, and then the output of that "application-specific" brython-modules.js would be placed at each project's folder.

But the actual behavior is brython-cli --modules generates NO output in current working directory; instead, it generates a brython-modules.js at .../site-packages/brython/data/brython_modules.js as a template for the subsequent brython-cli --install. Wouldn't this cause a the modules for project "Foo" be also "brython-cli --install" into project "Bar"?

I know I could use one virtual environment for each project. But the current brython-cli's behavior is still surprising to me. Is it an intended behavior?

PierreQuentel commented 3 years ago

No, this is not the intended behaviour, and not what I observe on my PC:

C:\Users\PC>pip install brython
Collecting brython
  Downloading brython-3.9.1.tar.gz (1.3 MB)
     |████████████████████████████████| 1.3 MB 6.4 MB/s
Using legacy 'setup.py install' for brython, since package 'wheel' is not installed.
Installing collected packages: brython
    Running setup.py install for brython ... done
Successfully installed brython-3.9.1

C:\Users\PC>e:

E:\>cd temp/brython

E:\temp\brython>mkdir 20210131

E:\temp\brython>cd 20210131

E:\temp\brython\20210131>brython-cli --install
Installing Brython 3.9.1
done

E:\temp\brython\20210131>dir

 Répertoire de E:\temp\brython\20210131

31/01/2021  09:55    <DIR>          .
31/01/2021  09:55    <DIR>          ..
31/01/2021  09:55           743 753 brython.js
31/01/2021  09:55         3 949 545 brython_stdlib.js
31/01/2021  09:55            44 189 demo.html
31/01/2021  09:55               334 index.html
31/01/2021  09:55               401 README.txt
31/01/2021  09:55         1 046 041 unicode.txt
               6 fichier(s)        5 784 263 octets

E:\temp\brython\20210131>brython-cli --modules
Create brython_modules.js with all the modules used by the application
searching brython_stdlib.js...
finding packages...
script in html demo.html
script in html index.html

E:\temp\brython\20210131>dir

 Répertoire de E:\temp\brython\20210131

31/01/2021  09:55    <DIR>          .
31/01/2021  09:55    <DIR>          ..
31/01/2021  09:55           743 753 brython.js
31/01/2021  09:55         2 416 890 brython_modules.js
31/01/2021  09:55         3 949 545 brython_stdlib.js
31/01/2021  09:55            44 189 demo.html
31/01/2021  09:55               334 index.html
31/01/2021  09:55               401 README.txt
31/01/2021  09:55         1 046 041 unicode.txt
               7 fichier(s)        8 201 153 octets

E:\temp\brython\20210131>

As you can see, __brython_modules is in the directory where brython-cli --modules is executed. It is not intended to be put in site-packages__, and it is not used by brython-cli --install.

Maybe this is something specific to the operating system ?

rayluo commented 3 years ago

Oh I see. Looks like I found a corner case. The issue I found would happen in these steps:

I am trying to manually add brython into one of my EXISTING project Foo, so I did not run brython-cli --install first. In other words, my project does not currently contain a brython_stlib.js. Now I know, this would normally result in a "FileNotFoundError: Could not find brython_stdlib.js in this directory or below" exception, which would prevent the following things from happening.
But I happen to be that kind of person that prefers to put the virtual environment inside my project. So, there is a built-in Foo/.venv/..../site-packages/brython/data/brython_stlib.js in a deep subfolder inside my project, and then brython-cli --modules proceeded to put a new brython_modules.js there.
At this point, later when I ran brython-cli --install on any project, it could copy all those .../data/* into my project, including that brython_modules.js generated at step 2.

I think, putting the venv inside a project is still a reasonable workflow. Better allow it. How about we add/use a signature file inside Brython's package, which can later be used to detect "hey this is a Brython directory" and then have list_modules.py to skip them?

UPDATE: Detecting Brython folder would not be sufficient, because the virtual environment would contain some other Python packages which are dev-dependencies that I do not want to include into brython_modules.js. Need more thoughts on this. :-/

UPDATE 2: Detecting Brython folder is at least still valuable as a safe guard to avoid polluting the environment-wide template. For the rest of the problem, there is a workaround. I would need to reorganize my current completely-flat project structure from:

/..
/.venv
.gitignore
my_proj.py
my_proj.html
requirements.txt
...

to:

/..
/.venv
.gitignore
requirements.txt
/website/my_proj.py
/website/my_proj.html
/website/...

then I can run cd website; brython-cli --install; brython-cli --modules as expected.

kikocorreoso commented 3 years ago

I see a similar behavior that the one exposed by @rayluo

I create a dir and cd newdir/.
I create a virtualenv on the new dir python -m venv env.
I activate the new virtualenv . env/bin/activate.
I install brython there (python -m pip install brython).
I do brython-cli --install and I get the index.html, menu.html, brython.js, etc.
I modify index.html and then I use brython-cli --modules. I get a brython_modules.js but in env/lib/site-packages/brython/data/ but not in the folder where I run the command (brython-cli --modules).

So, from this discussion I would need to have something like:

|_ newdir
  |_ env
  |_ webdir

and then cd newdir/webdir and then in this folder is where I have to run brython-cli --install and brython-cli --modules.

Is this correct?

We shall update the docs to reflect this behavior.

rayluo commented 3 years ago

@kikocorreoso

That is not just "similar", it is the same issue I found. Welcome to the club. :-)

In this case, the proper solution would not be to just document it. Nobody would notice this kind of tiny detail in the doc. I would still suggest this:

How about we add/use a signature file inside Brython's package, which can later be used to detect "hey this is a Brython template directory" and then have list_modules.py to skip them?

Meanwhile, yes the workaround would be to use such directory structure:

|_ my_project
  |_ env
  |_ website

FWIW, I even build a brython project template containing such pattern (and some more other goodies).

kikocorreoso commented 3 years ago

@rayluo

Or maybe we could consider adding args to brython-cli --modules. Something like:

brython-cli --modules ./ --stdlib_path ./static/js

to indicate where to look for python code and to indicate where is the brython stdlib you want to use. Maybe you installed the latest brython version with python -m pip install brython but you want to use an older or a development version of the stdlib and the brython engine instead of that installed using pip.

nacho00112 commented 1 year ago

I have a idea, brython-cli could process a special file with a special name something like nobrython.txt which his content could be a list of directories and files, with relative path and glob support, Telling to brython-cli to not process those archives, so if brython/data have a nobrython.txt with a "." Meaning the current directory, brython-cli would not process that directory and all would be fine, this isn't a possible solution? the git's .gitignore file already does something like

brython-dev / brython

[Question] Is this the intended behavior of brython-cli --modules? #1603