add-project-script missing in (dev) dependencies?

nerdoc commented 3 years ago

Describe the bug

manage.py initial_sockpuppet throws error during tutorial https://sockpuppet.argpar.se/setup-django

To Reproduce

Just follow the instructions on https://sockpuppet.argpar.se/setup-django

Expected behavior

There should be no errors

Screenshots or reproduction

./manage.py initial_sockpuppet
Wrote to /home/christian/Projekte/reflex/package.json:

{
  "name": "reflex",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "Christian González <...@...at>",
  "license": "ISC"
}

/home/christian/.nvm/versions/node/v13.6.0/bin/add-project-script -> /home/christian/.nvm/versions/node/v13.6.0/lib/node_modules/add-project-script/dist/index.js
+ add-project-script@1.0.2
added 11 packages from 6 contributors in 1.014s
/bin/sh: Zeile 1: add-project-script: Kommando nicht gefunden.
Build and watch already in package.json, so skipping these
removed 11 packages in 0.199s

Maybe add-project-script is missing and should be installed before (globally?) - this should be mentioned in the setup.

Versions

Django-sockpuppet

pypi: 0.5.0

External tools

Python: [e.g. 3.9]
Django: [e.g. 3.5.1]
OS: [Linux]

jonathan-s commented 3 years ago

The intention that add-project-script would be installed in initial_sockpuppet as you can see here -> https://github.com/jonathan-s/django-sockpuppet/blob/master/sockpuppet/management/commands/initial_sockpuppet.py#L13-L16

Not sure exactly what is happening on your machine. If you find out it'd be helpful!

nerdoc commented 3 years ago

ok, seems ok. I think the problem is on my machine. I tried to install npm install -g add-project-script directly, and got the same result. If I start a "Js venv" with `nvm use stable", it installs ok. So this is my fault for sure here, I somehow certainly messed up the npm env. but there are a few other small problems, I'll open another issue for that.

nerdoc commented 3 years ago

Hm, it's not as easy as I thought. There might be a bug/enhancement in django-suckpuppet as well, so maybe I'd like to reopen this bug, If it's ok for you. the script seems to have problems installing add-project-script globally with npm. I tried it with more a nvm environments, and the system node package, no success. I always get the error that it is already installed, and can't overwrite it.

npm ERR! EEXIST: file already exists, symlink '../lib/node_modules/add-project-script/dist/index.js' -> 
'/home/christian/.nvm/versions/node/v13.6.0/bin/add-project-script'.

I think the problem is that the script assumes that add-project-script is not installed. And if it is, if fails. a --force flag to npm (like it suggests it) helps and fixes my problem. May I suggest to add this to the script?

jonathan-s commented 3 years ago

That looks fine to me. There's a couple of failures in the tests (which I'd say is unrelated to your change, flakey tests). So I'll have a look into that myself.

You mentioned that you are somewhat of a novice, and a lot of people think the entire javascript/webpack kerfuffle is quite a hassle.

So I wanted to ask you.

To what extent does this workflow so far help you? (aside from the errors)?
Would you want to have an option to be able to use yarn instead?
Is it clear that you don't need webpack/npm (although there will be certain things you won't be able to do).
If you were to compare it to https://github.com/hotwire-django/stimulus-webpack-tools which works in similar ways, is that more helpful? (in that case it might be worth implementing some of that functionality here).

And thanks for taking the time to open an issue in the first place!

nerdoc commented 3 years ago

That looks fine to me. There's a couple of failures in the tests (which I'd say is unrelated to your change, flakey tests). So I'll have a look into that myself.

You mentioned that you are somewhat of a novice, and a lot of people think the entire javascript/webpack kerfuffle is quite a hassle.

Definitely. I spent months figuring out the "perfect" system with Django and Bootstrap, Django and Vue, Django and Svelte, Alpine.js, velocity, with Webpack, Rollup, Snowpack etc. in many combinations. Proxying the packing dev server through a middleware into the Django dev server, redirecting content, etc. etc. I feel somehow overwhelmed and the only essence I got out of it: technology is not far evolved enough yet. Everything half-baked and nearly-there. No solution could convince me really. Django & sockpuppet was the first thing that impressed me deeply - because I EVER wanted to keep the Django templates, which allow server side packaging of frontends in different apps. I am writing a Django plugin system (as Django apps are not pluggable): GDAPS - and wanted to implement an easy way to include frontends into Django - so I created a mgmt command like yours, to move some frontend files into place (using Django templates here too, instead of e.g. cookiecutter), and calling vue CLI, npm and similar Js/node commands from within Django. So I have some little experience with that, and I ran into some errors because the node env was not detected properly.

My frontend "generators" are pluggable - Vue is just one of them, Svelte should follow, but even Qt/PySide etc should be possible. My aim is that every GDAPS plugin (= a Django app which could be installed via pip, and is running instantly in tha current Django application) contains it's frontend package too But I NEVER got it right (just asked at the django-developers mailing list to get some clues) to have it both: Django templates, AND e.g. own webcomponents in the frontend to use components based development - and I NEVER got it to work yet. Until I found Dj-sockpuppet... ;-) Seems very nice to me. So maybe I'll ditch my GDAPS frontend part completely and just keep the backend part (the plugin system itself).

So I wanted to ask you.

* To what extent does this workflow so far help you? (aside from the errors)?

You mean the initial_sockpuppet mgmt command? This is really great! Only documentation is a bit confusing here... Maybe I can help you here too a bit. From a (dj-sockpuppet) "beginners" view it could be made "easier" to follow.

I would rename the command init_sockpuppet - or longer initialize_sockpuppet.
It should be more error tolerant. ATM it breaks/stops if errors occur - there could be done more within try:except blocks (npm not available, etc).

* Would you want to have an option to be able to use yarn instead?

Definitely. Have a look at GDAPS, I created "pluggable" Classes for that purpose: https://gitlab.com/nerdocs/gdaps/-/blob/master/gdaps/frontend/pkgmgr/__init__.py#L31 You can use them (and adapt to your needs) it you want. Don't mind the AGPL, I hereby relicense these Classes under the MIT ;-)

* Is it clear that you don't need webpack/npm (although there will be certain things you won't be able to do).

No, not clear. Normally the workflow is: webpack build -> manage.py collectstatic -> serve for production. But for development with HMR this is really hard to achieve IMHO. Things like django-webpack-loader and webpack-bundle-tracker are working well, but are IMHO very compicated to configure if you start from zero. No sane defaults, no easy config. AND worst: No use of Django templates. AND: webpack is bloated and slow. Compared to e.g. snowpack.

* If you were to compare it to https://github.com/hotwire-django/stimulus-webpack-tools which works in similar ways, is that more helpful? (in that case it might be worth implementing some of that functionality here).

Didn't know it - but reading the README seems very clear to me, yes. I integrated parts of that workflow in GDAPS.fronend too: I created e.g vue-extensions and integrated them into the GDAPS backends similarly to the stimulus-webpack-tools' webpack --bundle command. I think there is no other way to assemble distributed js snippets/entry points from different Django apps into one frontend.

And thanks for taking the time to open an issue in the first place!

You're welcome.

nerdoc commented 3 years ago

I'm a bit clearer now, regarding the Javascript/bundler topic. To answer your question about "is it clear you don't need webpack/npm" etc to do certain things" more exactly (what are these "certain things"):

From a beginner's view (I'm not really a beginner, but not very fluent with Js/bundlers like webpack), it's hard to wrap one's head arount how things work with sockpuppet. All other solutions work the other way round. And again: I've searched more than a few years for this almost-perfect solution, using Django templates, and "responsive" UI with states kept at the backend.

But my main problem here - and this is where more docs would be very helpful - is, how to combine webpack (rollup, parcel, snowpack etc.) with django-sockpuppet. Because there is no easy way to combine their develeopment server output with sockpuppet... in a way I know of. I don't want to make a SPA, I'd like to keep state in Django/Python and use Dj.templates, and URL routing in Django. But what is terribly needed is something like webcomponents, instead of doing repetitive blocks with divs/classes/styles again and again. You can create some easily with e.g. Svelte, but you have to compile them first. But How do I get them in to the mix with sockpuppet? There is a wc library named Shoelace, which works fine when including it using a CDN. But for applications that should work in a contained env (like an Intranet, without direct web access, or without wanting it to access external ressources) you hve to install this libs locally and bundle them. But how do I bundle eg. Svelte, or even Vue.js, or Shoelace, with a bundler, use their components (or own built webcomponents) - AND use django-sockpuppet? Especially forms are not easy - they are completely different. Using dj-sp, you should use a normal <form> tag, with e.g. Shoelace there is a<sl-form> tag that behaves different, and data must be uploaded manually by Javascript.

If you give me a hint here, I could try to improve the dj-sp docs here too - if this is desired.

jonathan-s / django-sockpuppet