Improvements to dev environment

[bbusenius]

Fixes #605

Changes in this request

• Fixes nodejs installation.
• Makes the source lw/bin/activate and cd /vagrant/ commands run automatically after vagrant ssh.
• Adds linting with prettier for React Javascript and gets base airbnb linting style rules working (previously was configured but the dependencies weren't installed). This is added to eslint which was working.
• Makes all code linting tools available in the Vagrant environment so they can be used there or checked against there.
• Adds vim setup for linting tools.
• Removes the yapf config.
• Adds black, autopep8, isort, and flake8 to the dev environment for Python linting.
• Adds eslint and prettier to the dev environment for React Javascript linting.
• Adds curlylint and djhtml to the dev environment for html template linting.
• Swaps out the deprecated babel-eslint parser for the new @babel/eslint-parser for Javascript.
• Makes static directories if they don't already exist to prevent "first time up" errors.
• Update the ITEM_SERVLET config base.py so it works in dev. It had the old setting and was being overridden in production by local.py during our switch to Folio.
• Makes the Vagrant build fail if any command fails by adding set -e to the beginning of the bash script.
• Upgrades to Webpack 5.75.0 and React 18.2.0.
• Fixes React code syntax to work with the newer version of React and gets our build tools working with the new code.
• Replaces the now unsupported react-render-html package with the htmr package.

I didn't do anything with elasticsearch, somehow this was just working. This means you should be able to remove the WAGTAILSEARCH_BACKENDS setting that points to the database search backend from your local.py if you're doing that. I also didn't do anything with the update_index command. It had already been added.

Template Linting The two tools I installed for this are curlylint and djhtml. I had already been looking at these and then saw that they're also used by the Wagtail project, so that sealed the deal for me.

Curlylint This is used to catch actual html and template tag errors, unclosed tags, etc. You can run it like this: curlylint --parse-only path/to/template_file.html

DJhtml This is mostly just a code formatter to enforce proper indentation. It doesn't do much else. It can be run like this: djhtml -i path/to/template_file.html

CGIMail Forms After some discussion and thought, I decided not to bring CGIMail into the dev environment. Though this is possible, Vagrant would have to install, setup, and run Apache for this. This would add more time and complication to the already long provisioning and I don't think it's necessary. That being said, I fixed the related ITEM_SERVLET setting that CGIMail uses and recreated my original development workflow so we can troubleshoot CGIMail forms in dev if needed. To test: http://wwwdev:8000/about/directory/staff/jean-luc-picard/test-cgimail-form/?bib=8068035&barcode=099422401

Add the following configurations to your local.py.

• SECURE_REFERRER_POLICY = 'origin'
• CGI_MAIL_SERVICE = 'https://www.lib.uchicago.edu/cgi-bin/cgimail/cgimail'

I'm not sure if we want to keep the SECURE_REFERRER_POLICY setting around, so it might be prudent to comment it out if you're not doing something CGIMail form related. This setting ensures that all requests get an HTTP Referer, which CGIMail requires. Alternatively to using this setting, you could use a browser plugin such as Tamper Data to add the header manually.

Linting with VIM inside the Vagrant environment VIM uses the ALE plugin to do Python and React linting. If you want to test this, open a file like so: vim base/models.py. Once in the file, you should see any linting errors highlighted. If you want to apply linting and styling fixes you can type :ALEFix. This will apply the fixers.

You can do the same with React code, e.g. vim lib_news/static/lib_news/js/NewsFeed.js. In this case ALE is a little slow to show the changes. For Javascript ALE is using eslint and prettier.

[bufordrat]

autopep8, isort, black, and flake8 are all working, both in the shell and via vim. w00t

[bufordrat]

curlylint and djhtml get a check mark too! I think the only thing left to test other than the npm stuff (currently ongoing) is the set -e erroring out...

[bufordrat]

Check it!

(lw) vagrant@ubuntu-jammy:/vagrant$ type eslint
eslint is /vagrant/node_modules/.bin/eslint

[bufordrat]

The linter seems like it isn't "project-aware" when I run it on individual .js files. For example:

(lw) vagrant@ubuntu-jammy:/vagrant/base/static/base/js$ eslint --fix transformer-tabs.js

/vagrant/base/static/base/js/transformer-tabs.js
  ...
  67:7   error  'propagate' is assigned a value but never used   no-unused-vars
  ...

Presumably it wouldn't think propagate was unused if it could see the html that invoked it, but in order to do that it'd have to be able to understand Django templates.

As I recall, this is a problem you flagged already; I think you said eslint is really only set up to understand stuff that falls under the structure of a React project. In this case, I think we'd need the linter to somehow be aware of a Django project, which feels like kind of advanced functionality.

[bufordrat]

Ok, absolutely everything seems to be working now in my fresh Vagrant image.