Closed smustgrave closed 2 years ago
Hey @smustgrave, sorry you're having trouble. I don't have a test project on docker container setup atm.
Does the environment meet all the requirements? Mainly Node v15, NPM, and gulp.
What does your gulpfile.js
look like?
Appears to be an issue with the log version function being asynchronous.
I'm just calling the buildsass command directly. I didn't copy the function or anything
Also didn't see a priority label or anything but an issue like this would be a no go for many projects being able to use this on their projects. A lot of projects are built out in containers, cirlceCI, etc now and if they can't build they can't deploy.
There's not enough information to add a priority label. Can you please answer the above and include some steps to reproduce?
We don't work in Docker environments in our day-to-day, but we definitely want to be able to support these users. If you have a solution, PRs are certainly welcome.
I'm on the latest version of nodejs, node, and gulp
This is my gulp file.
For testing I changed default to just run uswds.buildSass but no luck
/*
* * * * * ==============================
* * * * * ==============================
* * * * * ==============================
* * * * * ==============================
========================================
========================================
========================================
----------------------------------------
USWDS SASS GULPFILE
----------------------------------------
*/
const uswds = require("@uswds/compile");
// Custom variables
const pkg = require('./package.json');
const {series, watch} = require("gulp");
const gulp = require('gulp');
const sassLint = require('gulp-sass-lint');
const eslint = require('gulp-eslint');
const rename = require("gulp-rename");
const path = require("path");
let uglify = require('gulp-uglify-es').default;
const del = require("del");
/**
* USWDS version
*/
uswds.settings.version = 2;
/**
* Path settings
* Set as many as you need
*/
// Where to output
uswds.paths.dist.css = './css';
// Custom scss files
uswds.paths.dist.theme = './components/**';
uswds.paths.dist.fonts = './assets/fonts';
uswds.paths.dist.img = './assets/img';
uswds.paths.dist.js = './assets/js';
uswds.paths.src.projectSass = './components/**';
// Custom Functions
function cleanJs() {
return del([pkg.paths.dist.js]);
}
function cleanCss() {
return del([pkg.paths.dist.css]);
}
function scssLint() {
return gulp.src(pkg.paths.scss)
.pipe(sassLint())
.pipe(sassLint.format())
}
// After we let uswds compile build the css files we need to minify them.
// todo should we cleanup css output folder structure?
function minifyCss() {
return gulp.src(pkg.paths.css)
// .pipe(rename(function (file) {
// // this removes the last parent directory of the relative file path
// let parts = file.dirname.split("/");
// parts = parts.slice(1);
// parts = parts.join("/");
// file.dirname = path.dirname(parts + "/" + file.basename);
// // console.log("file.dirname = " + file.dirname);
// }))
.pipe(rename({"suffix": '.min'}))
.pipe(gulp.dest(pkg.paths.dist.css));
}
function buildJs() {
return gulp.src(pkg.paths.js)
.pipe(uglify())
.pipe(rename(function (file) {
// this removes the last parent directory of the relative file path
let parts = file.dirname.split("/");
parts = parts.slice(1);
parts = parts.join("/");
file.dirname = path.dirname(parts + "/" + file.basename);
// console.log("file.dirname = " + file.dirname);
}))
.pipe(rename({"suffix": '.min'}))
.pipe(gulp.dest(pkg.paths.dist.js));
}
function jsLint() {
return gulp.src(pkg.paths.js)
.pipe(eslint())
.pipe(eslint.format())
.pipe(eslint.failAfterError());
}
/**
* Exports
* Add as many as you need
*/
// copyAll + compile
exports.init = uswds.init;
exports.compileSass = series(cleanCss, scssLint, uswds.compileSass, minifyCss);
exports.compileJs = series(cleanJs, jsLint, buildJs);
exports.compileAll = series(this.compileSass, this.compileJs);
exports.lint = series(scssLint, jsLint);
exports.watch = series(this.compileAll, () => {
watch(pkg.paths.scss, series([this.compileSass]));
watch(pkg.paths.js, series([this.compileJs]));
});
exports.updateUswds = uswds.updateUswds;
exports.default = this.compileAll;
As far as the steps go
When instead my docker container if I try and run gulp to buildSass it fails
If I add add async to function buildSass like async function buildSass()
it works but that just doesn't feel right
Okay, thanks for providing that information. We'll try to take a look as soon as we can. Probably next week is more realistic.
Sounds good. I'll try and look too but will admit my asynchronous coding is light. I'm available to test anything too as we have a dozen or so projects that we will want to update
Took a shot but still seems not to be working within containers.
Tracked down the issue to this line .pipe( sass({ includePaths: buildSettings.includes }) .on("error", handleError) )
So far I've tested and haven't been able to replicate. I've tested the following without issues:
wodby/php:7.4-dev-4.20.2
with node 14.14.3The only issue I encountered was missing phantomJS because I was using node 14.
I've reached out to a coworker who is testing USWDS Compile on a lando project. I don't expect issues since Lando uses docker, but I'll report back when he is able to test and run.
CC @thisisdano
We had to patch this in order to get it working. Also includes changes for https://github.com/uswds/uswds-compile/issues/19
@smustgrave could you please paste the text via comment? For security concerns, we prefer not to open attachments.
I wasn't able to replicate the issue with the official docker php image. Could you provide more information on the image that you're using as well as the version of node?
Thank you
node v17.9.0 npm 8.9.0
Just retested and still happening
Patch file is
diff --git a/node_modules/@uswds/compile/gulpfile.js b/node_modules/@uswds/compile/gulpfile.js
index dde3ee8..9653e3e 100644
--- a/node_modules/@uswds/compile/gulpfile.js
+++ b/node_modules/@uswds/compile/gulpfile.js
@@ -3,7 +3,7 @@ const csso = require("postcss-csso");
const { src, dest, series, parallel, watch } = require("gulp");
const postcss = require("gulp-postcss");
const replace = require("gulp-replace");
-const sass = require("gulp-sass")(require("sass-embedded"));
+const sass = require('gulp-sass')(require('sass'));
const sourcemaps = require("gulp-sourcemaps");
const del = require("del");
const svgSprite = require("gulp-svg-sprite");
@@ -52,11 +52,12 @@ let settings = {
* - all paths are relative to the project root
*/
dist: {
- theme: "./sass",
+ scss: "./assets/uwds/scss",
img: "./assets/uswds/img",
fonts: "./assets/uswds/fonts",
js: "./assets/uswds/js",
css: "./assets/uswds/css",
+ custom: "./sass"
},
},
browserslist: [
@@ -95,9 +96,9 @@ USWDS specific tasks
*/
const copy = {
- theme() {
- log(colorBlue, `Copy USWDS theme files: ${getSrcFrom("theme")} → ${paths.dist.theme}`);
- return src(`${getSrcFrom("theme")}/**/**`.replaceAll("//", "/")).pipe(dest(paths.dist.theme));
+ scss() {
+ log(colorBlue, `Copy USWDS scss files: ${getSrcFrom("theme")} → ${paths.dist.scss}`);
+ return src(`${getSrcFrom("theme")}/**/**`.replaceAll("//", "/")).pipe(dest(paths.dist.scss));
},
fonts() {
log(colorBlue, `Copy USWDS fonts: ${getSrcFrom("fonts")} → ${paths.dist.fonts}`);
@@ -149,37 +150,39 @@ function buildSass() {
],
includes: [
// 1. local theme files
- paths.dist.theme,
- // 2. uswds organization directory (npm packages)
+ `${paths.dist.custom}/*.scss`,
+ // 2. uswds css files
+ paths.dist.scss,
+ // 3. uswds organization directory (npm packages)
getSrcFrom("uswds"),
- // 3. v2 packages directory
+ // 4. v2 packages directory
`${getSrcFrom("sass")}/packages`.replaceAll("//", "/"),
- // 4. local uswds package
+ // 5. local uswds package
getSrcFrom("sass")
],
};
return (
- src([`${paths.dist.theme}/*.scss`.replaceAll("//", "/")])
- .pipe(sourcemaps.init({ largeFile: true }))
- .pipe(
- sass({ includePaths: buildSettings.includes })
- .on("error", handleError)
- )
- .pipe(replace(/\buswds @version\b/g, `based on uswds v${pkg}`))
- .pipe(postcss(buildSettings.plugins))
- .pipe(sourcemaps.write("."))
- .pipe(dest(paths.dist.css))
+ src([`${paths.dist.custom}/*.scss`.replaceAll("//", "/")])
+ .pipe(sourcemaps.init({ largeFile: true }))
+ .pipe(
+ sass.sync({ includePaths: buildSettings.includes })
+ .on("error", handleError)
+ )
+ .pipe(replace(/\buswds @version\b/g, `based on uswds v${pkg}`))
+ .pipe(postcss(buildSettings.plugins))
+ .pipe(sourcemaps.write("."))
+ .pipe(dest(paths.dist.css))
);
}
function watchSass() {
return watch(
- [
- `${paths.dist.theme}/**/*.scss`.replaceAll("//", "/"),
- `${paths.src.projectSass}/**/*.scss`.replaceAll("//", "/")
- ], buildSass);
-};
+ [
+ `${paths.dist.scss}/**/*.scss`.replaceAll("//", "/"),
+ `${paths.src.projectSass}/**/*.scss`.replaceAll("//", "/")
+ ], buildSass);
+}
function buildSprite() {
const config = {
@@ -205,17 +208,17 @@ function buildSprite() {
return src(`${paths.dist.img}/usa-icons/**/*.svg`.replaceAll("//", "/"), {
allowEmpty: true,
})
- .pipe(svgSprite(config))
- .on("error", handleError)
- .pipe(dest(`${paths.dist.img}`));
+ .pipe(svgSprite(config))
+ .on("error", handleError)
+ .pipe(dest(`${paths.dist.img}`));
}
function renameSprite() {
return src(`${paths.dist.img}/symbol/svg/sprite.symbol.svg`.replaceAll("//", "/"), {
allowEmpty: true,
})
- .pipe(rename(`${paths.dist.img}/sprite.svg`.replaceAll("//", "/")))
- .pipe(dest(`./`));
+ .pipe(rename(`${paths.dist.img}/sprite.svg`.replaceAll("//", "/")))
+ .pipe(dest(`./`));
}
function cleanSprite() {
@@ -224,31 +227,31 @@ function cleanSprite() {
exports.settings = settings;
exports.paths = paths;
-exports.copyTheme = copy.theme;
+exports.copyScss = copy.scss;
exports.copyFonts = copy.fonts;
exports.copyImages = copy.images;
exports.copyJS = copy.js;
exports.copyAssets = series(
- copy.fonts,
- copy.images,
- copy.js
+ copy.fonts,
+ copy.images,
+ copy.js,
+ copy.scss,
);
exports.copyAll = series(
- copy.theme,
- this.copyAssets
+ this.copyAssets
);
exports.compileSass = series(logVersion, buildSass);
exports.compileIcons = series(buildSprite, renameSprite, cleanSprite);
exports.compile = series(
- logVersion,
- parallel(
- buildSass,
- this.compileIcons
- )
+ logVersion,
+ parallel(
+ buildSass,
+ this.compileIcons
+ )
);
exports.updateUswds = series(
- this.copyAssets,
- this.compile
+ this.copyAssets,
+ this.compile
);
exports.init = series(logVersion, this.copyAll, this.compile);
In the PR I changed sass-embedded to just sass package and it works
I tested out USWDS 3 with Lando and it works fine for me right off the bat. @smustgrave, here are a few things as to why your setup might not be working:
name: uswds3
recipe: drupal9
services:
database:
type: mariadb:10.3
node:
type: 'node:16'
scanner: false
ssl: true
sslExpose: false
globals:
gulp-cli: latest
overrides:
ports:
- '32854:32854'
command: tail -f /dev/null && cd /app/web/themes/custom/uswds3 && npm install
appserver:
build:
- composer install
xdebug: true
config:
php: lando/config/php.ini
type: php:7.4
overrides:
environment:
DRUSH_OPTIONS_URI: 'http://uswds3.lndo.site'
config:
composer_version: 2-latest
via: nginx
database: mysql:5.7
webroot: ./web
php: 7.4
xdebug: true
config:
php: lando/config/php.ini
# Add additional tooling
tooling:
node:
service: node
npm:
service: node
npx:
service: node
gulp:
service: node
dir: '/app/web/themes/custom/uswds3'
cmd: gulp watch
gulp-compile:
service: node
dir: '/app/web/themes/custom/uswds3'
cmd: npx gulp compile
gulp-watch:
service: node
dir: '/app/web/themes/custom/uswds3'
cmd: npx gulp watch
gulp-init:
service: node
dir: '/app/web/themes/custom/uswds3'
cmd: npx gulp init
build:
description: "Install NPMs"
cmd:
- node: './lando/build/theme.sh'
events:
post-rebuild:
- node: './lando/build/theme.sh'
in my theme.sh file, I have
#!/usr/bin/env bash
cd web/themes/custom/uswds3
rm -R node_modules
npm install
After a lando rebuild -y
I can run these commands
lando gulp-init
lando gulp-compile
Other than that, I used some very basic settings in my gulp file, no patches or anything like that needed.
const uswds = require("@uswds/compile");
/**
* USWDS version
*/
// Use version 3.
uswds.settings.version = 3;
/**
* Path settings
* Set as many as you need
*/
uswds.paths.dist.css = './assets/css';
uswds.paths.dist.theme = './src/sass';
uswds.paths.dist.img = './src/assets/uswds/img';
uswds.paths.dist.fonts = './src/assets/uswds/fonts';
/**
* Exports
* Add as many as you need
*/
// Init project
exports.init = uswds.init;
// Compile function.
exports.compile = uswds.compile;
exports.watch = uswds.watch;
exports.update = uswds.updateUswds;
exports.default = uswds.watch;
exports.copyAssets = uswds.copyAssets;
I would suggest trying a reduced test case like I have here to see if that fixes it.
Noting that I am on Lando v3.6.2 and Docker desktop for Mac using the exact version that comes bundled with Lando 3.6.2. If you are using Docker 4.x, that could also cause serious side effects as well as other versions of docker that are not the exact one to come bundled with Lando.
Using lando v3.6.4 and docker 4.8 Not sure my macbook will work properly with a lower version like that. Docker requirements page says I would need at least 4.3
We aren't on php 7.4 either. Most of our projects were fortunate and updated to php8
I tested a simple compile before without anything else and still get the failure.
I'll try downgrading node to v16.
But changing sass-embedded to 'sass' fixes the asynchronous issue
No luck
[18:18:21] Finished 'js' after 134 ms
[18:18:21] Starting 'logVersion'...
uswds.version: 2
[18:18:21] Finished 'logVersion' after 403 μs
[18:18:21] Starting 'buildSass'...
Compiling with USWDS 2.13.2
[18:18:21] Starting 'buildSprite'...
[18:18:24] Finished 'buildSprite' after 2.55 s
[18:18:24] Starting 'renameSprite'...
[18:18:24] Finished 'renameSprite' after 19 ms
[18:18:24] Starting 'cleanSprite'...
[18:18:24] Finished 'cleanSprite' after 19 ms
[18:18:24] The following tasks did not complete: default,
Using lando v3.6.4 and docker 4.8 Not sure my macbook will work properly with a lower version like that.
@smustgrave That is your issue right there (docker 4.8), please see the Lando support docs
https://docs.lando.dev/getting-started/installation.html#macos
When going through the installer you can choose to not install Docker Desktop, although we recommend you use the version of Docker Desktop that we ship for compatibility and support reasons!
cc / @mejiaj
I'm reluctant to downgrade that far back. And I'm the new Apple chip so sure it would break something.
Been using this setup for a year now and this has been the only package to ever cause this issue which also gives me pause.
This is our lando file
name: test
recipe: drupal9
config:
webroot: webroot
php: '8.0'
via: nginx
xdebug: true
config:
php: lando-src/php.ini
services:
appserver:
overrides:
ports:
- "0.0.0.0::80"
environment:
PHP_IDE_CONFIG: "serverName=test"
LANDO_HOST_IP: "host.docker.internal"
DRUSH_OPTIONS_ROOT: '/app/webroot'
DRUSH_OPTIONS_URI: 'http://test.lndo.site'
PHP_SENDMAIL_PATH: '/usr/sbin/sendmail -S mailhog:1025'
build:
- /usr/local/bin/composer global require --dev drupal/coder
- COMPOSER_PROCESS_TIMEOUT=1200 /usr/local/bin/composer install --working-dir=/app
build_as_root:
- apt-get update
- curl -sL https://deb.nodesource.com/setup_16.x | bash -
- apt-get install -y nodejs python3-pip clamav clamav-daemon rsyslog
- npm install -g npm@latest
- npm install gulp-cli gulp -g
- wget https://robo.li/robo.phar
- chmod +x robo.phar && mv robo.phar /usr/bin/robo
- pip3 install git-remote-codecommit
- echo "TCPSocket 3310" >> /etc/clamav/clamd.conf
- freshclam
- update-rc.d clamav-daemon enable
run_as_root:
- service clamav-daemon start
- service rsyslog start
- chmod 777 /app/lando-src/config/rsyslog.conf
- cat /app/lando-src/config/rsyslog.conf >> /etc/rsyslog.conf
database:
type: mariadb
portforward: 33308
redis:
type: redis
portforward: true
search:
type: solr:8.11
portforward: true
core: test
config:
dir: lando-src/solr-primary
build_as_root:
- mkdir -p /opt/solr/server/solr/mycores/test/conf
- chown -R solr:solr /opt/solr/server/solr/mycores
cypress:
type: node:custom
app_mount: false
overrides:
environment:
CYPRESS_DRUPAL_USER: admin
CYPRESS_DRUPAL_PASS: admin
image: 'mobomo/cypress'
volumes:
- ./tests/integration:/app/cypress/integration/custom
- ./tests/cypress.local.json:/app/cypress.json
- ./tests/videos:/app/cypress/videos
mailhog:
type: mailhog
portforward: true
hogfrom:
- appserver
proxy:
mailhog:
- mail.lndo.site
tooling:
logs-drupal:
service: appserver
description: Displays and tails Drupal logs using syslog module (because drush wd-show no longer supports tail)
cmd:
- /app/lando-src/scripts/logs-drupal.sh
user: root
robo:
service: appserver
description: Runs <robo> commmands
cmd: robo
npm:
service: appserver
node:
service: appserver
gulp:
service: appserver
xdebug-on:
service: appserver
description: Enable xdebug
cmd: "docker-php-ext-enable xdebug && pkill -o -USR2 php-fpm"
user: root
xdebug-off:
service: appserver
description: Disable xdebug
cmd: "rm /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && pkill -o -USR2 php-fpm"
user: root
test:
service: appserver
cmd: "php /app/vendor/bin/phpunit -c /app/phpunit.xml"
test-full:
service: appserver
cmd: "php /app/vendor/bin/phpunit -c /app/webroot/core/phpunit.xml.dist"
blt:
service: appserver
cmd: /app/vendor/bin/blt
cypress:
service: cypress
cmd: export DISPLAY="host.docker.internal:0" && cypress open --env TAGS='@e2e-test'
env_file:
- .lando.env
- .secrets.env
excludes:
- vendor
@smustgrave My 2 cents FWW, I'd suggest working with Lando support. Noting that when they discover that you are using an unsupported version of docker, they probably will not be willing to help you as they state in the link I posted above. In the meantime, you can have a look at https://github.com/lando/lando/issues/3370
I'll downgrade and see if that fixes the issue.
Still getting the error after downgrading docker desktop.
Copied your gulpfile.js too but no luck
Could this be releated? https://github.com/sass/embedded-host-node/issues/143 if sass-embedded is having issues in docker.
Know this is a pain so definitely appreciate looking into it.
May be unrelated. But I tried changing the gulp command from
exports.compile
to exports.compileUswds
as I think compile was being called by something else. When I follow it with phpstorm (holding cmd and clicking on it) I would expect compile to point to exports.compile but instead if goes to node_modules/sass/types/compile.t.ts and I changed all references and I get a different error now
[20:26:09] Starting 'buildSprite'... [20:26:10] 'buildSass' errored after 233 ms [20:26:10] Error: write EPIPE at afterWriteDispatched (node:internal/stream_base_commons:160:15) at writeGeneric (node:internal/stream_base_commons:151:3) at Socket._writeGeneric (node:net:817:11) at Socket._write (node:net:829:8) at writeOrBuffer (node:internal/streams/writable:389:12) at _write (node:internal/streams/writable:330:10) at Socket.Writable.write (node:internal/streams/writable:334:10) at AsyncEmbeddedCompiler.writeStdin (/app/web/themes/custom/uswds-drupal-theme/node_modules/sass-embedded/dist/lib/src/async-compiler.js:34:28) at PacketTransformer.writeInboundBuffer (/app/web/themes/custom/uswds-drupal-theme/node_modules/sass-embedded/dist/lib/src/compile.js:106:30) at PacketTransformer.writeInboundProtobuf (/app/web/themes/custom/uswds-drupal-theme/node_modules/sass-embedded/dist/lib/src/packet-transformer.js:66:18) [20:26:10] 'init' errored after 11 s
@danny-englander
Think adding async to the front of all the functions does also fix the issue.
async wasn't needed when switching sass-embedded with sass.
@smustgrave so if I followed along correctly you have a setup that's in the current state:
Is that correct? And your tasks are failing for USWDS/Compile? Do you have any custom tasks? Do those work fine?
This issue seems out of scope for us. Danny's verified it in his lando setup, I've tried a docker container, we build on several docker images via circleci. If you can create a repo with your setup maybe we can try to debug.
that is all correct. I downgraded everything but seem to be able to compile with sass-embedded being used. All other tasks work fine just the compile task. Seems someone else also reported this https://github.com/uswds/uswds-compile/issues/28
It may be an issue with the kind of image being used but not sure. I know
Issue being https://github.com/sass/embedded-host-node/issues/143
Something that may be the issue. I'm using a macbook with the M1 chip. Another team member is on Linux. Ubuntu 22.04 and works fine for her.
Thoughts @thisisdano?
If you do option2 is it possible to use a settings.local file that could be ignored. So if I use sass and the rest of my team uses sass-embedded we could have the same gulpFile but slightly different settings. If that makes sense.
@smustgrave you can import the tasks you want/need. For option 2, you could just have a custom SASS task instead. Which it sounds like you already have?
Closing this issue in favor of https://github.com/uswds/uswds-compile/issues/32.
Attempting to run this in docker generates
[01:01:51] Starting 'default'... [01:01:51] Starting 'logVersion'... uswds.version: 2 [01:01:51] Finished 'logVersion' after 1.27 ms [01:01:51] Starting 'buildSass'... Compiling with USWDS 2.13.2 [01:01:51] The following tasks did not complete: default, buildSass [01:01:51] Did you forget to signal async completion?