lefou
commented
3 years ago Without further inspection, I clearly see that /nonexistent/bin is the last part of your PATH variable.
Closed yurivict closed 2 years ago
lefou
commented
3 years ago Without further inspection, I clearly see that /nonexistent/bin is the last part of your PATH variable.
yurivict
commented
3 years ago Without further inspection, I clearly see that /nonexistent/bin is the last part of your PATH variable.
I see. This is because the user has no home directory and it is set to /nonexistent in the users db. This somehow gets into PATH.
Bad paths should be silently ignored.
This looks like a bug in mill that it doesn't ignore bad paths.
yurivict
commented
3 years ago Even after removing /nonexistent from the PATH variable it still fails the same way.
lefou
commented
3 years ago What's your $HOME env variable? Is it accessible? We (ammonite, to be exact) probably try to write into our cache location which is located under it. You may try to pass an accessible dir with --home to mill.
yurivict
commented
3 years ago HOME is set to some other directory that exists.
lefou
commented
3 years ago You may try to pass an accessible dir with
--hometo mill.
@yurivict Have you tried that?
You may also provide some more context/info. E.g. output of env, mill --version, your actual invocation of mill, the build.sc...
yurivict
commented
3 years ago You may try to pass an accessible dir with --home to mill.
This didn't help.
You may also provide some more context/info. E.g. output of env, mill --version, your actual invocation of mill, the build.sc...
Invocation:
cd /wrkdirs/usr/ports/math/yuck/work/yuck-20210501 && echo /usr/bin/env XDG_CACHE_HOME=/wrkdirs/usr/ports/math/yuck/work XDG_DATA_HOME=/wrkdirs/usr/ports/math/yuck/work XDG_CONFIG_HOME=/wrkdirs/usr/ports/math/yuck/work HOME=/wrkdirs/usr/ports/math/yuck/work TMPDIR="/tmp" PATH=/wrkdirs/usr/ports/math/yuck/work/.bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local LOCALBASE=/usr/local CC="cc" CFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing " CPP="cpp" CPPFLAGS="" LDFLAGS=" -fstack-protector-strong " LIBS="" CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing " MANPREFIX="/usr/local" BSD_INSTALL_PROGRAM="install -s -m 555" BSD_INSTALL_LIB="install -s -m 0644" BSD_INSTALL_SCRIPT="install -m 555" BSD_INSTALL_DATA="install -m 0644" BSD_INSTALL_MAN="install -m 444" mill --home=/wrkdirs/usr/ports/math/yuck/work yuck.launcher
/usr/bin/env XDG_CACHE_HOME=/wrkdirs/usr/ports/math/yuck/work XDG_DATA_HOME=/wrkdirs/usr/ports/math/yuck/work XDG_CONFIG_HOME=/wrkdirs/usr/ports/math/yuck/work HOME=/wrkdirs/usr/ports/math/yuck/work TMPDIR=/tmp PATH=/wrkdirs/usr/ports/math/yuck/work/.bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local LOCALBASE=/usr/local CC=cc CFLAGS=-O2 -pipe -fstack-protector-strong -fno-strict-aliasing CPP=cpp CPPFLAGS= LDFLAGS= -fstack-protector-strong LIBS= CXX=c++ CXXFLAGS=-O2 -pipe -fstack-protector-strong -fno-strict-aliasing MANPREFIX=/usr/local BSD_INSTALL_PROGRAM=install -s -m 555 BSD_INSTALL_LIB=install -s -m 0644 BSD_INSTALL_SCRIPT=install -m 555 BSD_INSTALL_DATA=install -m 0644 BSD_INSTALL_MAN=install -m 444 mill --home=/wrkdirs/usr/ports/math/yuck/work yuck.launcherEnvironment:
BSD_INSTALL_MAN=install -m 444
BSD_INSTALL_DATA=install -m 0644
BSD_INSTALL_SCRIPT=install -m 555
BSD_INSTALL_LIB=install -s -m 0644
BSD_INSTALL_PROGRAM=install -s -m 555
MANPREFIX=/usr/local
CXXFLAGS=-O2 -pipe -fstack-protector-strong -fno-strict-aliasing
CXX=c++
LIBS=
LDFLAGS= -fstack-protector-strong
CPPFLAGS=
CPP=cpp
CFLAGS=-O2 -pipe -fstack-protector-strong -fno-strict-aliasing
CC=cc
PREFIX=/usr/local
NO_LINT=YES
MK_KERNEL_SYMBOLS=no
MK_DEBUG_FILES=no
NO_PIE=yes
PATH=/wrkdirs/usr/ports/math/yuck/work/.bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/bin
TMPDIR=/tmp
HOME=/wrkdirs/usr/ports/math/yuck/work
XDG_CONFIG_HOME=/wrkdirs/usr/ports/math/yuck/work
XDG_DATA_HOME=/wrkdirs/usr/ports/math/yuck/work
XDG_CACHE_HOME=/wrkdirs/usr/ports/math/yuck/work
STATUS=1
WARNING_WAIT=0
ARCH=amd64
OPSYS=FreeBSD
SAVED_TERM=xterm-256color
NO_WARNING_PKG_INSTALL_EOL=yes
MASTERMNT=/usr/local/poudriere/data/.m/12amd64-local-workstation/ref
LANG=C
MAKELEVEL=1
MAIL=/var/mail/nobody
UID=65534
META_MODE=normal
MAKEFLAGS= .MAKE.LEVEL.ENV=MAKELEVEL ARCH=amd64 HAVE_COMPAT_IA32_KERN=YES OPSYS=FreeBSD OSREL=12.2 OSVERSION=1202504 _OSRELEASE=12.2-STABLE
POUDRIERE_BUILD_TYPE=bulk
OSREL=12.2
OLDPWD=/usr/ports/math/yuck
.MAKE.LEVEL.ENV=MAKELEVEL
_OSRELEASE=12.2-STABLE
P_PORTS_FEATURES=FLAVORS SELECTED_OPTIONS
PWD=/wrkdirs/usr/ports/math/yuck/work/yuck-20210501
MASTERNAME=12amd64-local-workstation
HAVE_COMPAT_IA32_KERN=YES
SCRIPTPREFIX=/usr/local/share/poudriere
DEVELOPER_MODE=yes
USER=nobody
POUDRIERE_VERSION=3.3.6
SCRIPTPATH=/usr/local/share/poudriere/bulk.sh
SHELL=/bin/sh
STRICT_DEPENDS=yes
GID=65534
LIBEXECPREFIX=/usr/local/libexec/poudriere
PACKAGE_BUILDING=yes
LOCALBASE=/usr/local
LC_ALL=C
DEV_WARNING_WAIT=0
OSVERSION=1202504
BLOCKSIZE=K
POUDRIEREPATH=/usr/local/bin/poudrieremill --version fails:
mill --version
Exception in thread "main" java.io.FileNotFoundException: out/mill-worker-n4LtYH4K4wIkt4sfoAH5mhwFG6E=-1/pid (No such file or directory)
at java.io.RandomAccessFile.open0(Native Method)
at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:124)
at mill.main.client.FileLock.<init>(Locks.java:56)
at mill.main.client.Locks$1.<init>(Locks.java:15)
at mill.main.client.Locks.files(Locks.java:14)
at mill.main.client.MillClientMain.main0(MillClientMain.java:130)
at mill.main.client.MillClientMain.main(MillClientMain.java:102)$ cat build.sc
// build.sc
import mill._
import mill.define.Sources
import scalalib._
import $ivy.`com.lihaoyi::mill-contrib-buildinfo:$MILL_VERSION`
import mill.contrib.buildinfo.BuildInfo
object yuck extends ScalaModule with BuildInfo {
def git(args: String*) = os.proc("git" :: args.toList).call().out.lines.head
def retrieveGitCommitHash() = T.command {git("rev-parse", "HEAD")}
def gitCommitHash = T {retrieveGitCommitHash()}
def retrieveGitCommitDate() = T.command {git("log", "-1", "--pretty=format:%cd", "--date=format:%Y%m%d")}
def gitCommitDate = T {retrieveGitCommitDate()}
def retrieveGitBranch() = T.command {git("rev-parse", "--abbrev-ref", "HEAD")}
def gitBranch = T {retrieveGitBranch()}
def shortVersion = T {gitCommitDate()}
def longVersion = T {"%s-%s-%s".format(gitCommitDate(), gitBranch().replaceAll("/", "-"), gitCommitHash().take(8))}
def version = T {if (gitBranch() == "master") shortVersion() else longVersion()}
override def buildInfoPackageName = Some("yuck")
override def buildInfoMembers: T[Map[String, String]] = T {
Map(
"gitBranch" -> gitBranch(),
"gitCommitDate" -> gitCommitDate(),
"gitCommitHash" -> gitCommitHash(),
"version" -> version()
)
}
override def scalaVersion = "2.13.5"
override def millSourcePath = os.pwd
override def sources = T.sources {millSourcePath / "src" / "main"}
override def resources = T.sources()
override def javacOptions = Seq("-source", "1.8", "-target", "1.8")
override def scalacOptions = Seq("-target:jvm-1.8", "-deprecation", "-unchecked", "-feature")
override def ivyDeps = Agg(
ivy"com.conversantmedia:rtree:1.0.5",
ivy"com.github.scopt::scopt:3.7.1",
ivy"io.spray::spray-json:1.3.5",
ivy"org.jgrapht:jgrapht-core:1.4.0",
ivy"org.scala-lang.modules::scala-parser-combinators:1.1.2"
)
val basicJvmConfiguration = Seq("-Djava.lang.Integer.IntegerCache.high=10000", "-XX:+UseParallelGC")
val jvmHeapSize = Option(System.getenv("YUCK_HEAP_SIZE")).getOrElse("2G")
val jvmConfiguration = Seq("-Xmx%s".format(jvmHeapSize)) ++ basicJvmConfiguration
override def forkArgs = jvmConfiguration
override def mainClass = Some("yuck.flatzinc.runner.FlatZincRunner")
def fullClasspath = T {(localClasspath() ++ upstreamAssemblyClasspath()).map(_.path)}
def fullClasspathAsString = T {fullClasspath().mkString(":")}
object test extends Tests {
override def millSourcePath = os.pwd
override def sources = T.sources {millSourcePath / "src" / "test"}
override def resources = T.sources()
override def testFrameworks = Seq("com.novocode.junit.JUnitFramework")
override def ivyDeps = Agg(
ivy"junit:junit:4.12",
ivy"org.jgrapht:jgrapht-io:1.4.0",
ivy"org.scalamock::scalamock:4.4.0"
)
override def runIvyDeps = Agg(
ivy"com.novocode:junit-interface:0.11"
)
val jvmHeapSize = Option(System.getenv("YUCK_TEST_HEAP_SIZE")).getOrElse("2G")
val jvmConfiguration = Seq("-Xmx%s".format(jvmHeapSize), "-XX:+AggressiveHeap") ++ basicJvmConfiguration
override def forkArgs = jvmConfiguration
override def mainClass = Some("yuck.test.util.YuckTestRunner")
def fullClasspath = T {(localClasspath() ++ upstreamAssemblyClasspath()).map(_.path)}
def fullClasspathAsString = T {fullClasspath().mkString(":")}
}
def documentation = T.source {millSourcePath / "doc"}
def miniZincBindings = T.source {millSourcePath / "resources" / "mzn" / "lib" / "yuck"}
def miniZincSolverConfigurationTemplate = T.source {millSourcePath / "resources" / "mzn" / "yuck.msc.in"}
def unixStartScriptTemplate = T.source {millSourcePath / "resources" / "bin" / "yuck.in"}
def winStartScriptTemplate = T.source {millSourcePath / "resources" / "bin" / "yuck.bat.in"}
def debianControlFileTemplate = T.source {millSourcePath / "resources" / "debian" / "control.in"}
def corePackage = T {
val packageName = "yuck-".concat(version())
val packageDir = T.dest / packageName
for (dependency <- upstreamAssemblyClasspath().map(_.path)) {
os.copy.into(dependency, packageDir / "lib", createFolders = true)
}
val yuckJarName = packageName.concat(".jar")
os.copy(jar().path, packageDir / "lib" / yuckJarName, createFolders = true)
os.copy(documentation().path, packageDir / "doc", createFolders = true)
os.copy(miniZincBindings().path, packageDir / "mzn" / "lib", createFolders = true)
val classPathComponents = (upstreamAssemblyClasspath().iterator.map(_.path.last).toSeq :+ yuckJarName)
val unixClassPath = classPathComponents.map(jarName => "$LIB_DIR/".concat(jarName)).mkString(":")
val basicMapping = Map("#YUCK_JAVA_OPTS#" -> basicJvmConfiguration.mkString(" "), "#MAIN_CLASS#" -> mainClass().get)
fillTemplateIn(
unixStartScriptTemplate().path,
packageDir / "bin" / "yuck",
basicMapping + ("#CLASS_PATH#" -> unixClassPath))
makeExecutable(packageDir / "bin" / "yuck")
val winClassPath = classPathComponents.map(jarName => "%LIB_DIR%\\".concat(jarName)).mkString(";")
fillTemplateIn(
winStartScriptTemplate().path,
packageDir / "bin" / "yuck.bat",
basicMapping + ("#CLASS_PATH#" -> winClassPath))
packageDir
}
def universalPackage = T {
val packageName = "yuck-".concat(version())
val packageDir = T.dest / packageName
os.copy(corePackage(), packageDir, createFolders = true)
fillTemplateIn(
miniZincSolverConfigurationTemplate().path,
packageDir / "mzn" / "yuck.msc",
Map("#VERSION#" -> version(), "#EXE_PATH#" -> "../bin/yuck", "#MZN_LIB_PATH#" -> "lib"))
val archiveName = packageName.concat(".zip")
os.proc("zip", "-r", archiveName, packageName).call(cwd = T.dest)
T.dest / archiveName
}
def debianPackage = T {
val packageName = "yuck-".concat(version())
val packageDir = T.dest / packageName
os.copy(corePackage(), packageDir / "usr" / "share" / "yuck", createFolders = true)
fillTemplateIn(
miniZincSolverConfigurationTemplate().path,
packageDir / "usr" / "share" / "minizinc" / "solvers" / "yuck.msc",
Map("#VERSION#" -> version(), "#EXE_PATH#" -> "/usr/bin/yuck", "#MZN_LIB_PATH#" -> "/usr/share/yuck/mzn/lib"))
fillTemplateIn(
debianControlFileTemplate().path,
packageDir / "DEBIAN" / "control",
Map("#VERSION#" -> version()))
os.makeDir.all(packageDir / "usr" / "bin")
os.symlink(packageDir / "usr" / "bin" / "yuck", os.Path("/usr/share/yuck/bin/yuck"))
os.proc("dpkg-deb", "--build", packageName).call(cwd = T.dest)
T.dest / (packageName.concat(".deb"))
}
private def fillTemplateIn(template: os.Path, target: os.Path, mapping: Map[String, String]): Unit = {
os.write.append(
target,
mapping.foldLeft(os.read(template))({case (s, (k, v)) => s.replace(k, v)}),
createFolders = true)
}
private def makeExecutable(path: os.Path): Unit = {
import java.nio.file.Files
import java.nio.file.attribute.PosixFilePermission
val perms = Files.getPosixFilePermissions(path.toNIO)
perms.add(PosixFilePermission.GROUP_EXECUTE)
perms.add(PosixFilePermission.OWNER_EXECUTE)
perms.add(PosixFilePermission.OTHERS_EXECUTE)
Files.setPosixFilePermissions(path.toNIO, perms)
}
}It fails in the VM when run under the user w/out the home directory.
On a normal system it works:
$ mill --version
Mill Build Tool version 0.9.6
Java version: 1.8.0_282, vendor: OpenJDK BSD Porting Team, runtime: /usr/local/openjdk8/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "FreeBSD", version: 12.2-STABLE, arch: amd64Have you tried to run mill with --no-server? Just in case...
Edit: Because, mill fails already when you run mill --version, so a mill --no-server --version should work. Probably, the mill client-server communication doesn't work as expected.
yurivict
commented
3 years ago --no-server didn't help the build either.
lefou
commented
3 years ago What's the output of mill --no-server --version ?
yurivict
commented
3 years ago $ mill --no-server --version
Mill Build Tool version 0.9.6
Java version: 1.8.0_282, vendor: OpenJDK BSD Porting Team, runtime: /usr/local/openjdk8/jre
Default locale: en_US, platform encoding: US-ASCII
OS name: "FreeBSD", version: 12.2-STABLE, arch: amd64But adding --no-server to the build command line still fails in the same way.
yurivict
commented
3 years ago For a perspective, we build packages in VMs that are run under a very restricted user (nobody) and without network connectivity.
mill fails in this environment, but succeeds in a normal environment, with a regular user.
yurivict
commented
3 years ago @lefou Any chance to resolve this?
lefou
commented
3 years ago Do you have any clue which resource can't be accessed? Because of your restricted environment it's rather hard for anybody else to reproduce your issue. E.g. using mill with --no-server removes one level of error. mill --debug probably helps too. You should try to minimize your setup to isolate the issue and find a smaller repro. I don't think it's mill which causes the issue, but one of it's used dependencies. Probably coursier or ammonite.
Can you run mill with an empty build.sc, e.g. mill --no-server resolve __?
yurivict
commented
3 years ago You should be able to reproduce the restricted user environment by running: sudo -u nobody mill ....
lefou
commented
3 years ago You should be able to reproduce the restricted user environment by running:
sudo -u nobody mill ....
This requires a special sudo configuration.
lefou
commented
3 years ago I assume that ammonite needs to create some (temporary) files/dirs, but can't. I personally find it still to hard to reproduce your environment, so I think you are on your own to find the right settings. You may want to ask in an ammonite channel for help.
lefou
commented
2 years ago As it this now it's a "can't fix". I still need more info on how to reproduce it.
For one project (yuck) it builds in one environment and breaks in another environment with this message:
mill-0.9.6