Closed Blaisorblade closed 3 years ago
In both case, it shouldn't error with a simple permission denied
.
For openssl
, opam should fallback to internal sha computing. Could you share the stack trace of opam repo list
?
The code for permission check can help to locate the problem.
Could you share the stack trace of
opam repo list
?
I'm afraid I have no idea how
The code for permission check can help to locate the problem.
Shouldn't that code use https://caml.inria.fr/pub/docs/manual-ocaml/libref/Unix.html#VALaccess instead? In C, access
is implemented by a separate kernel API, which is not guaranteed to have the implementation you use.
Looking at 41d44f9687cffbea9a9762b93c15880a7eb55b02 and https://github.com/ocaml/ocaml/issues/7640, I see why you reimplement PATH
lookup, but that shouldn't prevent you from using access
(2).
You should have the stack trace by setting OCAMLRUNPARAM
environement variable : OCAMLRUNPARAM=b opam repo list
For acls, it is well documented in #4265 /cc @dra27
The ACL stuff shouldn’t be happening unless you’re on Cygwin - although it’s making me wonder if that’s strictly necessary there too, now (although I expect I would still have hit the same bug in Cygwin even using access
!)
For openssl, opam should fallback to internal sha computing.
That part is fixed in #4449.
The permission errors remain — printf debugging suggests that getgroups
throws an Invalid_argument("getgroups not implemented")
— opam list
attached below.
The ACL stuff shouldn’t be happening unless you’re on Cygwin - although it’s making me wonder if that’s strictly necessary there too, now (although I expect I would still have hit the same bug in Cygwin even using
access
!)
AFAIK Darwin has its own ACLs, but I wasn't asking about OpamACL.get_acl_executable_info
(I have no idea about ACLs), just the rest of the access
reimplementation. You've made it much better, but Unix systems have multiple ways to add extra permission checks, including SELinux.
Context:
$ id
uid=501(pgiarrusso) gid=20(staff) groups=20(staff),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),701(com.apple.sharepoint.group.1),33(_appstore),100(_lpoperator),204(_developer),250(_analyticsusers),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh),400(com.apple.access_remote_ae)
$ opam list
# Packages matching: installed
# Name # Installed # Synopsis
base v0.13.2 Full standard library replacement for OCaml
base-bigarray base
base-bytes base Bytes library distributed with the OCaml compiler
base-threads base
base-unix base
biniou 1.2.1 Binary data format designed for speed, safety, ease of use and backward compatibility as protocols evolve
cairo2 0.6.1 Binding to Cairo, a 2D Vector Graphics Library
cmdliner 1.0.4 Declarative definition of command line interfaces for OCaml
conf-adwaita-icon-theme 1 Virtual package relying on adwaita-icon-theme
conf-cairo 1 Virtual package relying on a Cairo system installation
conf-findutils 1 Virtual package relying on findutils
conf-gtk3 18 Virtual package relying on GTK+ 3
conf-gtksourceview3 0+2 Virtual package relying on a GtkSourceView-3 system installation
conf-m4 1 Virtual package relying on m4
conf-perl 1 Virtual package relying on perl
conf-pkg-config 1.3 Virtual package relying on pkg-config installation
coq 8.12.1+flambda-native pinned to version 8.12.1+flambda-native
coq-autosubst dev.coq86 Autosubst is a Coq library for parallel de Bruijn substitutions
coq-ext-lib 0.11.2 A library of Coq definitions, theorems, and tactics
coq-iris 3.3.0 Iris is a Higher-Order Concurrent Separation Logic Framework with support for interactive proofs
coq-iris-string-ident 0.1.0 Add support for Gallina names in intro patterns to the Iris Proof Mode
coq-lens 1.0.1+8.12 Generation of lenses for record datatypes
coq-metacoq-template 1.0~beta1+8.12 A quoting and unquoting library for Coq in Coq
coq-serapi 8.12.0+0.12.0 Serialization library and protocol for machine interaction with the Coq proof assistant
coq-stdpp 1.4.0 std++ is an extended "Standard Library" for Coq
coq-unicoq 1.5+8.12 An enhanced unification algorithm for Coq
coqdep 8.12.0 custom coqdep package for Coq
coqide 8.12.1 IDE of the Coq formal proof management system
cppo 1.6.6 Code preprocessor like cpp for OCaml
csexp 1.3.2 Parsing and printing of S-expressions in Canonical form
cudf 0.9-1 CUDF library (part of the Mancoosi tools)
dose3 5.0.1 Dose library (part of Mancoosi tools)
dune 2.7.1 Fast, portable, and opinionated build system
dune-configurator 2.7.1 Helper library for gathering system configuration
easy-format 1.3.2 High-level and functional interface to the Format module of the OCaml standard library
extlib 1.7.7 A complete yet small extension for OCaml standard library (reduced, recommended)
lablgtk3 3.1.1 OCaml interface to GTK+3
lablgtk3-sourceview3 3.1.1 OCaml interface to GTK+ gtksourceview library
num 1.4 The legacy Num library for arbitrary-precision integer and rational arithmetic
ocaml 4.07.1 The OCaml compiler (virtual package)
ocaml-compiler-libs v0.12.3 OCaml compiler libraries repackaged
ocaml-config 1 OCaml Switch Configuration
ocaml-migrate-parsetree 1.8.0 Convert OCaml parsetrees between different versions
ocaml-secondary-compiler 4.08.1-1 OCaml 4.08.1 Secondary Switch Compiler
ocaml-variants 4.07.1+flambda Official release 4.07.1, with flambda activated
ocamlbuild 0.14.0 OCamlbuild is a build system with builtin rules to easily build most OCaml projects.
ocamlfind 1.8.1 A library manager for OCaml
ocamlfind-secondary 1.8.1 ocamlfind support for ocaml-secondary-compiler
ocamlgraph 1.8.8 A generic graph library for OCaml
opam-depext 1.1.5 install OS distribution packages
opam-file-format 2.1.1 Parser and printer for the opam file syntax
parsexp v0.13.0 S-expression parsing library
ppx_derivers 1.2.1 Shared [@@deriving] plugin registry
ppx_deriving 5.1 Type-driven code generation for OCaml
ppx_deriving_yojson 3.6.1 JSON codec generator for OCaml
ppx_import 1.7.1 A syntax extension for importing declarations from interface files
ppx_sexp_conv v0.13.0 [@@deriving] plugin to generate S-expression conversion functions
ppx_tools_versioned 5.4.0 A variant of ppx_tools based on ocaml-migrate-parsetree
ppxlib 0.15.0 Standard library for ppx rewriters
re 1.9.0 RE is a regular expression library for OCaml
result 1.5 Compatibility Result module
seq base Compatibility package for OCaml's standard iterator type starting from 4.07.
sexplib v0.13.0 Library for serializing OCaml values to and from S-expressions
sexplib0 v0.13.0 Library containing the definition of S-expressions and some base converters
stdlib-shims 0.1.0 Backport some of the new stdlib features to older compiler
yojson 1.7.0 Yojson is an optimized parsing and printing library for the JSON format
Okay, ocaml 4.07.1's configure gives (after instrumenting) getgroups() not found.
The problem seems to boil down to 2 issues:
First, the configure test for getgroups
is buggy (it should include unistd.h
), and gcc got stricter about it:
$ cc -o tst getgroups.c
getgroups.c:24:7: error: implicit declaration of function 'getgroups' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
if (getgroups(NGROUPS_MAX, gidset) == -1) return 1;
^
1 error generated.
The opam package uses -Wno-implicit-function-declaration
, but it only patches "config/auto-aux/hasgot"
, while the offending test also uses ./runtest
:
if sh ./hasgot -i limits.h && sh ./runtest getgroups.c; then
The new title is tentative and is based on extrapolation — I've estimated the impact based on https://github.com/ocaml/opam-repository/commit/3d6ee077dffbb09a2889ee2277e1a0357bf30fd6.
I can also confirm that adding -Wno-implicit-function-declaration
to runtest
restores a few missing functions — not just getgroups
, but also setgroups
, initgroups
and "Asynchronous I/O".
Source patch (EDIT: now also https://github.com/Blaisorblade/ocaml/commit/c4d1ba56515afdd6210594aca145c4644a8de467).
diff --git a/config/auto-aux/runtest b/config/auto-aux/runtest
index c889a0dbd..861598e65 100755
--- a/config/auto-aux/runtest
+++ b/config/auto-aux/runtest
@@ -15,7 +15,7 @@
#* *
#**************************************************************************
-cmd="$cc $cflags -o tst $* $ldflags $cclibs"
+cmd="$cc $cflags -Wno-implicit-function-declaration -o tst $* $ldflags $cclibs"
if $verbose; then
echo "runtest: $cmd" >&2
Effect on configure logs:
--- configure-before.log 2020-11-26 00:50:53.801818505 +0100
+++ configure-after.log 2020-11-26 00:51:14.434824601 +0100
@@ -41,7 +41,11 @@
symlink() found.
waitpid() found.
wait4() found.
+getgroups() found.
+setgroups() found.
+initgroups() found.
POSIX termios found.
+Asynchronous I/O are supported.
setitimer() found.
gethostname() found.
uname() found.
FWIW: while the main issue is due to old OCaml versions, and the use of access
is not the main question, I've checked two implementations of which
, and they both seem based on access
, but not as clearly as I expected; most likely any change would need additional testing. Posting the links I found for reference:
https://github.com/freebsd/freebsd/blob/c5a1b0313c98d82c7d73ace37d69cee697f25cbf/usr.bin/which/which.c#L105-L121 (there's a workaround for root
, maybe because access
pretends root has all permissions — but that seems more limited, and not clearly relevant for ocaml).
https://salsa.debian.org/debian/debianutils/-/blob/3e1637e1f54052d633cf99511f4b7049d864a259/which#L39 (uses test -x
, which FreeBSD maps to (e)access
: https://github.com/freebsd/freebsd/blob/c5a1b0313c98d82c7d73ace37d69cee697f25cbf/bin/test/test.c#L377-L389).
dev meeting 26/2/21:
@Blaisorblade how important is support for OCaml <4.08 to you for compiling the opam binary? We are going to make 4.08 the minimum supported version for compiling opam 2.1.0 onwards.
Note that there is a make cold
target that uses a bundled OCaml compiler if your system OCaml is too old, and that this does not affect older compilers for the purposes of opam switch
(woops, that was @avsm above in the wrong Firefox container ;-)
@bactrian OCaml 4.07.1+flambda is still one of the best compilers for Coq, but I need that in-switch. If I create another Opam patch, I’ll create a new switch.
And I suppose could also use ocaml-secondary-compiler
like dune, tho I’d probably have to contribute that myself if I wanted.
oh I get it, meant to tag @avsm
@Blaisorblade - I'm not sure you do need a secondary compiler here. The issue is that the opam binary itself was built with OCaml 4.07, which you don't need to do, presumably?
Once you've built opam with OCaml 4.08+ you can use it to set-up whichever version of OCaml you like. It looks to me like this issue is resolved, therefore?
No, but nevermind.
Just for the record: ocaml-secondary-compiler would avoid the need to install another compiler for the only purpose of building opam development versions. That's the context for
If I create another Opam patch, I’ll create a new switch.
And I suppose could also use
ocaml-secondary-compiler
like dune, tho I’d probably have to contribute that myself if I wanted.
On mac os x 10.15, opam master fails to run openssl, or sw_vers, while opam 2.0.7 works fine, and openssl and sw_vers are both available:
The error persists after make install.
The programs work.
opam config report output, as requested: