Closed richb-hanover closed 2 years ago
Thanks for the report. I wonder if I pushed some breaking changes since the last release. I won't be able to look into it until about 12 hours from now. In the meantime two suggestions:
I know more now. I submitted a PR to fix the ghcr.io link to the container. I can now run the pre-built image as described in the README.md file.
The build problem remains, but it's less time critical. Thanks for the speedy response!
NB I did check out the 0.0.11 tag, and ran into the same error. But again, I'm no longer blocked. Thanks again.
It's so odd that even at that tag it doesn't build on Mac because it built fine on Github and builds on my Ubuntu machine as well. I thought the whole point of Docker was that things should be independent of the host OS?
I'm not really sure how to proceed with this because I don't have access to a Mac. If you can tinker with the Docker file and get it to build on your side then I can test on my platforms as well?
It passes for me too, on a Mac, on a clean main
branch.
Can we reproduce the failure? It is surprising that it works on one machine but not the other...
More debugging info:
I have rust 1.63 installed natively on my Mac and cannot build prql-query
with cargo test
.
I suspect there are problems with mixed versions between prql
and prql-query
. I get different results using rust 1.63 & 1.64 as a base for the Dockerfile.
I also looked through the Cargo.toml
files for both projects and found different versions between the packages:
Since I know so little about the rust environment, I'm not sure whether these differences are relevant. Nor do I know how to update to 1.64 - I can't remember whether I used brew, or the native rust installer, or...
Would one of you look through these versions/dependencies and give me advice? Many thanks.
Though all that should be isolated by docker. Can we reproduce docker build .
failing on a fresh checkout of the git repo? My guess is that that will work and so something more specific is causing the failure — but if that fails then we have a different problem :)
@snth FWIW running cargo build
does cause a small diff in Cargo.lock
— generally that shouldn't happen — I doubt that's the cause but does reduce the strength of the "we're all on the same files" point. Do you see the same thing? I'll PR the diff so you can compare
Yes - I would have expected Docker to isolate all this. I am currently trying to build a fresh checkout of prql-query
as a docker container.
NB I also am running cargo test
after using the procedure on the Rust Install page to upgrade to Rust 1.64.0
Film at 11...
It fails with fresh checkout, getting the same error message.
cargo test
and cargo build-all
work as expected after I upgraded to rust 1.64.0.
What other thoughts? Thanks.
Possibly it's this, and rustfmt
isn't in the image?
#12 832.1 running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-std=c++11" "-o" "/app/target/release/build/libduckdb-sys-68a72700699906fe/out/duckdb/duckdb.o" "-c" "duckdb/duckdb.cpp"
#12 832.1 cargo:warning=c++: fatal error: Killed signal terminated program cc1plus
#12 832.1 cargo:warning=compilation terminated.
#12 832.1 exit status: 1
#12 832.1
#12 832.1 --- stderr
#12 832.1 error: 'rustfmt' is not installed for the toolchain '1.64.0-x86_64-unknown-linux-gnu'
#12 832.1 To install, run `rustup component add rustfmt`
#12 832.1 Failed to run rustfmt: Internal rustfmt error (non-fatal, continuing)
#12 832.1
#12 832.1
#12 832.1 error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-std=c++11" "-o" "/app/target/release/build/libduckdb-sys-68a72700699906fe/out/duckdb/duckdb.o" "-c" "duckdb/duckdb.cpp" with args "c++" did not execute successfully (status code exit status: 1).
#12 832.1
But I tried uninstalling rustfmt locally, and it still seems to compile. Possibly it has different ways to compile, and it hits the unlucky one in your image?
We could try adding rustfmt
in the image. And we could raise this upstream — seems like there's an implicit dependency there. This looks to be the line https://github.com/wangfenjin/duckdb-rs/blob/main/libduckdb-sys/build.rs#L242
@max-sixty I wonder if the Cargo.lock differences are because I don't build the package locally anymore? The builds take so long that I've just been relying on building those on GH.
@richb-hanover I wonder if you have Docker BuildKit installed / enabled? The only time I've had inconsistent build experiences with Docker was when I tried BuildKit. This was on a different project and I wanted the parallel builds and better caching but I kept getting errors which I couldn't resolve and I had to stop using it.
I did try installing rustup in the Dockerfile - same error, but no warning about rustup not being installed (see Details below). Asking upstream might be helpful.
Also: I am not using BuildKit - just a default install of Docker Desktop (now 4.13.0)
I'll need to look into Docker Desktop. In the meantime please could you try the following command for me:
DOCKER_BUILDKIT=0 docker build -t pq .
Your sample output just looks very different from mine and looks very reminiscent of BuildKit which has me wondering whether BuildKit is enabled by default in Docker Desktop.
Your sample output just looks very different from mine and looks very reminiscent of BuildKit which has me wondering whether BuildKit is enabled by default in Docker Desktop.
I believe it is — but mine completes successfully, also using BuildKit...
FYI it also works with DOCKER_BUILDKIT=0
(though tbc, it probably is worth trying with that on @richb-hanover 's system)
Thanks @max-sixty . I tried on my end where BuildKit is usually disabled but I was also able to build successfully with DOCKER_BUILDKIT=1
so that doesn't seem to be the issue.
(Using the checkout from this morning) I disabled BuildKit by setting the Docker Engine pref's to:
{
"builder": {
"gc": {
"defaultKeepStorage": "20GB",
"enabled": true
}
},
"debug": true,
"experimental": true,
"features": {
"buildkit": false
}
}
I get substantially the same error message (see details below)
I will try resetting BuildKit on (by default - see this informative article https://pythonspeed.com/articles/docker-buildkit/) and report back.
Thanks for all your testing with this. It's hard to know what to say since I can't reproduce the error.
One other thing I noticed in your Docker Engine prefs is the "experimental": true,
. Do you maybe want to try setting that to false? I'm really just grasping at straws here. Not sure what else to try. 🤷♂️
No (interesting) difference, even after adding RUN rustup component add rustfmt
and DOCKER_BUILDKIT=0 to the command line.
I appreciate that we're "grasping at straws"... What if we created a small Cargo.toml file that simply installs the libduckdb-sys and whatever else it might need? (It would shorten the development cycle.) Thanks
Another idea, @richb-hanover do you have an M1 Mac by any chance? Just wondering if it's an arm64 vs amd64 issue perhaps?
@max-sixty What kind of Mac do you have?
Just wondering if it's an arm64 vs amd64 issue perhaps?
@max-sixty What kind of Mac do you have?
It could be — I'm on an M1, I think @richb-hanover is on an Intel, looking at the traceback.
One approach would be to run these as GHAs and try and reproduce. @richb-hanover has provided the best tracebacks anyone could hope for, but we're still unable to repro it unfortunately. In any other circumstance, I would suggest we try a tool called docker, oh the irony!
What if we created a small Cargo.toml file that simply installs the libduckdb-sys and whatever else it might need? (It would shorten the development cycle.) Thanks
Yeah, I think this would be a good step, to the extent folks want to invest more here. To the extent it runs without docker, up to others on whether we push more on this break.
I have a 2019 Intel MacBook Pro (i9 - 8 cores). I have a friend with a M1 mac - I'll see if I can persuade them to let me install Docker...
Thanks for the feedback folks. I think that blows that theory out the water. My machines are Intel/amd64 and it compiles on those and since it also compiles on max-sixty's M1 that covers the arm64 case (unless there is some MacOS <-> amd64) interaction effect. I know with Rust you can cross-compile for other platforms but I don't know about Docker.
Folks, I'm out of my depth and out of ideas here. Any other ideas?
Any other ideas?
Nothing magical from my end — try and repro on GHA, or suggest that people who have issues with docker don't use docker...
OK. I'm going to close this (after all, I can run the native pq
app). If I work up the energy, perhaps I'll post this to some Docker community...
@max-sixty @snth - Oh wow! I got a terrific response from the Docker Community. See https://forums.docker.com/t/baffled-dockerfile-builds-on-all-but-one-of-our-teams-computers/130642/2
rimelek asked that we collect the environments where we had success/failure so he could do further testing.
Would you send your system info to me and I'll forward? (OS version, underlying hardware, Docker version, etc). My report would be:
Building the
prql-query
Dockerfile FAILS on macOS 10.15.7 on Intel MacBook Pro, running Docker Desktop 4.13.0 and Docker version 20.10.20, build 9fdeb9c. It has trouble with libduckdb-sys as described in this issue.
Many thanks.
Great — it succeeds for me on macOS 12.6, ARM M1 Max, Docker version 20.10.17, build 100c701.
@max-sixty @snth - I sent over your reports, and received updated interest from rimelek on this... Please feel free to let me know of any additional info that comes to mind. Thanks.
https://github.com/mlpack/mlpack/issues/2775 https://github.com/soedinglab/hh-suite/issues/280
seems you need to increase the VM of docker @richb-hanover
@wangfenjin
seems you need to increase the VM of docker
BINGO! That did the trick. I used Docker Desktop on my Mac to increase the Resources -> Memory from 8GBytes to 10 GBytes, and the docker container built without error. I ran docker run --rm -it -v $(pwd):/data -e HOME=/tmp pq --help
and got the expected help. Thanks!
Thank you @wangfenjin and @richb-hanover . I've added a note to the Docker section in the README about the memory requirement for building the image.
Thanks! Let's leave this closed, because we have a good explanation.
I cloned the
prql-query
repo, then tried to build the Docker container locally. I'm using macOS 10.15.7, Docker 4.12.0. Here's the error message (Update - used<details> ... </details>
to hide the trace info)