search

unipept / unipept-database

Makes database tables and indices for Unipept
MIT License
0 stars 2 forks source link

Remove all Java and JS tools, use Lz4 #40

Closed stijndcl closed 9 months ago

stijndcl commented 10 months ago

This PR adds most of my changes so far to a new branch to play around with. This is also published as a Docker image:

docker pull ghcr.io/unipept/stijn-thesis:rust-complete
pverscha commented 10 months ago

After trying to start the pipeline, I already have one comment :P It would be very nice if you could add write an extra script that automatically builds all required Rust-utilities for the build_database.sh script and that automatically moves the compiled binaries to the right directory. This can be created in the scripts directory with the name compile_binaries.sh. We'll discuss during the next meeting if we should integrate this compilation step in the build_database.sh script or not.

stijndcl commented 9 months ago

I wanted to do this with a Makefile but then I heard about your history with those :p Definitely open to quickly making a bash script, just keep in mind that compiling binaries requires the Rust toolchain to be installed as well.

pverscha commented 9 months ago

@stijndcl No worries about the Rust toolchain. For this reason, I would (at least temporarily) make a separate script for compiling the binaries.

pverscha commented 9 months ago

Ok, so I decided to already merge this code (since we are currently going to use it for a sideproject) and then update the branch feature/stijn-changes later with the two things that I left as a remark :)

pverscha commented 9 months ago

Ok, one more remark. When the pathname of one of the arguments contains spaces, I get the following error: 

Compressing 1 of 44 for swissprot
./build_database.sh: line 451: /Volumes/T7.lz4: Operation not permitted

(The full pathname was /Volumes/T7 (II)/stijn/index for reference)