search

extendr / rextendr

An R package that helps scaffolding extendr-enabled packages or compiling Rust code dynamically
https://extendr.github.io/rextendr/
Other
175 stars 25 forks source link

Tools to build a package with all dependent crates bundled? #300

Closed eitsupi closed 5 months ago

eitsupi commented 1 year ago

From https://github.com/pola-rs/r-polars/pull/308#issuecomment-1632405677 Ref: https://cran.r-project.org/web/packages/using_rust.html, https://github.com/yutannihilation/string2path, https://yutani.rbind.io/post/rust-and-cran-repository-policy/

CRAN announced the following as of a few days ago.

So downloading of Rust ‘crates’ will in future require the explicit permission of the CRAN team.

A better alternative is to bundle all the Rust software in the package. Thanks to Hiroaki Yutani for providing an example of doing that in his string2path package (version 0.1.5 at the time of writing). Even if the bundle is too large to include in the package sources, it could be downloaded from elsewhere (and check-summed).

Currently the extendr template does not include dependent crates in the source package and assumes that the user downloads them from the Internet. It would be great if an easy way could be provided to build a package with all the dependency crates bundled.

string2path commits all dependent crates to Git as a single compressed file, but I think it is better to have the bundled crates gitignored if possible, since this will cause constant diffs of huge files.

Ilia-Kosenkov commented 1 year ago

CRAN never seizes to amaze me. @yutannihilation, I summon you for help & explanation

yutannihilation commented 1 year ago

Sorry, I'm so exhausted. I have no energy to explain from scratch. Please ask CRAN, not me.

My stance stays the same as the post I wrote last year. Before thinking about how to satisfy CRAN, extendr project should think seriously whether it targets CRAN or not.

JosiahParry commented 11 months ago

See somewhat relevant issue in Cargo repo: https://github.com/rust-lang/cargo/issues/10071

It could be interesting if cargo can solve the checksum checking that cran wants to enforce. It might be work giving a different frame of reference in that issue as well.

eitsupi commented 11 months ago

I'm looking at this in prqlr, but the capacity limit seems to be very strict. prqlr absolutely cannot submit the archive to CRAN because it has reached 14MB.

yutannihilation commented 11 months ago

You might be interested:

https://github.com/r-rust/hellorust/pull/16

prqlr absolutely cannot submit the archive to CRAN because it has reached 14MB.

In your case, I think you should ask for the permission to download the vendored archive from somewhere.

eitsupi commented 11 months ago

I have already addressed the problem with the help of string2path and hellorust source code (eitsupi/prqlr#148), thanks.

By raising the compression level to the highest I was able to reduce the size to 12MB, so I will try to post this for now when CRAN is back accepting submissions.

As a side note, for me downloading from somewhere controlled by an individual is less reliable than crates.io.......

eitsupi commented 11 months ago

After all 12MB was not allowed so I resubmitted with the huge Rust dependency file removed, and prqlr 0.5.0 was released just passing the automatic check...

I guess I will have to be afraid for a while that CRAN will suddenly send me an e-mail complaining about it. Edit: prqlr 0.5.0 seems to have been removed from CRAN without prior warning.

CGMossa commented 5 months ago

@JosiahParry Isn't this done in some PR or another? Maybe #313?

JosiahParry commented 5 months ago

Yes @CGMossa

CGMossa commented 5 months ago

Thanks to everyone for participating in fulfilling this issue. I'm closing it and counting it as succesful!