search

PIP-Technical-Team / pip_ingestion_pipeline

0 stars 0 forks source link

Use RENV #81

Open tonyfujs opened 2 years ago

tonyfujs commented 2 years ago

Hey guys,

I'm trying out the new pipeline, but running into some frictions getting things setup. Can we agree to keep the renv.lock file up to date. It currently references old versions of pipaux and pipload. I know renv has its quircks, but it still is the easiest way to ensure we have the same developement environment.

Thanks!

cc @randrescastaneda @Aeilert

randrescastaneda commented 2 years ago

Yes. That is something I don't understand. Before running the pipeline, I always restart R and renv prompts me to snapshot() or restonre() whatever is the case. So I always keep the renv up to date. I don't know why it is referencing old versions of the packages. Something that is ever weirder is that we don't even use pipaux anymore in the pipeline, but it seems to be referenced still, even though I made sure it was removed... I honestly, don't know why it works that way.

Let me give another example. For some reason that I don't understand, when the repo is modified by Aleksander and then I am going to run it again, most of the times it requires me to install a bunch of packages that I don't use. I guess that happens because when Aleksander runs the pipeline or works on it he updates the renv file with the packages that he is using at the moment, even if those packages are not used by the pipeline. Sometimes, I am even requested to downgrade the packages that I already upgraded in the previous round of renv... anyhow, this is confusing and just to let you know that what is happening to you also happens to me most of the times I start working from there Aleksander left off... I guess the same happens to him when he works from the versions, right Aleksander?

cc. @Aeilert @tonyfujs

Aeilert commented 2 years ago

Hey Tony, Andres,

I haven't updated or changed the renv.lock file in this repo for a very long time, so no idea why it is incorrect or causing issues.

Regarding general usage however I agree with Andres. I have had a lot of issues with both out-of-sync packages and corrupt package repositories when using renv in pipdm and previous versions of the ingestion pipeline. This is such as an impediment to constructive development that for pipdm I just disabled renv locally. Why these issues occur I'm not sure. It could be, like Andres mentions, caused by inconsistencies when the file has been updated by different users, but I have also seen issues when I was the only one working on a specific repo.