search

econ-ark / REMARK

Replications and Explorations Made using the ARK
Apache License 2.0
19 stars 56 forks source link

Release cycle and dependency management: design and documentation #56

Closed sbenthall closed 3 years ago

sbenthall commented 4 years ago

Some questions have been raise about REMARK release cycle and dependency management. This is a ticket to get clarity on those issues and document them.

This is my proposal:

My understanding is that there is an obstacle to this proposal:

As a consequence of the latter issue, the REMARK repository is currently being developed in an ad hoc way, with no releases or dependency locking at all.

llorracc commented 4 years ago

That sounds about right to me.

sbenthall commented 4 years ago

It looks like REMARKs are developed, controlled, and maintained by their team of authors. That squares with the fact that they can be submodules.

Is the use of submodules in REMARKs here to stay?

The submodule link can be pointed to a specific branch. This seems to be the only supported way to link to a "stable" version of a submodule repository: https://medium.com/faun/git-submodule-cheatsheet-29a3bfe443c3 https://github.com/econ-ark/REMARK/blob/master/.gitmodules#L12

sbenthall commented 4 years ago

The individual binder/ directories seem to cover the issue of locking dependencies. It's just a matter of putting the dependency pegs in.

Having REMARKs have individualized release numbers is more difficult because they are all in one repository. Normally, everything in a repository is released together.

A different way to do this would be if REMARKs were all in their own repositories, and then rather than be submodules, this REMARK repository could have Docker container configurations that each knew the location and latest release of the REMARKs.

MridulS commented 4 years ago

we should definitely make the binder/ subdirectory a requirement for all REMARKs, which has a versioned requirements.txt file, not just for HARK but numpy/scipy/ all the other dependencies required to run that REMARK. For example for the TFI blog post REMARK, I have pinned it to 0.10.5 using the configurator.py. I will also add a binder/requirements.txt file.

Is the use of submodules in REMARKs here to stay?

We have had multiple conversations regarding this and I think we should take a clear stand regarding this, there are some REMARKs which are like a folder and some are submodules.

There is another "naive" way of controlling this, we can just a link in a markdown file to the actual repository in the REMARKs if it's a different repository. For example BufferStockTheory is a submodule which is hosted at llorracc/BufferStockTheory, we can update the markdown file BufferStockTheory.md and move it to REMARKs/BufferStockTheory/BufferStockTheory.md with a normal link to https://github.com/llorracc/BufferStockTheory/ , this effectively decouples econ-ark/REMARK with all the other REMARKs with respect to submodules while still maintaining an index of all REMARKs. This will be much more new user friendly, and people without much experience with git will be able to keep contributing to this REMARK index without worrying about submodules.

If there are some REMARKs where it doesn't make sense to keep them in an individual repository we can follow the current way of doing it, where the REMARK lives inside the REMARKs folder.

[This will also raise the question of effectively maintaining the binder images, submodules allows us to have a meta-binder image and we can run all the remarks through econ-ark/remark]

llorracc commented 3 years ago

@MridulS make sure we have a checklist for REMARK releases, and then either close this or mark the REMARK parts as done.

MridulS commented 3 years ago

This has been largely streamlined with a starter template available at https://github.com/econ-ark/REMARK-starter-example and https://github.com/econ-ark/REMARK as the index. Documentation part needs to be added to the REAMDE (https://github.com/econ-ark/REMARK/issues/86)