search

HPInc / HP-Digital-Microfluidics

HP Digital Microfluidics Software Platform and Libraries
MIT License
2 stars 0 forks source link

Refactor repo to prepare for open source release #305

Closed EvanKirshenbaum closed 5 months ago

EvanKirshenbaum commented 5 months ago

Now that I've gotten word that the decision has been made to open-source the codebase, it seems as though it would be a good time to do a bit of refactoring in the code base, with the goals of

  1. Allowing the erk and quantities packages to be forked off to a different repo and made available through pip and
  2. Simplifying the maintenance of everything else by
    1. Getting rid of the layer above mpam (i.e., not pretending that anything else is part of thylacine)
    2. Possibly making a better split between the interpreter, UI, DML language, and devices.
      • At some point, we will likely want to split those up into separately installable libraries.

For PyPI reasons, we will probably want to rename erk to erktools and quantities to something like dquant, since we can't reuse a name that somebody else already has in pip.

As part of the splitting, I want to go through the code, especially through types.py and type_supp.py and move pretty much everything that isn't DMF-specific to either erk or quantities. This will include things like Delayed and Operations (probably to a sched subpackage) as well as Missing. The goal is to have as much as possible outside of the DMF repo.

I also want to make it so that most of the commonly used things can be imported from the packages directly (via their __init__.py files) and discourage especially going through erk.basic and quantities.core or .dimensions. Since the latter is automatically generated using gen_dims.py, it would be nice to be able to simply say from .dimensions import *, but if we do that, we'll probably want to add an __all__ at the beginning to make it so that only pulls in the dimension classes themselves.

Also, to avoid polluting namespaces on imports, I'll want to do a pass through and make sure that anything that isn't something we want people to import (especially TypeVars) start with single underscores.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 29, 2023 at 3:01 PM PDT. Closed on Jan 29, 2024 at 9:29 AM PST.
EvanKirshenbaum commented 5 months ago

This issue was referenced by the following commits before migration:

EvanKirshenbaum commented 5 months ago

One thing I'm noticing is that in doing the refactoring, I'm rendering a bunch of the doc comments wrong. That should be fixed.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Oct 05, 2023 at 10:39 AM PDT.
EvanKirshenbaum commented 5 months ago

Pretty much everything that isn't strictly DMF-related is now either in erk or quantities. There are new modules: quantities.ticks and erk. sched, grid, monitored, color, sample, loglevel, and afs. Things like Missing are now in erk.basic. cmd_line is now in erk. A bunch of unused classes are gone.

Still to do:

Migrated from internal repository. Originally created by @EvanKirshenbaum on Oct 19, 2023 at 3:10 PM PDT.
EvanKirshenbaum commented 5 months ago

The directory structure now seems right for publication:

Still to be done:

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jan 22, 2024 at 12:40 PM PST.
EvanKirshenbaum commented 5 months ago

I think we're probably as close to ready as we'll be. The LICENSE.md file is now there, and there's a README.md file that both explains the repo and also gives basic instructions for installing and running, as well as a bit on a few development tasks and an explanation about why the issues list will look so weird.[^issues]

[^issues]: GitHub really doesn't make it easy to migrate issues from one repo to another. I knew I was going to have to write programs to fetch them from the internal repo and push them to the public one, but I didn't realize that it was going to assert that all of the issues and comments were made by me and not allow me to back-date them. So I had to add text identifying when they actually happened. Then the commits happened earlier, so they don't show up on the pages for the issues they refer to, so I had to add extra comments for those. Then I had to put any images in the repo so they could be referred to. (They were added in commit cc0e7b9448ff2c4272e28dd5b7aba4a45679aeeb in folder migrated-issue-images and then taken out the next commit. The comments refer directly to that commit.) Then I found out that the public GitHub's secondary rate limit means that I can't do the upload at once and have to wait about an hour between batches. Sigh.

I got rid of everything from the doc directory except for the DML manual (which got a couple of problems fixed and changed the copyright notice), and I pruned the opentrons dir of everything other than what's needed to run the protocol. For the inputs folder, I got rid of the stuff that I knew was mine, but I left in all of the other stuff, because I don't know who relies on it. That should be cleaned up, too (#310), and I left a README.md file there to that effect.

I have not gone through any of the actual code to prune commented-out blocks or get rid of printed or logged messages. That will have to wait.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jan 29, 2024 at 9:28 AM PST.