search

OBOFoundry / OBOFoundry.github.io

Metadata and website for the Open Bio Ontologies Foundry Ontology Registry
http://obofoundry.org
Other
163 stars 201 forks source link

Migrating the testing system to tox #1684

Open matentzn opened 2 years ago

matentzn commented 2 years ago

@cthoyt has converted me in the last year to using tox (and many other associated tools) to improve my python code 100 fold. He is proposing to migrate our OBO Foundry testing framework to tox, and I am happy to starting thinking along these lines. However, there are some concerns about maintainability (we have very few resources to fix stuff if something breaks). The current system is based on a mix of json schema checking and ad-hoc python code with lots of custom logic - which all by itself is not super easy to extend - but doable.

I think we should seriously consider updating our pipelines, but before we do so, we need to understand what is needed to migrate everything, who will support it, who will pay for the migration, and what the risks are.

@cthoyt provided a summary of his additions here: https://github.com/OBOFoundry/OBOFoundry.github.io/pull/1661

Anyone who feels like they have experiences for and against employing tox for testing please feel free to contribute here. What we do not want is to start adding tox on top of what is already there - if we go for it, then we will want to migrate what is currently there over as well, and have one single pipeline to handle everything.

matentzn commented 2 years ago

(one tiny issues is that I dont know how many hours a good programmer would need to toxify the hole pipeline - that's a bit of an issue).

jamesaoverton commented 2 years ago

I don't want to sound like an old grump, but I've learned some painful lessons that I don't care to repeat. I've been contributing to OBO for more than a decade now, and somewhere along the line my role changed from only "pushing for improvements" to largely "protecting what we've built". So I end up saying "no" a lot. I really, really hate that. I remember how much it stung (and still does) to hear people push back against my great ideas for improving OBO. But I feel personally responsible for OBO infrastructure, so I want to share my personal opinions.

Chris Mungall wrote the original code for this repo, and many people have helped out, but my team has been responsible for a lot of the work and maintenance. We do not have direct funding for OBO infrastructure. I'm not happy with the current code, but at least it's a known quantity. I would prefer to improve the current code and tooling incrementally.

My team works with Python but also several other languages every day. One reason we use GNU Make is that it's not tied to any one language. We use linters and formatters and testing tools, such as 'flake8' and 'black' and 'pytest' For Python, but we run them from Make, like the rest of our automation tasks. Another reason is that Make is really mature -- it's been around longer than I have, it's installed everywhere, and it's not going anywhere.

I'm increasingly unhappy with the churn and bitrot with packaging and tooling in the Python ecosystem. I've been bitten by the same things in the JavaScript ecosystem years ago and now I'm very wary.

Bluntly, my general concern is that we "modernize" our Python code to use the latest shiny tools, then my team gets left holding the bag when those tools are abandoned in favour of the next shiny thing. It's painful enough when paying for unexpected fixes comes out of my budget, but the worst thing is that fixing something right now means that I break a promise and miss a deadline for whatever work had to be rescheduled.

So those are my strongest reasons to be very conservative.

If some volunteers want to step up and commit to maintaining the code in this repo for the next five years or something, that would address most of my concerns. It seems important to me that these people be members of the OBO Operations Committee Technical Working Group -- we are happy to invite new members.

At the very least, I would like to see a plurality of current TWG members supporting the proposed changes before they are implemented. So far, I haven't heard from any TWG members other than @matentzn.