aai-institute / lakefs-spec

An fsspec implementation for the lakeFS project.
http://lakefs-spec.org/
Apache License 2.0
37 stars 4 forks source link

Split lakefs-spec test job into multiple jobs #261

Closed nicholasjng closed 5 months ago

nicholasjng commented 5 months ago

This way, it is easier to understand whether a CI failure is a lint, test, or docs build failure. Previously, this was not possible as the pipeline did not tolerate failure.


Bump developer dependencies using pip compile --upgrade

This addresses a number of security advisories related to Jupyter.

Since we do not distribute Jupyter code or expose any instances, these are not as important, but we can still bump the dependencies to silence the dependabot warnings.

nicholasjng commented 5 months ago

Unfortunately, service containers do not work on non-Ubuntu Github runners, so we cannot currently test lakefs-spec on Windows and MacOS in CI.

All the more reason to get a deployment recipe going to test against a cloud instance!

codecov[bot] commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (8a53184) 94.08% compared to head (eda647d) 94.08%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #261 +/- ## ======================================= Coverage 94.08% 94.08% ======================================= Files 5 5 Lines 406 406 Branches 72 72 ======================================= Hits 382 382 Misses 15 15 Partials 9 9 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

nicholasjng commented 5 months ago

This cuts down CI runtime from about 2m30s minutes to around 1m30s. The CodeQL job with above two minutes average runtime is the bottleneck now, but this is still a nice improvement in my opinion.

cc @janwillemkl

nicholasjng commented 5 months ago

I'm going to check this in unsquashed to a) modularize the CI pipeline asap and b) shorten the diff in the transaction redesign PR.