Closed kcpevey closed 2 months ago
conda-lock
invokes conda
with --json
, which hides all normal output.
Unfortunately I don't think conda
knows how to handle both -vvv
(or the equivalent env var CONDA_VERBOSITY=3
) and --json
at the same time; -vvv
overrides --json
, which would break the JSON expectation in conda-lock
. But verbose logging should go to stderr
, so this sounds like a bug in conda
I think the only way would be to contribute something in conda
and then ask for a revendor in conda-lock
.
I think this should be moved to the conda-store
issues repo because all of these things are about the server. The UI only links to logs generated by the server ATM.
Action items:
action
and where it's used to write to self.log
. cc @peytondmurray Long reply to @kcpevey's questions (feel free to skip if you don't need context):
I'm in agreement with @nkaretnikov that at the very least there are issues here which need to first be addressed on the conda-store-server side of things. IMO improving the logging situation there is the first step, then we can think about how to implement user-facing feedback in the UI next. I think for this reason it makes sense to first move this to https://github.com/conda-incubator/conda-store. With respect to the individual issues:
The conda config is confusing. The output from the conda-config command shows channel_priority: flexible which shouldn't be true for conda-store (we have checked this many times).
conda-store uses many tools internally and we print logs from them without any processing. The actual channel priority is set via --strict-channel-priority and is printed later. See the previous discussion here https://github.com/conda-incubator/conda-store/issues/737#issuecomment-1913780222 and the PR that implements it here: https://github.com/conda-incubator/conda-store/pull/693
@kcpevey Looks like conda config
is not the right source of information about your config when you use conda-store. I can see a few of ways of addressing this, but do you have any ideas about your preferred solution here? I'd be interested to hear them if so. @jaimergp @nkaretnikov The fact that this has come up before means we need to make it clearer to the user.
Can we insert a shim for conda config
and provide the actual configuration used by conda-store? Or maybe just display a message saying "You've got conda-store installed. FYI conda-store has a couple of settings which might differ from your conda config..." or similar.
As an end user, it would be nice to know the build sequence so that I can understand where I am in the process and what part I might be hung on. Currently, its hard to tell what is happening on the backend. From the logs, I have no idea what the process is
It sounds like it might be possible to get more information about what conda-lock
is doing by following @nkaretnikov's suggestion. Let's try that first since we have the most control over that. I could also see adding something like a progress bar or some other visualization (diagram of the build steps and their current status?) but this would require upstream changes in conda
and conda-lock
.
TLDR: there are a few things we can start to take action on.
conda-lock
while actions are running. starting build of conda environment 2024-03-21 14:52:09.332184 UTC
isn't really enough to give the user an idea about the current status of a build. It sounds like conda
output is currently only available on successful builds, but let's make it accessible for failed builds as well; that way users can at least start to track down build issues.conda config
output. Need ideas about how to address this.If we can't make progress with (1) above, we'll probably need to make upstream changes to conda
and conda-lock
. Later on we can follow up about changes we can make to conda-store-ui
.
I'm also wondering why a build which took 3m initially took 47m on a subsequent build. Is this some network issue, or something else? If it's anything other than a network issue I'd be interested in getting that fixed - 47m is way too long to wait for an environment to build. @kcpevey would you be able to share the environment you copied over?
I actually think this one has to do with a bug that had been recently introduced where "queued" builds were showing up as "building". So this can be disregarded.
As for reasonable output given what I know now, I propose the logs simply output 2 additional pieces of information:
conda-lock create ...
)Peyton, Chuck, and I discussed this today. It was agreed that I'll focus on log-flushing since I have the most context.
We'll split this into self-contained issues and close this one to avoid getting sidetracked:
Feature description
As an end user, if I have a build that fails, I expect to use the logs (via the UI, not Loki) to help me debug. I find that the logs available to me are very frequently insufficient to help me debug.
starting build of conda environment 2024-03-21 14:52:09.332184 UTC
. Perhaps they are not flushed frequently enough?channel_priority: flexible
which shouldn't be true for conda-store (we have checked this many times).starting build of conda environment 2024-03-21 14:52:09.332184 UTC
..conda
file. I don't know what conda-store is running so I can't try to reproduce it on my local machine. That said, I'm also not sure if there is any additional logging from conda-lock that would be helpful in debugging?I will also say the conda output that I expect as an end user IS available, but only after a successful build and it looks a little different given that we're building from a conda-lock file. At the very least, I think if an error happens during the build, there should be a stdout/stderr flush.
Value and/or benefit
End users who can debug failed builds and have visibility into the progress of a build.
Anything else?
Example failure: