We'd love your feedback!

kylemann commented 3 years ago

Thanks for dropping by! 👋

We've been iterating and updating the web app experience and would love your feedback about:

What do you think about the new UI changes?
What would you'd like to see improved?
What are challenges you've experienced?
How do you use the tool today? What parts do you use?
If you don't use the web tool: why not?
How could we better help your workflow?
Any general thoughts you'd like to share!

We greatly appreciate your time and thoughts - looking forward to hearing from you ❤

Codecov team

This issue is intended to share and collect feedback about the tool. If you have support needs or questions, please see our support page.

friedrichatgc commented 3 years ago

I miss the per commit tree view: e.g. https://codecov.io/gh/mbsim-env/openmbv/tree/1f7675e59685fa23c9149a7b308e300e30189054 seems not to exist in the new interface (the link "switch to the new user interface" does not work and I can't find it in the new interface)

drazisil commented 3 years ago

I miss the per commit tree view: e.g. https://codecov.io/gh/mbsim-env/openmbv/tree/1f7675e59685fa23c9149a7b308e300e30189054 seems not to exist in the new interface (the link "switch to the new user interface" does not work and I can't find it in the new interface)

Are you referring to https://app.codecov.io/gh/mbsim-env/openmbv/commit/1f7675e59685fa23c9149a7b308e300e30189054 ?

vinniefalco commented 3 years ago

Wasted-space elephant says hi :)

vinniefalco commented 3 years ago

If I'm being completely honest, switching to a proportional-width sans-serif font for source code is not ideal:

codecovdesign commented 3 years ago

If I'm being completely honest, switching to a proportional-width sans-serif font for source code is not ideal:

@vinniefalco indeed. this is a known bug, that will be updated soon.

vinniefalco commented 3 years ago

So... I browse to my recent commit:

I click on it:

Now I press "Files" so I can get an overview of all my files and see which one has the most uncovered lines, to target that area of my library next:

I apply a sort and you can see that url.ipp has 37 uncovered lines:

Okay so what is this about? Well that workflow is with the OLD UI (which still has some new "improvements" that I don't like).

If I switch to the new UI, then I get this interface instead:

I cannot for the life of me figure out which button or link I press here to get the "Files" view. In fact I am starting to think that it is not present. Why not?

codecovdesign commented 3 years ago

@vinniefalco thanks for sharing and great points. Here are a few follow ups:

which button or link I press here to get the "Files" view

The new commit page only shows the files impacted by the commit changes. Regarding viewing all files:

Now I press "Files" so I can get an overview of all my files and see which one has the most uncovered lines

If you'd like to see general file coverage, per the latest commit, you can view that on the repo overview page and select the file (without needing to go to the commits page). Currently, the repo page is the old UI and the selected file will be a temporary UI. The page is in-progress, but once updated you'll be able to view the files tree and individual files without navigating to a new page.

codecovdesign commented 3 years ago

I miss the per commit tree view: e.g. https://codecov.io/gh/mbsim-env/openmbv/tree/1f7675e59685fa23c9149a7b308e300e30189054 seems not to exist in the new interface (the link "switch to the new user interface" does not work and I can't find it in the new interface)

@friedrichatgc about the the file tree; it's not currently in the new commit page as it's only showing files that were impacted by the changes in the commit. You can view the full and latest file tree in the repo overview page, which shows files per the latest commit in the designated branch. Could you share more about why you miss it in the individual commit page? Is the file tree in the repo overview helpful? (atm it's still a mix of the old UI and file viewer)

@friedrichatgc @vinniefalco @drazisil here is 📽️ quick walkthrough video on the topic, with some more context. Let me know if something stands out to you and/or if you have any other thoughts or feedback about the direction. Thank you 🙏

codecovdesign commented 3 years ago

@rennokki thank you for the feedback! Just a quick note to mention it's been heard and I really appreciate the suggestion 💯

the main idea here is that the uncovered lines can be a lightish blue of rgb(136 138 201) and covered lines can be a strong rgb(29 0 255). If I have to sum up better accessibility, covered lines can be a strong opaque color while uncovered lines might be really transparent, with an opacity of 0.2-0.3 or something. When facing accessibility issues like these, shades differentiate better than colors

codecovdesign commented 3 years ago

trying to copy-paste the YAML but it's also copying the numbers

@miparnisari thanks for surfacing that! Filing a bug report now. 🙏

codecovdesign commented 3 years ago

The partly covered lines should show a little context and the ability to expand the context. But it just removes the color. Why so I can scroll down through the entire file searching for the partly covered lines??

I can see the code but there is no indication of what lines are changed

commit message is a span and doesn't preserve whitespace

@rouilj thank you for your feedback 🙏 . It makes sense and we're looking to improve these points in future iterations.

codecovdesign commented 3 years ago

The new UI looks good to me. Just a suggestion: when I work with codecov I look at the coverage of a file, for a particular commit. Then I make changes in the code, go to the process of updating code coverage, then look at the file again. Since my view is for an old, specific commit, I need to switch somehow to the lastest commit, then search for the file again. It would be nice, if there is no easy way to do that already, to have a view that shows the latest coverage in a file. I would then just refresh the view (ex: F5 in Google Chrome on Windows) to see what changed.

@dlebansais thank you for your feedback and describing the workflow. Linking this comment; which seems related to your point. This walkthrough show current ability to look at latest file coverage and direction. Would the repo files, that show the latest coverage of last commit, help the workflow your describing?

vinniefalco commented 3 years ago

If you'd like to see general file coverage, per the latest commit, you can view that on the repo overview page and select the file (without needing to go to the commits page).

If you mean this page for example:

https://app.codecov.io/gh/vinniefalco/url

The directory view is missing the button in the top right that lets you view by file instead of by directory:

I usually always view all files, so I can sort by uncovered lines descending. This is the button I'm referring to, which is not available on the repo overview page:

I mentioned this earlier. Speaking of earlier mentions, are we ever going to go back to highlighting the entire line to show coverage or missing coverage? Codecov has been conspicuously silent on this matter.

codecovdesign commented 3 years ago

If you mean this page for example: https://app.codecov.io/gh/vinniefalco/url The directory view is missing the button in the top right that lets you view by file instead of by directory: I usually always view all files, so I can sort by uncovered lines descending. This is the button I'm referring to, which is not available on the repo overview page:

@vinniefalco Yes, I'm referring to the URL you sent. Again, thanks for sharing the workflow, your preference to use the commit files w/ the sort ability makes 💯 sense! The new repo UI will have capability to 1) sort by coverage low to high, 2) view in list style as you reference. In this scenario, do you think you'd use the repo files? Since they are the most up to date and offer sorting/etc.

are we ever going to go back to highlighting the entire line to show coverage or missing coverage? Codecov has been conspicuously silent on this matter.

This is not yet determined and is actively being considered. This is because we are weighting the tradeoffs, such as:

feedback about confusion about git diff view vs coverage on the former diff page. Do you find the git diff view helpful or do you mainly focus on coverage information?
as mentioned earlier, color accessibility issue. So keeping the colors and usage of colors to a minimum; while exploring alternatives like shades, patterns, toggle ability, etc.
showing other data points such as changed coverage and other features that would also need to "highlight" a line in one way or another.

The current UI is still there while we iterate and identify the tradeoffs that work best for all users. The feedback here has been really helpful for the team 🙏

vinniefalco commented 3 years ago

In this scenario, do you think you'd use the repo files?

I'm not sure what's being asked here.

Do you find the git diff view helpful or do you mainly focus on coverage information?

I never use the diff. By the time I look at coverage reports I am already aware of what I've changed, via local gitk or github. There's no need to see it again. What I care about is the coverage. Which files are "done" (no uncovered lines). Which files have uncovered lines, which files should I focus my effort on (largest number of uncovered lines), and where are the uncovered lines.

Primarily my work is in C++, where there is no code emitted for function template definitions unless they are instantiated. This means that I could have a bunch of code that is neither covered nor uncovered. So I rely heavily on visible highlighting to quickly understand what I should do next.

Note that as of now, none of the UI changes have improved any part of my workflow... and if I'm being honest the experience is worse.

vinniefalco commented 3 years ago

What about a per-user stylesheet?

vinniefalco commented 3 years ago

https://www.youtube.com/watch?v=JSUDTAG72WQ&ab_channel=CodecovDesign

I watched and rewatched this video and here are my comments by timestamp:

https://youtu.be/JSUDTAG72WQ?t=57

Why do I want to see all the files on the commit page? When I am in "coverage mode" (i.e. I am at the stage in my project where I am trying to get coverage up as high as possible) I am mainly doing two things:

Observe that my latest commit now covers the previously uncovered lines, as a result of my changes.
Determine which file I should focus on next

I think that for 1. the per-commit Coverage report / Impacted files is great, to see precisely what coverage changes came about from the commit. I have some concerns about the layout and wasted space which I outlined above but that just diminishes ergonomics rather than impeding workflow.

For 2. I use the commit-specific Files panel (not available in new UI), choose "View as list":

then usually sort by Uncovered lines descending. Sometimes I sort by covered lines because, hey - a good flex once in a while is nice as a reward for the effort. That sort looks like this:

As you can see, url.ipp needs some work. Given that the library's focus is on parsing, modifying, and serializing URLs, I better get cracking on those 37 uncovered lines.

https://youtu.be/JSUDTAG72WQ?t=77

It says all the file data is from the latest commit. Which I suppose is okay, although it is strictly a regression in terms of functionality, as we are no longer able to view the file data from previous commits. This doesn't impact my workflow specifically but generally speaking - users don't like it when features are removed especially when they are helpless. By helpless I mean that users of web applications typically have no control over UI changes which are imposed on them. Unlike a desktop application they cannot simply refuse to upgrade.

The bigger problem with the "latest commit file data" panel is that it does not provide "View as list." Specifically the control missing from the top right:

As Boost libraries typically have lots of folders and files, this is a regression in both function and ergonomics.

In the new UI (there is no control I can find that takes you from the commit changes summary to the page with the latest commit file panel:

https://app.codecov.io/gh/vinniefalco/url/commit/f892c14fdca8f0a416fc9890fadd4de3d309d373/include/boost/url/impl/ipv6_address.ipp

If the commit changes summary included such a control, and the file data panel allowed for "View as list", it would restore most of the lost functionality.

https://youtu.be/JSUDTAG72WQ?t=85

Clicking a file from the latest commit file data viewer (assuming View as list is added) and going to the "File viewer" would become my primary and favorite means of viewing coverage data, since the file viewer takes up the full width of the browser (I put reports and browser windows on a portrait screen not landscape, so horizontal space is precious).

However in the video it is stated that this view is temporary?

https://youtu.be/JSUDTAG72WQ?t=151

Regarding: Clicking a file in the new repo overview page inlines the file instead of opening a new window. I can't say that I'm a fan. Opening the file on its own new page is preferred. Even better if I could open in New Tab so that I could view coverage for two different files side by side (currently not possible).

https://youtu.be/JSUDTAG72WQ?t=166

"We wanted to try to have a little bit more focus on the commit coverage changes."

There's nothing wrong with this in principle, and I could see myself even using it as a first step before viewing files. But my spider senses are tingling. There is great danger in changing how everything works, especially if functionality is removed. Some users will be happy with the changes but many others will be quite dissatisfied since they are effectively at the mercy of the developers. If there is a way to add this new functionality in a way that does not disturb existing features, or has minimal impact, it might be worth exploring.

"First, do no harm" may be a good guiding principle here.

drazisil commented 3 years ago

For 2. I use the commit-specific Files panel (not available in new UI), choose "View as list":

Personal observation, I never knew that was there, TIL. That might be why/how it was lost.

(I put reports and browser windows on a portrait screen not landscape, so horizontal space is precious).

I understand your focus on unused white space more.

Note: Comments made under this account are my own and should not be considered to have any connection to my position at Codecov.

dpelle commented 3 years ago

Another issue: when source file has > 1000 lines, the line numbers are badly displayed. They wrap on 2 lines as illustrated on this screenshot:

codecov-line-num-wrap-with-more-1000-lines

This is a screenshot of https://app.codecov.io/gh/vim/vim/blob/adfde122ac9c8c46ffed8e9956bc0d6be9e23c46/src/ex_docmd.c

In other words a line number like 1234 is displayed as

123
  4

instead of:

Not only it makes the line numbers ugly and hard to read, but it also causes annoying extra spacing between lines of source code.

kupka commented 3 years ago

Not sure if this is UI related... But how is this possible? Less Patch Coverage resulting in higher overall coverage?

H0R5E commented 3 years ago

Rendering of large commit messages is funky! Maybe support markdown syntax in commit messages, also?

large_commit_msg

H0R5E commented 3 years ago

The change data differs between the old and new interface for commit comparison (I guess the old interface is wrong):

OLD

__init__old

NEW

__init__new

H0R5E commented 3 years ago

As the commit page is a diff (against the last commit), I'd really like to see the coverage diff as well. So rather than this:

diff_new

I'd prefer something like this:

diff_old

This is a super user feature, that is challenging to reproduce locally.

ErisDS commented 3 years ago

Here to report a couple of small niggles, thanks for making it easy to share these:

I encounter loading issues quite frequently, where a page just shows me the animated codecov logo for a very very long time, if not forever.
The back button doesn't work when navigating around the sunburst, which trips me up every single time 😬 It throws me back to the project list page... and then when clicking into a project, problem 1 happens 😬

victorjkhalaf commented 3 years ago

https://github.com/codecov/Codecov-user-feedback/issues/1#issuecomment-970434846

The back button doesn't work when navigating around the sunburst, which trips me up every single time 😬 It throws me back to the project list page... and then when clicking into a project, problem 1 happens 😬

This happens to me with the files table at the bottom. I have the assumption that when I navigate down the file list, the browser would also track my progress so if I press Back, I would just navigate Up the tree, not go to the previous screen. Like you, it trips me up every single time! :D

ErisDS commented 3 years ago

Yeah happens to me with the code table as well. My expectation is the same, that it will navigate back up the code tree.

I make this mistake 3/4/5 times in a session and then I give up and generate the reports locally on my computer instead.

robeady commented 3 years ago

Looking at commit coverage, the new red and green colours are too difficult to tell apart. See the screenshots a few comments above (https://github.com/codecov/Codecov-user-feedback/issues/1#issuecomment-970130593) - when scrolling through such a page, with the old colours you could easily spot uncovered lines. With the new colours they don't stand out at all.

I imagine for people who are colour-blind this is a huge accessibility issue

I feel like pointing out that the whole purpose of codecov is to identify lines of code that are not covered by tests, so I don't understand why you would change to such a subtle and hard to distinguish colour scheme

vinniefalco commented 3 years ago

Overall, these UI changes are a regression, and I am losing confidence that it will be addressed.

mosra commented 3 years ago

I loved the previous Codecov design, it was clean, clear and easy to navigate. I migrated to Codecov from Coveralls because the UI was so much better and faster, and because of the report merging, Now, however:

Are you serious, a proportional font? Given that GitHub and most editors use a fixed-width font, comments sometimes contain ASCII art etc, I don't see any potential advantage of this. Any recognizable patterns or blocks in the code the eyes would attach to are gone here because it all suddenly looks completely different. (Yes, you said you're working on a fix, but I feel it bears repeating.)
The washed-out low-contrast color scheme is an accessibility nightmare. I don't really consider myself that much blind or colorblind but it requires an extreme effort to read #eaebef code comments on a #ffffff background. Case in point is below -- and I doubt it's immediately recognizable that one of the continues is uncovered:

It's all somehow visibly slower. One thing that sold Codecov for me was the instant navigation, but the new interface doesn't feel that snappy anymore and I spend more time looking at the umbrella animation and the UI shaking itself than at my own code. Not sure what's to blame, but I'm not happy about this trend.
I'm regularly looking at coverage state of the repository at a particular commit (the /tree/ URLs in the previous UI, as mentioned above). This is not in the new UI anymore -- the /commit/ URL isn't enough, as that shows just files that changed, not the whole file tree.
I rely on the coverage reports a lot for my work, so to retain productivity I wanted to stay on the previous UI until this new one is ironed out. However, even though I'm browsing the repository tree on the codecov.io domain, viewing files redirects me to the new washed-out proportional misery on app.codecov.io and I have to manually remove the app. from the URL to see the previous UI again.

But this choice is somehow never remembered and next time I have to do the same. Didn't find any settings page where I could tell it to forcibly stay on one or the other version of UI -- if there is, please tell me.
Already mentioned above, the "switch to the new interface" link on pages such as this points to https://app.codecov.ionone/ (yes, none). I suppose this is somehow related to the /tree/ views not being in the new UI? This was my "first impression" with the new UI, and because it led nowhere, it didn't give me much confidence so I didn't even bother exploring further at that time.

That would be the most glaring issues I encountered in my workflow. Please, you have a great service, don't let it be ruined by unfortunate UI choices :)

vinniefalco commented 3 years ago

I migrated to Codecov from Coveralls because the UI was so much better and faster, and because of the report merging...

Same.

We need to coin a phrase to describe the commonly recurring situation where a company providing a popular free (or paid), essential web application gets the urge to completely revamp the UI for the worse.

joachimmetz commented 3 years ago

What would you'd like to see improved?

"Files" view is missing (or at least not obvious) in the new UI

codecovdesign commented 3 years ago

@mosra thank you for your feedback! It's very helpful and I've shared it with the team. Quick note: the former UI is and will continue to be available for usage. We are rolling out the new UI with minimal iterations to address performance issues and obtain early feedback to build on. Specific follow ups:

a proportional font?

This is a known bug and will be updated soon.

I have to manually remove the app.

In the case you are in the old UI looking at a file, then clicking to view, it should not revert to the new. Thank you for surfacing this, it looks like a bug and will be updated. 🙏

mdabrowski1990 commented 2 years ago

I have just moved from Coveralls to CodeCov. The main reason for the change was the feature that I desired (flags and marking uploaded files e.g. unit-tests, integration-tests).

What I had the most problem with:

CodeCov Uploader configuration in GitHub Actions to properly upload coverage files with flags marked
creating and configuring .codecov.yml file (from what I see it is not discovered on commit - example, but it is discovered for the project - https://app.codecov.io/gh/mdabrowski1990/uds/settings/yaml)

What I enjoy the most:

free for open source projects
features
- flags
- graphs
- badge/shield
- bot and possiblity to configure its verbosity
documentation - I have a few problem with it, nevertheless the most things is described there in rather logical way. Probably, you could have to have direct feedback from user to optimize some parts of it, so it up to date and easier to understand.

What I would enjoy to have:

separate graphs for each flag (total and each flag to be separated)
separate history of coverage for each flag (total and each flag to be separated)

My general feedback is: I like your tools, it is probably the best solution that I have found in the market. It has all the necessary features and decent documentation. You could improve it a bit and add some flashy features, but in general I am more than happy with what I have.

jerrodcodecov commented 2 years ago

Hey @mdabrowski1990 thanks so much for the notes!

Quickly jumping on the documentation point to say, you can suggest edits to our docs where they are not optimal (see below). Anything specific we can make better?

mdabrowski1990 commented 2 years ago

@jerrodcodecov Thank you. Honestly, I have not noticed that before.

calvin-cdev commented 2 years ago

Hey! I'm new to codecov but the first thing I noticed while working in the middle of the night is a lack of dark mode :eye: :hurtrealbad: Otherwise, I'm finding the UI to feel intuitive so far :D

nalandial commented 2 years ago

I'm new to codecov so it's probably just my ignorance of how things are set up, but I found it super difficult to find out how to increase the number of users in my organization.

vinniefalco commented 2 years ago

it isn't though, the green highlight does not extend across the line, only in the gutter.

I guess we can assume, given the lack of response or change, that this is not going to happen and the new style will be forced on us?

Ismoh commented 2 years ago

Infinite loading on: https://app.codecov.io/gh/Ismoh/NoitaMP/compare/develop...unit-tests/tree/noita-mp/files/scripts/util/file_util.lua

Developer console error:

Adblock Plus and Ghostery disabled.

vinniefalco commented 2 years ago

The new code font is worse than the old one. It looks thin and inconsistent. Thumbs down.

ahopkins commented 2 years ago

https://app.codecov.ionone/

reactive-firewall commented 2 years ago

Fist The issue: The Commit time view is different in the new UI from the old (which was working)

#EDIT: Clarification The Timestamp on the same commit shows differently between the old web UI and the new web UI. For example some commits show future dates (as in wrong time zone) for commits on the new UI.

TL;DR - Future dates make NO sense and contribute no value to the reader who still must figure out the actual time themself.

see example of same commit attached: NEW:

OLD:

Second the feedback:

What do you think about the new UI changes?

Definitely the right direction to make the UI more fluid and responsive

What would you'd like to see improved?

The data stability/consistency especially when there are defects in the source repo (where analysis tools are MOST useful) for example the time issue mentioned

What are challenges you've experienced?

no feedback on this sorry

How do you use the tool today? What parts do you use?

mostly to combine numerous builds and to review code coverage goals in pull-requests but also sometimes to review line by line differences between environments from tests

If you don't use the web tool: why not?

the phrase "there's an app for that" comes to mind

How could we better help your workflow?

remain focused on what you do best and don't over bloat the tool with things that are better coupled then integrated.

Any general thoughts you'd like to share!

just an honest thank you for supporting open-source projects with the open-source plan!

phh commented 2 years ago

Uh!

Meh:

qhenkart commented 2 years ago

I'm sure someone else complained about this. I've been switching back and forth from the new user interface to the old one to access the diffs, files, etc. It looked the main features were turned off in the new UI. This experienced matched others I was working with until I realized that this random little hash link is what leads to the actual codecov page. I'm not sure why a random hash link would be assumed to redirect to the main screen that shows the diffs and files etc. Especially next to a Github popout link. From a UI perspective, I would have expected this to link to the circleci build since it is next to CI passed. Or the Github PR since it uses Github's icon

Pencab commented 2 years ago

Thanks for dropping by! 👋

We've been iterating and updating the web app experience and would love your feedback about:

What do you think about the new UI changes?

What would you'd like to see improved?

What are challenges you've experienced?

How do you use the tool today? What parts do you use?

If you don't use the web tool: why not?

How could we better help your workflow?

Any general thoughts you'd like to share!

We greatly appreciate your time and thoughts - looking forward to hearing from you ❤

Codecov team

This issue is intended to share and collect feedback about the tool. If you have support needs or questions, please see our support page.

vinniefalco commented 2 years ago

Please for the love of God stop making codecov worse. What the heck is this? https://app.codecov.io/gh/CPPAlliance/url/compare/118/tree/include/boost/url/impl/const_string.ipp

dpelle commented 2 years ago

@vinniefalco wrote:

Please for the love of God stop making codecov worse. What the heck is this? https://app.codecov.io/gh/CPPAlliance/url/compare/118/tree/include/boost/url/impl/const_string.ipp

Consider attaching a screenshot, because I see an error message when I click on your URL:

Unauthorized Github API rate limit error: Forbidden Check on Codecov's status or see our docs for common support. Error 403

vinniefalco commented 2 years ago

Consider attaching a screenshot, because I see an error message when I click on your URL:

Yeah. So, lines that were previously covered are not covered even though the code hasn't changed:

New, uncovered:

Previous, full coverage (100%)

And now what is this fresh hell? two columns, some green some red, some lines are highlighted with color all the way through some are not and there's a weird lowercase i in a circle

And I guess this line is both covered and not covered

This is so much harder to read than before, that it has made codecov less useful for me.

dpelle commented 2 years ago

@vinniefalco wrote:

And I guess this line is both covered and not covered

I'm not affiliated to codecov, but you seem to be looking a a diff between two versions. The 3rd screenshot shows a line that was covered in a previous git change and no longer covered in the current git change.

The 2nd screenshot is indeed confusing to read.

Now I don't know why your code coverage changed. It may not be related to codecov.

In the project that I follow (vim) I noticed that code coverage recently dropped, apparently because coverage is now counted differently for if statements like this:

   if (foo) {
     bar();
   }

Previously, if foo was true in tests, the if block was entered, and all lines were covered. But now, the line with the 'if' is showing in yellow (for partially covered) if foo is always true. So now to have 100% coverage, we need tests that execute if (foo) with foo being true and tests with foo being false. It's more work to get 100% code coverage, but it makes sense and coverage is more accurate.

codecovdesign commented 2 years ago

@ahopkins @phh thank you citing the bug issue. We have identified the issue and are fixing it ASAP.

codecov / feedback

We'd love your feedback! #1

Thanks for dropping by! 👋

Thanks for dropping by! 👋