search

aws-amplify / amplify-cli

The AWS Amplify CLI is a toolchain for simplifying serverless web and mobile development.
Apache License 2.0
2.81k stars 823 forks source link

Pain points summary from an external contributor. #5646

Open r0zar opened 3 years ago

r0zar commented 3 years ago

While most Issues are about quadrant 1, this one's about quadrant 2.

image

Overview

As a contributor to amplify-cli, I've noted a few pain points that both create friction for new developers to begin working on real bug fixes in the framework, and slow down the velocity of experienced external developers who are looking to make a positive impact on the project. I believe while these are not-urgent tasks, they are very important and directly relate to the health and ultimately success of the project.

Pain Points

Contributing is harder than it needs to be.

  1. There are too many Github Issues for anyone to effectively manage. It's hard to know what is a priority.
  2. The project roadmap is not visible to outside contributors. It's hard to know if my ideas on the future state of the framework align with the core teams.

My PRs take a long time to merge- is the current CI strategy working?

  1. The git hook on commit is a terrifically frustrating user experience. I'm certain there must be a better way to solve whatever problem this is trying to solve.
  2. There are a lot of CI integrations that (while in some cases adding value) distract from the more important metrics of what makes a contribution valuable.

It's hard to comprehend the code base.

  1. Intellisense does not work on some of the most crucial aspects of the code like context.amplify.helperUtils(). Dynamic imports and other dependencies techniques make it extremely hard to trace which functions are being called, what they do, and where they are defined.
  2. Types are inconsistently applied in the code. Would probably have been better off as just javascript for conciseness and a lower barrier to developer comprehensibility.

The project architecture is confusing.

  1. The code base is structured as a monorepo with lots of sub-modules, but there is no documentation on how to independently build and test packages outside of the monolithic CLI tools. It's unclear if its that's possible with cross dependencies. Theoretically, if they could be installed and run standalone, it would make it a lot easier to black box them in isolation and verify they are fully independent modules. What if FE devs could use a amplify env "lite" CLI for a shared-backend workflow?
  2. It's unclear how a new plugin transitions to become a core part of the CLI tools. I see similarities in the code structure, but no documentation on this.
  3. There is a lot of added complexity in the framework to support multiple providers, but the only provider that is supported and will receive support is (AFAIK) awscloudformation. Feels like a big case of YAGNI here.

There's a disconnect between user pain points and what contributors work on.

  1. Many believe bugs fixes should be prioritized higher than new features, if there is a reason this isn't the case, communicating it would alleviate their concerns.
  2. Refactoring seems to only be prioritized when coupled with adding support for new features. This is a problem faced by every software project, but it's one that will get harder to fix as time goes on.

Amplify CLI Version latest

To Reproduce Be an active open source contributor.

Expected behavior It shouldn't be this hard to contribute.

Additional context Odds are good I'm somewhere in the first half of this chart when it comes to working under-the-hood on the Amplify CLI code base. image I'm making this post now before I enter into the middle of it and resign myself to the problems and friction I felt early on.

If I wait until I'm in the second half, I'd almost certainly have forgotten all the pain points for a new contributor, and lost touch to the point where I wouldn't be able to relate to them or consider alternatives.

My points may not all make sense to someone who's been working on the framework for a long time. I know they do resonate with many who like me are trying to improve the framework, and feel there is more resistance than there should be.

If you believe this has value, please share/tag others. Otherwise, feel free to close.

renebrandel commented 3 years ago

hi @r0zar - thanks for summarizing this feedback! This will help us to rethink some of the current processes. I have a few follow-up questions to get more details:

  1. I'll bring this up with the team. I assume you mean the requirements around the commit names etc?

  2. Our CI integrations are increasing, we've done some optimizations to parallelize them to speed up the builds & tests but I'm curious which set of integrations are excessive. Our code obviously has significant impact to our customer's backend, so we want to make sure to have as extensive of testing as possible to prevent regressions and other bugs. We should look into how to optimize them as well. Happy to jump on a call together with some of our team members to walk through this.

--

  1. We've been trying to do a better job at communicating this. We've now got a "Bug bash" to track our bugs. We reprioritize them every Thursday to see what are some weekly incoming bugs that we can get the team to focus on. Overall the bug counts have been trending downwards and we've also added an explicit "Bug count banner" onto the repo README now to track that. Are you also looking to have tags for "first-good-issue" to encourage external contributors?

Curious to also hear more process suggestions. This issue is really insightful and thank you @r0zar for putting this together.

litwicki commented 3 years ago

@r0zar as @renebrandel said, thanks very much for such a thoughtful post. @renebrandel and I are going to dive deep into your feedback here. We take this very seriously and particularly the parts on the complexity of the codebase and the architecture are things we are already starting to discuss and put serious effort into improving.

r0zar commented 3 years ago

Thanks for your responses @renebrandel , @litwicki

  1. Mainly the whole process of not being able to commit unless using the guided process run by using yarn/npm run commit It takes me a couple tries per commit to get it right. If I mess up I have to start over. Dealing with the hidden character limits. Not knowing what component I should assign on multi-component concerns. Feeling responsible to look through 800+ issues to see if my PR interacts with any of them. On top of that, git hooks often don't play nicely with git extensions in editors.

  2. I think they are running fast and that's great! Typically, once a metric is set up people will naturally want to improve that metric, even if it adds little value to the end user. I guess my point is while we value things like code coverage, we should value things (for example) like number of errors thrown in user environments higher. Code coverage cant account for external factors like a user environments cloud or local state, and so I'm just cautioning against using these metrics synonymous with progress. FWIW; this is a really hard problem to solve that I've been thinking about my entire career. I don't have an answer here, but my instincts tell me more / faster tests don't address the fundamental issues at play.

  3. I really like the idea of the Bug Bash project. I've been watching my PR move along in there which has been great. I didn't know about the bug count banner, that's a smart idea too. Honestly, I don't have a lot to say about this point (and paradoxically have a lot to say) because I'm actively spending a fair bit time fixing issues. This whole post is really just an attempt to make the process of fixing issues easier for myself, and others like me. While I do think it's always good to have an option for newer devs to get involved, I think points 1-9 make it hard for them to make a impact beyond cleaning up trivial changes. I think a bug/refactor bounty program would be a realistic solution if it's feasible within your org.

jamesonwilliams commented 3 years ago

Thank you for sharing this feedback, @r0zar! I resonate with much of this. Some of these bullets have unique affinity to the amplify-cli. However, I see the majority of this feedback as being cross-cutting to the Amplify projects as a whole. We who focus primarily on the Amplify client libraries can also benefit from this discussion. cc: @jpignata @sammartinez @drochetti.

RossWilliams commented 3 years ago

Some additional contributor pain points:

  1. e2e tests, especially mock projects, print too many log lines. For new users this wall of text can be hard to parse. Ideally only failing tests should print log lines.
  2. Development environment setup instructions seem to be missing instructions. Did I notice recently that e2e tests now require Dynamo Local to be running? I didn't see any explicit instructions, but that seemed to have unblocked me.
  3. It is hard to contribute to the project when outstanding tasks or features do not appear to be tracked in GitHub
    1. Other projects use tags such as "good-first-issue" to point to bug fixes the community can make.
    2. Other projects take a feature request and turn them into acceptance criteria to signal to the community what it would take to accept a PR with the feature.
  4. I'm not clear what type of features would be accepted by the project. For example, I have a @ttl directive. There is not sufficient documentation to indicate if this would be accepted. It takes me a few hours to prep a PR, but I'd rather not burn a Sunday afternoon if it is unlikely to be merged.
  5. PRs from the community (or at least from me) can take a long time to get assigned for review, let alone merged.
    1. I have 4 current PRs submitted, mostly 6-8 weeks old. But I know they can move faster because security bug fixes have been reviewed and merged same-day.
    2. Another community pr for @auth 'source' has been sitting for something like 6 months without a review, or acknowledgement that it is being seen.

In general, I'm seeing the word 'community' being used quite a bit more. I'd like this to be accompanied by a 'community commitment' so that contributors and potential contributors are clear on expectations. I'd like to know how this project defines it, and whether it includes the following:

  1. Community members can gain commit access
  2. Roadmaps are published
  3. All work is managed in the open
  4. Planning sessions are conducted in the open
  5. Planning decisions and meeting notes are published
  6. PRs and issues from the community are given the same consideration as from AWS employees
  7. Community members are invited or able to be assigned to implement new features on the roadmap
  8. Standards for code, issues, and PRs are open and clear
    1. CONTRIBUTING.md suggests opening an issue to discuss features. This requires feedback and engagement in issues. Instead when a feature or bug is considered not important, there is no response.
  9. AWS employees are held to the same standards as community members
    1. Note that CONTRIBUTING.md suggests opening an issue before submitting a PR, I rarely see this applied to for AWS employees

edit: The microsoft/vscode wiki documents a IMO well-thought way to work with community contributors.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.