search

duckduckgo / p5-app-duckpan

DuckDuckHack OpenSource Development Application
http://metacpan.org/module/App::DuckPAN
Other
53 stars 47 forks source link

IA's are untestable until PR is made #380

Closed mintsoft closed 7 years ago

mintsoft commented 8 years ago

Hello,

When I knock up IA's I usually whack out a working version (walking skeleton, whatever you want to call it) before I do the PR. At the moment, IA's are untestable through duckpan until a PR is made. There's also a delay between the PR being made and the metadata being updated. So 'quickly knocking up an IA' takes like 15 minutes for what (I think) should be a 2 minute job!

Thoughts?

It was changed in https://github.com/duckduckgo/p5-app-duckpan/commit/ebb557dfb76b8c23c540d7aab86b95b684d30209 I'm guessing it's something to do with cheatsheets @GuiltyDolphin ?

Thanks, Rob

GuiltyDolphin commented 8 years ago

@mintsoft We use the metadata to look up some information about the Instant Answers (rather than relying on (potentially broken) convention). We also generally want users to make a pull request so we can help out from an early stage, though this does potentially negatively affect some 'super-users'.

I have noticed that it sometimes does take quite a while for the metadata to update, which can be annoying.

Perhaps we should provide an 'override' flag that will attempt to determine the necessary data from the IA name, but with the provision that it may not be entirely correct.

mintsoft commented 7 years ago

I agree in that we want to encourage users to make PRs so we can get involved, but this has actually broken the development process. Just try and make a new instant answer and follow the instructions that you'll be given at certain steps and it doesn't work.

If the instructions given within the IA pipeline on duck.co were changed to provide accurate instructions then they'd be hilarious nonsense.

Basically the flow is as follows:

  1. Create IA idea on duck.co
  2. Fork the repo
  3. Create an instant answer (i.e. create a .pm file) - at this point it's impossible to know if you've done it correctly or not as it simply will not work no matter what. If you're a new user, how on earth are you supposed to know whether what you've done here is correct or not?
  4. Commit and push into the repo blindly
  5. Create a PR
  6. Wait an undetermined period of time until magically the status is changed - again, you have no way of knowing this without refreshing either the PR or duck.co/ia until the page changes
  7. Clear the cache on local duckpan (assuming you know that this is a thing)
  8. Test Step 3 and now iterate until the IA works.

This is mental in my opinion.

I also totally disagree with an override flag, the default behaviour which is what the least experienced people will use, should work; it doesn't.

GuiltyDolphin commented 7 years ago

@mintsoft I think the workflow is supposed to be along the lines of:

  1. Create IA Page.
  2. Use duckpan new to create initial files.
  3. Push to repo.
  4. Test, build, develop... If you get a notice that the IA already exists then stop!

Unfortunately, due to the delays the step between 3 & 4 is a bit broken. It should really be instantaneous.

The alternative to a 'development' stage is just to put all the IAs in the metadata, and have more rigorous reviews to prevent spam.

mintsoft commented 7 years ago

Well with the current requirement for a PR prior to testing (even locally) it has to be:

  1. Create IA Page
  2. Create new IA
  3. Push to repo/fork whatever
  4. Open PR!
  5. Test stuff

I don't think delays are necessarily the problem; it's the jumping around to different bits of magic information in order to progress. When an IA page is created there's an info icon that says something along the lines of "what next" and it basically says "create your IA using codio". But that can't actually be done until the user has created it, committed it, pushed it; then gone to GitHub and created a PR. You can't put that in the description of what to do next because it's convoluted!

GuiltyDolphin commented 7 years ago

@mintsoft Ah, yeah, sorry - that was supposed to be after 'push'.

Hmm, if possible, it might be nice to have a way of selecting that you would like to start work on the IA, then it create an appropriate branch including the initial files, and perhaps even a PR? I'm not sure how extensive the GH API is, but I'd imagine this were possible.

Perhaps we could try and have the main body of metadata, then if an IA is not found within that it can do a quick API/GET to grab metadata for the specific IA?

ihatetoregister commented 7 years ago

I'd like to see a way to bypass the need of an IA page to enable us to create, test and play around with IAs that are not currently fitting the focus group. My developent is currently stalled becase of this: https://github.com/duckduckgo/zeroclickinfo-spice/pull/3039

mintsoft commented 7 years ago

@ihatetoregister There is a PR in-flight switching the error out to a warning: https://github.com/duckduckgo/p5-app-duckpan/pull/381 if you're comfortable with perl stuff you could replace your duckpan with that branch and test it out against the usecase you've described

ihatetoregister commented 7 years ago

@mintsoft That's great to hear. Unfortunately I don't know how to replace duckpan, but I'll try to find out when I got time, also I'm on Codio. Otherwise I guess it will be available in an official release soon? So a duckpan update/upgrade should do it then?

mintsoft commented 7 years ago

Fixed in: https://github.com/duckduckgo/p5-app-duckpan/pull/383