search

platformio / platformio-examples

PlatformIO Project Examples
Apache License 2.0
581 stars 175 forks source link

Unit testing examples conveniently omit the fact that IT COSTS MONEY TO UNIT TEST #24

Closed ianfixes closed 5 years ago

ianfixes commented 6 years ago

After seeing Travis CI docs that recommend PlatformIO for CI testing I put some effort into adapting the wiring-blink PlatformIO unit test example into my own sample project.

After a few hours of work (reading docs), and feeling pretty good about the progress, I ran platformio test -e uno and got this message:

$ platformio test -e uno
PlatformIO Plus (https://pioplus.com) v0.13.1
You are not logged in. Please log in to PIO Account using
`pio account login` command and try again.

If you don't have PIO Account yet, please create it using
`pio account register` command.

It's not unreasonable that you operate a service and charge money for it. However, it's inconsiderate that your documentation makes absolutely no mention of the fact that PlatformIO is a paid service, not a free program.

ivankravets commented 6 years ago

We have. See http://docs.platformio.org/en/latest/plus/unit-testing.html

New in version 3.0: (PlatformIO Plus)

Please click on "PlatformIO Plus"

ianfixes commented 6 years ago

Note that this bug report doesn't say "you didn't document it"; it says "you omitted it" from the example and the unit testing page itself. As a new user, it was not clear to me until I ran the unit test command that a login was required.

Based on your instruction to "Please click on "PlatformIO Plus"", you are assuming that (as a new user) I would know the following in advance:

  1. "New in version 3.0: (PlatformIO Plus)" is not a link to a changelog (indicating the version after which the feature is available) but instead a link to a pricing page that is a prerequisite for setting up unit testing.
  2. That I will need to scroll down the page to look for "PIO Unit testing", which appears below the fold
  3. That because unit testing appears under the paid plan listing, it is therefore not available as part of the free plan. Or that "plus" is meant in the "you pay for it" sense, not the "C++" sense.

Expected unit testing documentation

Unit testing is available in the paid plans of PlatformIO Plus, as of version 3.0. Account setup is a prerequisite for unit testing.

Actual unit testing documentation

New in version 3.0: (PlatformIO Plus)

Expected example documentation

  1. Create a PlatformIO account and sign up for a paid plan that includes PIO Unit testing
  2. Install PlatformIO Core
  3. Download examples source code
  4. Extract ZIP archive
  5. Run these commands: 
    <the command to log in to PlatformIO, and/or check whether you're logged in>

Actual example Documentation

  1. Install PlatformIO Core
  2. Download examples source code
  3. Extract ZIP archive
  4. Run these commands:
rjopson commented 5 years ago

I second this. I've spent about an hour getting everything set up only to realize it's paid. Not unreasonable, but would have been nice to know before I put the time in...

sleeper commented 5 years ago

Great! Actually In understood my problem thanks to this issue ;) Ok, so off from Platformio: paying to be able to do unit tests (especially the native ones) is a no go to.

ivankravets commented 5 years ago

Sorry, we are working on resolving this issue. Mostly, people pay for PIO Plus just to help us maintain everything that we do for PlatformIO.

Please note that PlatformIO is a fully neutral organization. We don't belong to any silicon vendor.

If you need PIO Unit Testing, please mail to contact@pioplus.com and we will upgrade your account for free.

pejgstoehl commented 5 years ago

More than half a year has passed and there is still no change. I also spent a morning getting things set up just to stumble across this issue.

The "New in version 3.0: (PIO Plus)" disclaimer is not clear enough by any means. I had no idea PIO Plus even existed until now and I have been using PlatformIO for several years. Please just add something like "This feature requires a paid PIO plus subscription". The way it is now it feels like you are trying to hide that fact.

ivankravets commented 5 years ago

Sorry for the delay!

PlatformIO Plus Goes Open Source, Improving Embedded Development Community Worldwide.

ianfixes commented 5 years ago

Which commit (or range of commits) contains the added functionality? https://github.com/platformio/platformio-core/commit/04dc6230e7f4a8799e90ca7f8e0e9bcc58019eb3 looks like a docs change. Also, does this functionality require a network connection to platform.io's servers?

ivankravets commented 5 years ago

It does not require Internet connection. See