search

KKoovalsky / PlatformIO-Helpers

Contains scripts and utils to improve PlatformIO usage experience
MIT License
10 stars 4 forks source link

[Enhancement] Get framework mbed path automatically #1

Open Copper-Bot opened 4 years ago

Copper-Bot commented 4 years ago

Hello !

First thanks for the script, it helps me a lot since the new PIO update with MBED 6 support, which explodes compilation time on my computer.

Just a bit enhancement in your script, instead of manually enter mbed framework path, you can get it automatically with this macro: mbed_os_dir = env['PROJECT_CORE_DIR'] + '/packages/framework-mbed'

It work both on Linux and Windows (don't have a Mac to test it).

Have a good day !

KKoovalsky commented 3 years ago

Thanks, that's a great input!

(Sorry for such a late reply. I didn't expect that somebody will ever create an issue here.)

Do you think that this improvement is still valid for implementation and further use?

Copper-Bot commented 3 years ago

Hello @KKoovalsky ,

Well, actually since this issue, I forked the project and find a better way to get the mbed framework folder: https://github.com/Copper-Bot/PlatformIO-Helpers/blob/master/mbedignore.py#L189-L197

With this trick, the script automatically get the right mbed path, in case someone is using a specifying MBED version (which change the mbed-framework folder name)

KKoovalsky commented 3 years ago

I see. How do you call the script? Do you use the extra_script.py file?

Copper-Bot commented 3 years ago

Not anymore, I call it directly in my platformio.ini file by using extra_scripts = pre:mbedignore.py

KKoovalsky commented 3 years ago

Ok, I didn't expect from the users to use this script as the actual extra_script. I think that, when I was writing the script, there was only one extra_script allowed. Then the scenario was to load and use any other utilities one needs, from the extra_script.py.

So the solution you propose is really good. I will try to find time during coming days and make the script loadable as an actual pre-extra script, without needing so much setup from the user.

Copper-Bot commented 3 years ago

I think the script could be added to PIO Core, there are just a few steps todo:

There was a lot of discussion about that feature some time ago, like this issue. Right now, PIO support .mbedignore if (and only if) the file is in the MBED framework folder. But every project that use MBED will automatically use this .mbedignore. If this script is used, it becomes project dependant. That could be a huge feature to add for PIO.

KKoovalsky commented 3 years ago

I am going to contribute to PIO with that feature, if it's still valid.

I think that we don't want to support the official .mbedignore file format. According to the documentation there is no possibility to create a single .mbedignore with all the paths to be ignored listed.

mbedignore.py is implemented in such a way that for each directory in PIOs .mbedignore it creates .mbedignore in MBED-os with pattern matching everything in that directory.

I would stick to that solution. We can rename the file to .pio_mbedignore for clarity.

@Copper-Bot what do you mean by?:

Find a better way to remove properly mbedignore file when not needed.

Copper-Bot commented 3 years ago

Hi @KKoovalsky,

By "supporting official .mbedignore file format", I was meaning: "When the script read the line mbed-os/connectivity/drivers/ble/*, it understands as connectivity/drivers/ble"

For an example, I took this file, and just removed the mbed-os at the start and the /* at the end to get this file instead, for the script to comprehend. Wouldn't be better if the script does that automatically, to improve user experience ?

About the line on the TODO list :

Found a better way to remove properly mbedignore file when not needed

Sometimes, I dont need the mbedignore script when creating a new project. BUT, the .mbedignore files are still in the PIO MBED framework folder, and building could go wrong. So right now, when necessary, I am launching a blank project with the mbedignore script and a clean .mbedignore, to force PIO to use the script to remove all .mbedignore files in the MBED framework folder. Kind of overkill ^^

But if the script is directly incorporated into PlatformIO, this problem should be no more. (When launching a new MBED project, if PIO doesn't detect any .mbedignore file in the root of the project, then remove all remaining .mbeignore files in the PIO MBED framework folder before start building).

Anyway, I still think this mbedignore script is valid and should be added to PIO 👍

KKoovalsky commented 3 years ago

Ok, I see.

Does it mean that there is only a single .mbedignore needed within the mbed-os? No need to create .mbedignore in each subdirectory that we want to ignore? @Copper-Bot could you confirm/deny?

That will fix everything, because what we need to do is:

  1. Put .mbedignore to the PIO project.
  2. During pio invocation, check if .mbedignore exists in the current project and copy it to the MBED OS directory.
  3. If .mbedignore doesn't exist in the current PIO project, remove the .mbedignore from the MBED OS directory, if exists.
  4. Done :question:

If there is no need to iterate through each ignored directory, then the script itself will be simplified a lot. (I know I could test that by myself, but I don't have pio at hand).

Copper-Bot commented 3 years ago

So,

What you pointed out would be the ideal solution. Unfortunately, after some tests and research, I discover that PIO doesn't care about the .mbedignore put in the ROOT of PIO MBED Framework folder. It needs to be split for each subfolder (not sub-subfolder or sub-sub-subfolder, only the first one).

I tried multiple tests to measure building time, with PIO CLEAN and PIO RE-INIT between each:

Judging by this documentation, this seems to be the normal PIO behaviour.

So I see two options here:

I can't find anything in the github of PIO related to the reading of the .mbedignore file... I don't understand how PIO read it.

To be more clear: Using the current version of PIO, If you have this .mbedignore file in the PIO project folder:

/* Device Key */
mbed-os/drivers/device_key/*

/* Cellular */
mbed-os/connectivity/cellular/*
mbed-os/connectivity/drivers/cellular/*
mbed-os/connectivity/netsocket/source/Cellular*.*

/* Tests */
mbed-os/TESTS/*

Then the script/feature of Platformio must create THREE .mbedignore files in the PIO MBED Framework folder with theses contents:

... and then only, it works.

@valeros is this the normal behaviour of PlatformIO NOT to use the .mbedignore file at the root of the PIO MBED Framework folder? Should we open an issue about that ? I'm using Windows 10 and PIO 5.2.0.

KKoovalsky commented 3 years ago

@Copper-Bot

  1. Regarding the mbed-os prefix in the .mbedignore, e.g. mbed-os/drivers/device_key/*: it comes from that it was used inside a project which uses mbed-os without pio, right? Your intention is to allow users to use the very same .mbedignore for raw mbed project and pio project with mbed as RTOS?
  2. When running a pure mbed project (without pio), the .mbedignore is put under the root directory project and MBED os sources are put under mbed-os, and this is why .mbedignore with lines like mbed-os/drivers/device_key work?

P.S. Thank you for the research and the benchmarks. I really appreciate your input and engagement :)

Copper-Bot commented 3 years ago
  1. Exactly.
  2. Yes. In a raw mbed project, the tool mbed-cli download MBED in a mbed-os folder at the root of the project (So, in every raw mbed project, there is an entire copy of MBED framework in a subfolder)
KKoovalsky commented 3 years ago

I have created a PR to platformio: https://github.com/platformio/builder-framework-mbed/pull/26

Let's hope it will be merged. ;)