Implement newer syntax changes

[moqmar]

For example the new override syntax from https://docs.yoctoproject.org/migration-guides/migration-3.4.html, or stuff like SRC_URI[name.md5sum], currently breaks the highlighter.

[ShaneMurphy2]

Out of curiosity, anyone know why they changed from _prepend to :prepend and the like?

[moqmar]

That's explained in the link above, seems like it makes it easier to parse internally and also easier to read:

For reference, this conversion is important as it allows BitBake to more reliably determine what is an override and what is not, as underscores are also used in variable names without intending to be overrides. This should allow us to proceed with other syntax improvements and simplifications for usability. It also means BitBake no longer has to guess and maintain large lookup lists just in case e.g. functionname in my_functionname is an override, and thus should improve efficiency.

[jaskij]

@ShaneMurphy2 on the easier to read part: try porting files from old syntax to new syntax manually. You will make mistakes. Showing that yes, it's hard to know what is and what isn't an override.

[DavidAntliff]

This plugin is almost working with the new syntax, but it's still not quite right.

I see there's a pending PR to add support: https://github.com/vitalibo/bitbake-intellij-plugin/pull/8

@vitalibo what would it take to get this PR into the published plugin? Maybe we could get a build to test, please?

[moqmar]

Here's a build of the PR branch of #8, feel free to test with it:

BitBake-1.3.zip BitBake-1.3.zip

I'll merge it in a couple of days, but would be glad to get some feedback from people who're already using the newer syntax.

[DavidAntliff]

Here's a build of the PR branch of #8, feel free to test with it:

BitBake-1.3.zip

I'll merge it in a couple of days, but would be glad to get some feedback from people who're already using the newer syntax.

Thanks - I gave it a try (installed, restarted PyCharm Pro) but it doesn't seem to be catching all cases:

It's difficult to tell but I don't think it's picking up the new syntax at all?

EDIT: Just to be sure, I completely uninstalled and then reinstalled from the .zip but I still see the same as the screenshots above.

[moqmar]

Whoops, sorry, I've uploaded the wrong file - link should now be fixed.

[DavidAntliff]

That works much better - thank you.

There are still some issues with some of the .bb and .bbclass files I am working with:

In a .bbclass file: meta-swupdate/classes-recipe/swupdate.bbclass:

Another .bbclass: meta-xilinx-tools/classes/fpgamanager_dtg.bbclass: (followed by "Unresolved reference 'd'", which might be a global?), then for the underlined python devicetree_do_compile:append() there's "BitBakeTokenType.OPERATOR expected, got 'devicetree_do_compil...'".

Same file - no tooltip for the underlined "." in fw_env.config:

There are others too, but they may be caused by those. Happy to help keep testing new builds.

[ShaneMurphy2]

I'm also trying out the new build, seeing similar errors to @DavidAntliff.

[DavidAntliff]

Just leaving a comment to say that I'm still happy to test any updates to this plugin. I have a whole bitbake project with red squiggles I can set a new build of this plugin against, and report back any remaining cases :)

[fbezdeka]

So... Here we go. I updated #8 with the result of my one day holiday hacking. Feedback welcome! Testing will start here as well.

[DavidAntliff]

I'm trying the version from this build and it's working better overall, but I do see some irregularities still. Here's a few:

[fbezdeka]

Thanks for providing the feedback!

I think these are different issues and should not be mixed with the "newer syntax" issue here. Could you open two different issues (as you are the real reporter I think this makes sense) please?

• One for a broken comment parsing (## at the beginning of a line - with no further content - breaks the parser).
• One for export statements require a value assignment (export VAR breaks the parser, while export VAR="value" is OK)

I'm not able to reproduce the "illegal target for variable annotation" yet.

I can handle the "illegal target for variable annotation" in this MR. It's kind of related. Bug in the python language injection.

[fbezdeka]

I updated #8 again. Latest build artifacts can be fetched from here: https://github.com/fbezdeka/bitbake-intellij-plugin/actions/runs/10202860522/artifacts/1765892776

I think I covered all reported problems. Feedback welcome.

[DavidAntliff]

@fbezdeka thanks, I've just installed this and it seems to have cleared up the issues above - fantastic.

I am now going through a big set of .bb files looking for any outstanding or new cases.

In most cases I can provide the .bb file if that helps - just ask...

... looks like there's only a few left now:

That "= after export" one you mentioned before, I suspect:

(Also happens with the := in export FOO := "...").

I guess this next one is just because there's no active Python environment when editing .bb files:

This is from a weird Xilinx file. I'm sure what they are doing with that import before function docstring, but the syntax highlighting seems to turn off, and there's a strange error right at the end:

Other than those it's looking really good - almost there!

Thank you.

[DavidAntliff]

I think these are different issues and should not be mixed with the "newer syntax" issue here. Could you open two different issues (as you are the real reporter I think this makes sense) please?

* One for a broken comment parsing (## at the beginning of a line - with no further content - breaks the parser).

* One for export statements require a value assignment (`export VAR` breaks the parser, while `export VAR="value"` is OK)

I think the first is now fixed. Would you still like me to open a new issue for the second one?

There's also EXPORT_FUNCTIONS as per screenshot above.

[fbezdeka]

Everything related to export and EXPORT_FUNCTION should be fixed now. The latest build result is linked in #8.

The python "no module named ..." might be fixed by proper configuration of the python interpreter. I think we can't handle that on plugin level.

The last remaining is the broken highlighting for this python function. Might it be that this function is at the end of the file? Could you provide the file? No longer necessary. I was able to reproduce. Already fixed in #8

[DavidAntliff]

@fbezdeka looks really good! This is such a great improvement. Thank you.

There are two issues remaining that I can see. I'll mention them here but if you'd prefer a new issue then let me know and I'll create one (or two).

This is from a .inc file, commonly included by a .bb:

Multiple EXPORT_FUNCTIONS on the same line:

I took a look into the python interpreter. I'm using IDEA so this is set under Project Structure > Project Settings > Project, where I have the Project SDK set as Python 3.10:

I see the breadcrumbs at the bottom of the IDE showing it as "... > externalxsctsrc.bbclass > \<Injected Python file>".

But then I found this, so removing the Project Settings > Modules entry and then creating a new Modules entry with an existing Interpreter fixed this issue. So I think we're good on that one now.

[fbezdeka]

Hopefully the last two remaining issues (flags parsing + multiple functions in EXPORT_FUNCTIONS) are fixed now. MR has been updated.

[DavidAntliff]

Thanks @fbezdeka, it looks really good to me. I found a couple of minor issues but I can create new issues for them at some stage, they are minor and don't relate to any of the issues reported above.

Thank you again for your efforts with this, it will make it a much better experience when working with .bb files.

Let's get this reviewed and merged. Pinging @vitalibo...

[fbezdeka]

8 was merged. Closing as everything else should be handled in separate issues.