More 2102 ath79 config fixes

aimacintyre commented 1 year ago

Additional cleanups and fixes:

set devices without USB ports to gargoyle-basic profile and tighten the kernel options some more to further reduce image size - I think this has now got sysupgrade images building for all devices in the config, though I'm not sure a couple of the 8MB gargoyle-usb devices will have enough free flash space for settings to save even with the package changes below...
bring some plugin configuration settings up to date with recent package changes
set packages not required for the gargoyle-basic profile to build as installable (=m) rather than built-in (=y) to further reduce image size for gargoyle-basic devices

lantis1008 commented 1 year ago

I'm probably going to fold this into 2203 instead. Let me stew on this for a moment. I'd also like to bring in ipq40xx, bcm27xx and mt76x8. I've only just put 2 and 2 together and joined your forum username with github 🙃

obsy commented 1 year ago

Yes, please drop 21.02 and use 22.03 as base.

lantis1008 commented 1 year ago

ipq806x and x86.x64 are ready to use. I'm compile testing ath79 now.

I may work on migrating to nftables much further down the line as that will be significant work.

I've got some momentum at the moment but I go back to work Monday. Will try to keep moving on it. 2022 was a very busy year for me personally, I couldn't spend much time on this.

aimacintyre commented 1 year ago

Fair enough. I only noticed that you'd created a public 22.03 branch after I created the PR...

2 observations about 22.03:

not all arches are on DSA - particularly ipq806x (where DSA is a major performance regression)
it's going to be even harder to support 8MB flash devices :-(

If DSA vs swconfig doesn't significantly affect the Gargoyle code, no problem.

At this point I've come to the conclusion that supporting most of the 8MB flash devices - especially those with USB - will require using WolfSSL rather than OpenSSL for the -usb and -basic profiles. This would require some package development to have both variants of affected packages available and have the profile meta-packages pull in the best alternative. Thoughts on the value of this?

For a couple of 8MB -usb devices - the TL-WR902ac v3 comes to mind - I've wondered about a profile that is USB networking only (no local storage) as a compromise on function and likely usage (i.e. travel router) vs image size...

I've just been waiting until the build changes were pretty settled before PRing ipq40xx (all -large) and mt76x8 configs (mix of all 3 profiles) with a couple of additional devices over what I had for 19.07 - I can still do that against 22.03 if you like. I'd also like to add an ath79 nand profile to pick up the GL-AR300M and GL-AR750S (and anything else of interest in that OpenWrt profile).

For bcm27xx, the Pi, Pi2 and P3 profiles should be straightforward. For the pi 4 profile it would be great to support the DFR router board (CM4 + Realtek PCI 2nd NIC) and a couple of other CM4 "router" boards with built-in 2nd USB NICs. I haven't had a chance to play with my DFR + CM4 yet.

I'd also like to suggest that the bcm27xx profiles (all of them) get all the language plugins by default. I just think that particularly on this platform it would make Gargoyle more approachable for a wider audience upfront - the default SD card build easily has enough space and the language plugins aren't that big anyway. Doing this on a couple of other platforms with no shortage of storage such as x86 (generic and amd64 at least) might also have some value.

lantis1008 commented 1 year ago

If DSA vs swconfig doesn't significantly affect the Gargoyle code, no problem.

Already crossed that bridge so no problem. Was not that bad to deal with in the end.

At this point I've come to the conclusion that supporting most of the 8MB flash devices - especially those with USB - will require using WolfSSL rather than OpenSSL for the -usb and -basic profiles. This would require some package development to have both variants of affected packages available and have the profile meta-packages pull in the best alternative. Thoughts on the value of this?

I'd prefer mbedtls if at all possible, but wolfssl may need to be the midpoint. I'm open to switching some targets to wolfssl as needed. At this stage to use OpenVPN it is unavoidable to ship libopenssl due to our reliance on easyrsa. WolfCLU is still in its infancy.

For a couple of 8MB -usb devices - the TL-WR902ac v3 comes to mind - I've wondered about a profile that is USB networking only (no local storage) as a compromise on function and likely usage (i.e. travel router) vs image size...

Sure, sounds like another meta profile yes?

I've just been waiting until the build changes were pretty settled before PRing ipq40xx (all -large) and mt76x8 configs (mix of all 3 profiles) with a couple of additional devices over what I had for 19.07 - I can still do that against 22.03 if you like. I'd also like to add an ath79 nand profile to pick up the GL-AR300M and GL-AR750S (and anything else of interest in that OpenWrt profile).

Sure, to be a pain, is it possible to rebase this PR against 2203 (i assume all of it is still applicable?) and then is there anything else you wanted settled before progressing the other targets? I've already picked up ath79-nand and enabled the images I think are worthwhile.

For the pi 4 profile it would be great to support the DFR router board (CM4 + Realtek PCI 2nd NIC) and a couple of other CM4 "router" boards with built-in 2nd USB NICs. I haven't had a chance to play with my DFR + CM4 yet.

If you're happy to do the testing and push support for it, i'm happy to accept it 👍 I 100% support people who want to add devices that mean something to them.

I'd also like to suggest that the bcm27xx profiles (all of them) get all the language plugins by default. I just think that particularly on this platform it would make Gargoyle more approachable for a wider audience upfront - the default SD card build easily has enough space and the language plugins aren't that big anyway. Doing this on a couple of other platforms with no shortage of storage such as x86 (generic and amd64 at least) might also have some value.

Open to this, need to check again the i18n script impacts here. I have a funny recollection that it might try to \<M> all of the ones that aren't the "target language".

aimacintyre commented 1 year ago

Sorry, I tried to respond to this PR earlier via email but that appears to have vanished into the ether...

At this point I've come to the conclusion that supporting most of the 8MB flash devices - especially those with USB - will require using WolfSSL rather than OpenSSL for the -usb and -basic profiles. This would require some package development to have both variants of affected packages available and have the profile meta-packages pull in the best alternative. Thoughts on the value of this?

I'd prefer mbedtls if at all possible, but wolfssl may need to be the midpoint. I'm open to switching some targets to wolfssl as needed. At this stage to use OpenVPN it is unavoidable to ship libopenssl due to our reliance on easyrsa. WolfCLU is still in its infancy.

The only reason for proposing WolfSSL is that it's the only current alternative to OpenSSL for wpad/hostapd. I saw a reference to work being underway on adding the necessary support to mbedtls, but that's also an unknown on when it would be available.

I had in mind that OpenVPN was tied to OpenSSL, so was expecting that 8MB WolfSSL images could reject runtime installation of libopenssl dependent packages through the use of CONFLICTs in affected packages, though most of the 8MB devices will struggle to have enough free space for runtime installation of any substantial packages.

I'm suspecting that using WolfSSL would reduce the root FS sizes by ~300kB - not huge but potentially quite helpful especially for USB capable devices.

We may be able to get OpenVPN into 8MB devices without USB, in which case we could add OpenVPN into the basic profile meta-package for as long as that is sustainable.

For a couple of 8MB -usb devices - the TL-WR902ac v3 comes to mind - I've wondered about a profile that is USB networking only (no local storage) as a compromise on function and likely usage (i.e. travel router) vs image size...

Sure, sounds like another meta profile yes?

Yes. Once I have mt76x8 building on 22.03 I'll have a look at the dynamics (for the OpenVPN case above too).

Sure, to be a pain, is it possible to rebase this PR against 2203 (i assume all of it is still applicable?) and then is there anything else you wanted settled before progressing the other targets? I've already picked up ath79-nand and enabled the images I think are worthwhile.

As best I can tell, this PR wraps up the ath79 changes from my POV. I haven't yet pulled your latest 22.03 changes so don't know for sure what's not still applicable.

Are you able to give me any hints or pointers to recipes for doing the rebase you want? I'm still very much at the beginner stage with git :-(. I understand basically what the process does, but how to stitch things up across branches and repos is beyond my current levels of familiarity.

For the pi 4 profile it would be great to support the DFR router board (CM4 + Realtek PCI 2nd NIC) and a couple of other CM4 "router" boards with built-in 2nd USB NICs. I haven't had a chance to play with my DFR + CM4 yet.

If you're happy to do the testing and push support for it, i'm happy to accept it +1 I 100% support people who want to add devices that mean something to them.

Apart from me having one, I'm hoping that Gargoyle natively supporting some of the CM4 router setups out of the box might attract some users - with OpenWrt it can be a bit awkward to get these devices fully operational.

I'd also like to suggest that the bcm27xx profiles (all of them) get all the language plugins by default. I just think that particularly on this platform it would make Gargoyle more approachable for a wider audience upfront - the default SD card build easily has enough space and the language plugins aren't that big anyway. Doing this on a couple of other platforms with no shortage of storage such as x86 (generic and amd64 at least) might also have some value.

Open to this, need to check again the i18n script impacts here. I have a funny recollection that it might try to all of the ones that aren't the "target language".

My existing 19.07 bcm27xx images already have all languages installed. I tested the Pi2 image and that worked normally - although I didn't change language my recollection is that the displayed package list in the UI reported all the expected language packages (I included all the theme plugins in the Pi images as well and definitely changed theme successfully). As of a short while ago there have been 23 downloads of the bcm27xx archives from which I've had no feedback - either it's all working as it should (if anyone changed the language) or there was nothing worthwhile and all downloaders moved on without comment...

Providing that there aren't any i18n script impacts, I can see 2 ways to do this:

in a diffconfig or diffconfig arrangement, you just set all the language packages "=y" rather than "=m"
the language plugins build state can be set via Config-gargoyle.in.

IIUC the Config-gargoyle.in approach would look something like this:

config GARGOYLE_MULTI_LANGUAGE
    bool "Include all language translations"

config GARGOYLE_LANGUAGE_PKGS
    tristate "Language translation packages"
    default y if GARGOYLE_MULTI_LANGUAGE
    default m
    select PACKAGE_plugin-gargoyle-i18n-Arabic-AR
    ...

The device config (or diffconfig) would then just include GARGOYLE_MULTI_LANGUAGE=y to include all the languages in the profile's images whenever defconfig/menuconfig was run.

aimacintyre commented 1 year ago

I've had a look and the merge into 22.03 appears messy enough to me that I'd rather you just close this PR without applying it and I'll start a new one directly against the 22.03 branch.

lantis1008 commented 1 year ago

All OK with me. And yes closing and starting again on 2203 is the easiest way in my opinion. If you could close it though that would be great. I don't have any special rights on this git instance so I can't administer it. I can push to the actual Gargoyle git instance which mirrors to here but that's it.

By the way, the CONFLICTS on the different SSL libs may not be recommended as we don't want someone with USB to be prevented from installing as many SSL libs as they want to run the packages they want.

lantis1008 commented 1 year ago

BTW @obsy, everything compiles now so feel free to explore. I need to do some work on ramips as not many images are generating, i expect they are too big.

aimacintyre commented 1 year ago

Ok, will do.

BTW, with ramips check for device renames too, particularly in _profileimages - there were quite a few of those in ath79 between 19.07 and 22.03, some of them quite subtle.

aimacintyre commented 1 year ago

Closing - to be superceded with a 22.03 based PR.

ericpaulbishop / gargoyle

More 2102 ath79 config fixes #963