Track: FRR Quagga fork

[mimugmail]

https://github.com/FRRouting/frr

Just to track the steps this project goes. It's a very active fork of Quagga and currently implementing EIGRP, NHRP and ISIS. Some limitations on BSD, some only for OpenBSD.

pfSense will replace their mixup with this.

ATM I don't see any reason to go for it but it seems worth to track.

Changelog FRR 3.0: https://github.com/FRRouting/frr/wiki/FRR-2.0-%E2%86%92-FRR-3.0-Changelog

[obrienmd]

Given support from the Linux foundation, a more open dev model with well-documented governance, and (less absolutely positive but I like) Github-based development, I'd love OPNSense to give this serious consideration.

FRR already has a number of interesting improvements over Quagga (BGP AddPath, BGP unnumbered, 32-bit route tags, ldpd, etc.), and development velocity seems pretty fantastic ATM: https://github.com/FRRouting/frr/pulse

[obrienmd]

pfSense, as it stands right now, has frr as a package which is mutually exclusive with the quagga-based routing packages. Given quagga is not in core, perhaps a "middle way" with less risk would be to allow both a quagga-based routing package and an frr-based one?

[fichtner]

I would think having both would be less beneficial in the future. We should pick the one that gives us the best results in the mid and especially long term...

@fabianfrz and @mimugmail should decide, they work on the plugin regularly.

[fabianfrz]

@fichtner I think we cannot support both but if we switch we will have to be careful not to break old installations. this means we would have to move the plugin to frr and provide a meta package called quagga with a newer version which installs the frr plugin as a dependency.

If the vtysh output and the config file format has not changed, we can simply switch, otherwise we will have to adjust the plugin code (especially the regular expressions in quagga.rb).

[obrienmd]

I ran through the migration with a few non-trivial configs on a linux box (BGP, OSPF, didn't try a RIP config). Config files were not changed and worked as expected, and vtysh output didn't change.

Cumulus has a doc on the transition on their OS which provides useful context: https://docs.cumulusnetworks.com/display/DOCS/Upgrading+from+Quagga+to+FRRouting

The commit activity on frr is astounding. On its own, that's not necessarily a good thing (tm), but most of it seems to not be window dressing - real outstanding bugfixes and cleanup that have been long-standing on Quagga are being resolved.

[mimugmail]

Recently there was a discussion on the Quagga ML:

https://lists.quagga.net/pipermail/quagga-users/2017-August/014824.html

Seems there are some global players not interested in any changes since it runs fine for them :)

[obrienmd]

I read that differently - that entrenched interests are not super interested in a continuously improving open source routing engine competing against their wares :)

[fichtner]

frr will be in packages starting with 17.7.3. Do you guys want to keep discussing or should we close this ticket in favour of action-based follow up tickets?

[obrienmd]

Love it. I'm open to closing this up, and look forward to testing 17.7.3. Do you think it will drop by end of September? I have a deployment starting testing the last week of September that I'd like to try frr out on, based on some initial solid results I've had with frr in linux-land.

[fichtner]

17.7.3 is scheduled for Tuesday if all goes well. Worst case Wednesday. 😉 I can't speak for pulling the old switcheroo with the plugin itself (using the quagga configuration logic with frr underneath), but the vanilla package will certainly work just as good as quagga in its place.

On 17. Sep 2017, at 22:10, Michael OBrien notifications@github.com wrote:

Love it. I'm open to closing this up, and look forward to testing 17.7.3. Do you think it will drop by end of September? I have a deployment starting testing the last week of September that I'd like to try frr out on, based on some initial solid results I've had with frr in linux-land.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[obrienmd]

Awesome, thanks :) Looking forward to playing with it. No need for the switcheroo, this is a pretty minimal config.

[fabianfrz]

I have to look how much time I have but I may be able to give it a try.

[obrienmd]

Quick report: Installed frr via pkg install, replaced quagga, configured manually in /usr/local/etc/frr/ files, and OSPF is working beautifully against 2 other OPNSense hosts running quagga.

[obrienmd]

@fichtner I take back my previous comment, and add a couple reports - if you want to switch this to separate action-based issues please feel free!

• The previous 'success' was the box rememering Quagga routes.
• When I try to start ospfd using the same conf file Quagga used, I get, for each interface: MPLS-TE(initialize_linkparams) Could not find corresponding OSPF Interface for (e.g.) igb0
• I suspect this might be an issue with perms for the frr user, given the following:
• vtysh is running as frr user, so 'copy run start' fails in vtysh (though I don't know if that's critical, as I don't know if OPNSense's Quagga stuff will use vtysh), as it cannot write to the root:wheel owned /usr/local/etc/frr/* files. chowning that dir -R as frr:wheel solved this problem, but may cause many others :)

[fabianfrz]

@obrienmd the current quagga files are owned by quagga - this is done by the setup script. VTYSH is used by quagga.rb for printing out the routing tables etc.

[obrienmd]

Thanks for the context - so, my manual chown of the frr config files should have worked. I'll dig into the OSPF error message.

[obrienmd]

Still trying to figure this one out. It can clearly see the interfaces in 'show interface' on vtysh, but when I try to start the ospf service, it just pukes with that error. Odd one.

[obrienmd]

@fichtner would it be possible to package a clone of the os-quagga UI plugin for frr, so I can see if GUI config just implements something I'm missing?

Not sure how big an ask this is, i.e. if it's just pointing it to a different config directory, or something more complex. Happy if it's broken, not-qc, and I'm just a guinea pig :)

[fichtner]

I wouldn't want a full clone yet, maybe we can build a type of QUAGGA_IS_FRR switch into the current quagga port?

[fichtner]

On the plugins.git level, not from the GUI. /CC @fabianfrz @mimugmail

[fabianfrz]

@fichtner there are also some path adjustments so it will not completely work. 'quagga' -> 'frr' However the models and controllers will be probably untouched which means in worst case we have to add some changes to the config files - but I will probably not work on it now.

[obrienmd]

@fichtner @fabianfrz I'd need some guidance, but I'd be happy to help!

[obrienmd]

@fichtner @fabianfrz Note, I have a fork of plugins with os-frr alongside os-quagga (not installed at same time), and it works as well as manual config did (i.e. still has MPLS-TE(initialize_linkparams) Could not find corresponding OSPF Interface for (e.g.) igb0 error on all ints).

I'm digging in with frr on slack on that right now. Edit: Or would if it was an open slack workspace, apparently :)

[obrienmd]

Ha! Apparently those MPLS-TE log messages are warnings. With my find/replace version of os-quagga installed as os-frr, OSPFv2 is working as expected. Awesome!

Plugins fork with os-frr as 0.1.1 is here: https://github.com/obrienmd/plugins

Let me know if you want a pull request - I suspect based on previous comments probably not :) Hope the fork is helpful for experimentation regardless, if only just to save a few minutes of mundane case-sensitive find/replace.

[fichtner]

Nice, we can put the changes in a branch and try to integrate them so they expand on plugin build/install. That way we can keep and test both and decide about a replacement timeline. This can take a few months, but come 18.1 we should have a clear decision / strategy of what we want to use in the future. Sounds good?

All I’m saying is we should not carry both. A similar decision may be OpenSSL/LibreSSL in the long term... :)

On 27. Sep 2017, at 18:29, Michael OBrien notifications@github.com wrote:

Ha! Apparently those MPLS-TE log messages are warnings. With my find/replace version of os-quagga installed as os-frr, OSPFv2 is working as expected. Awesome!

Fork is here: https://github.com/obrienmd/plugins

Let me know if you want a pull request - I suspect based on previous comments probably not :) Hope the fork is helpful for experimentation regardless, if only just to save a few minutes of mundane case-sensitive find/replace.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[mimugmail]

I'd wait till v3 .. no matter if 18.1 is behind or ahead :)

[fichtner]

Fair enough. All the more reason to integrate FRR into the current port so that new features benefit both flavours. I’ll look at this in detail in October.

On 27. Sep 2017, at 18:46, Michael notifications@github.com wrote:

I'd wait till v3 .. no matter if 18.1 is behind or ahead :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[obrienmd]

Makes sense. Let me know if I can help in any way!

[fabianfrz]

I think @mimugmail is right - there is a RC available at the moment: https://github.com/FRRouting/frr/releases

So for integration waiting for stable is probably the best idea before any changes are done.

[obrienmd]

Cool, makes sense.

[mimugmail]

I'm going to love FRR and hate Quagga. All my 4 core routers were affected by this:

https://lists.quagga.net/pipermail/quagga-users/2017-September/014830.html

All iBGPs were flapping and 2 of the 4 cores crashed. If 4 were crashed the complete AS would be gone.

FFR fixed this in code ... and Quagga will stay with it I believe. :/ https://github.com/donaldsharp/frr/commit/084002351fbfd6b4e2d9c4c218288b2324ad81cd

[fabianfrz]

@mimugmail you can ask @fichtner if he can build an rc package of frr 3.0 for you and then you can start porting your stuff. Just keep everything except those things which have be changed (paths, filenames etc.) to keep the plugin functional and don't loose the configuration (model refers to the same XML element).

[obrienmd]

@fichtner I'd be interested in that rc package of frr 3.0 as well!

[fichtner]

3.0 seems to have been released: https://github.com/FRRouting/frr/releases/tag/frr-3.0

I'm working on a port update at the moment. :)

[fichtner]

Done, packages tomorrow... https://github.com/opnsense/ports/commit/5f90b819d

I can't find a 3.0 release announcement, looks like this either isn't final or still being assembled...

[fichtner]

We have pushed FRR to FreeBSD ports as well. It looks like it's working ;)

[fichtner]

FRR will be version 3.0.2 in 17.7.8. If anyone wants to start migrating/copying the quagga plugin, that would be a good time to start it on our way to 18.1 ;)

From the model perspective, maybe we can keep "quagga" naming, but only change the internal service usage.

[fichtner]

Plugin exists, probably hitting 1.0 with 18.1.

[fabianfrz]

@fichtner if this is released, os-frr should be released at the same time.

[fichtner]

@fabianfrz yes, this will both automatically land on stable/18.1. maybe os-frr should be bumped right now to 1.0 as well?

[fabianfrz]

sure

[fichtner]

great, thanks.. see https://github.com/opnsense/plugins/commit/7a8fe4a6