This Simply Doesn't Work.

[jbhorner]

This does not generate a usable XML file for OPNSense. Using version 23.7. Based on my manual review of the file, this does not work with earler versions, either.

We can start with line 1--the absence of <?xml version="1.0"?> in the download.

[jerry110]

in my test environment i will second that .... it corrupts the root password as well

[mwood77]

Hi @jbhorner and @jerry110 👋

I'll need to see your config files to see what fields/objects are missing in the generated output. The config mapper is implemented in a 'best effort' type manner - it'll map known fields and include unknown fields as they are (aka. it will not remap pfsense fields, but will include them as is if it doesn't know what to map them to).

---

in my test environment i will second that .... it corrupts the root password as well

@jerry110 is it only the root password? What is the structure of the root password's user credentials object?

We can start with line 1--the absence of in the download.

@jbhorner I don't understand what this means. Can you please explain this a bit more, or tell me what fields you expect to be remapped?

[jerry110]

Hi @jbhorner and @jerry110 👋

I'll need to see your config files to see what fields/objects are missing in the generated output. The config mapper is implemented in a 'best effort' type manner - it'll map known fields and include unknown fields as they are (aka. it will not remap pfsense fields, but will include them as is if it doesn't know what to map them to).

in my test environment i will second that .... it corrupts the root password as well

@jerry110 is it only the root password? What is the structure of the root password's user credentials object?

We can start with line 1--the absence of in the download.

@jbhorner I don't understand what this means. Can you please explain this a bit more, or tell me what fields you expect to be remapped?

I sent you an email from your website

[mwood77]

@jerry110 it didn't come through - looks like another thing to fix.

In the mean time, you can email me here: pf2opn@protonmail.com

[jbhorner]

The first line of the file: <?xml version="1.0"?> is absent. That is what I was referring to. Does it matter? I don't know. But it is a variation from an OPNSense XML output file.

[mwood77]

The first line of the file: <?xml version="1.0"?> is absent. That is what I was referring to. Does it matter? I don't know.

<?xml version="1.0"?> is an optional metadata-type declaration and not strictly required. Do you know if your file is failing at a certain point beyond that?

[jbhorner]

The first line of the file: is absent. That is what I was referring to. Does it matter? I don't know.

<?xml version="1.0"?> is an optional metadata-type declaration and not strictly required. Do you know if your file is failing at a certain point beyond that?

Inserting that does not change the failure; it was the first thing I tried. OPNSense does not indicate where the failure happens on a restore; it's a binary response. If it works it loads the file and reboots. If not, it says it failed.

My particular configuration is beyond a simple configuration. I have multiple VLANs, IPv6 (tracking and DHCPv6), Wireguard, OpenVPN, TOTP integration, multiple DHCP static assignment, and DNS overrides. My firewall rules are likely what one would expect for this type of configuration.

The method of mapping you describe above is a double-edged sword. If it were me, I would have only mapped the fields I knew about and note the others in some type of exception file. It's an incremental delivery at that point; many might not mind the incomplete conversion as long as they have a usable XML file and reference as to what they might need to build beyond what they've successfully restored. It would also provide a means by which they could provide feedback and potential XML snippets for the areas that could not be converted.

I don't think many people will provide a full file, but I could be wrong. It's a pretty big ask, as it is essentially the keys to the kingdom. Trying to remove the sensitive data wouldn't be an easy task, so doing that before sending would be almost as big of an ask.

[mwood77]

The first line of the file: is absent. That is what I was referring to. Does it matter? I don't know.

<?xml version="1.0"?> is an optional metadata-type declaration and not strictly required. Do you know if your file is failing at a certain point beyond that?

Inserting that does not change the failure; it was the first thing I tried. OPNSense does not indicate where the failure happens on a restore; it's a binary response. If it works it loads the file and reboots. If not, it says it failed.

My particular configuration is beyond a simple configuration. I have multiple VLANs, IPv6 (tracking and DHCPv6), Wireguard, OpenVPN, TOTP integration, multiple DHCP static assignment, and DNS overrides. My firewall rules are likely what one would expect for this type of configuration.

If I had to take a guess, I'd wager the rule mappings might be incorrect depending on the complexity of your setup. Alternatively, it could've been that your password mapping was also sha-512 hashed. This has been fixed with release v0.1.4. If your config file still doesn't work, please read on.

The method of mapping you describe above is a double-edged sword. If it were me, I would have only mapped the fields I knew about and note the others in some type of exception file. It's an incremental delivery at that point; many might not mind the incomplete conversion as long as they have a usable XML file and reference as to what they might need to build beyond what they've successfully restored. It would also provide a means by which they could provide feedback and potential XML snippets for the areas that could not be converted.

This is a great idea! I'd love to see a PR introducing an optional incremental conversion method.

I don't think many people will provide a full file, but I could be wrong. It's a pretty big ask, as it is essentially the keys to the kingdom. Trying to remove the sensitive data wouldn't be an easy task, so doing that before sending would be almost as big of an ask.

Most do and I've seen some pretty complex ones (5000+ line config files). If you're willing to help out, you can email me a sanitized version (or snippets) of your config at: pf2opn@protonmail.com

[jbhorner]

If I had to take a guess, I'd wager the rule mappings might be incorrect depending on the complexity of your setup. Alternatively, it could've been that your password mapping was also sha-512 hashed. This has been fixed with release v0.1.4. If your config file still doesn't work, please read on.

It failed. As I mentioned in my original feedback, I believe there are more fundamental issues here. Below I've provided a snippet of one area where there is a difference between what a converted file looks like, and what a true OPNSense file looks like:

OPNSense Actual
    <user>
      <name>root</name>
      <descr>System Administrator</descr>
      <scope>system</scope>
      <group-name>admins</groupname>
      <password>Some Hashed Password</password>
      <uid>0</uid>
    </user>
    <user>
      <password>Some Hashed Password</password>
      <scope>user</scope>
      <name>ActualUser</name>
      <descr>Actual User Name</descr>
      <expires/>
      <authorized keys/>
      <ipsecpsk/>
      <otp_seed/>
      <shell>/bin/tcsh</shell>
      <uid>2000</uid>
    </user>

Converted pFsense
    <user>
      <0>
        <name>admin</name>
        <descr>System Administrator</descr>
        <scope>system</scope>
        <groupname>admins</groupname>
        <uid>0</uid>
        <priv>user-shell-access</priv>
        <expires></expires>
        <dashboardcolumns>2</dashboardcolumns>
        <authorizedkeys></authorizedkeys>
        <ipsecpsk></ipsecpsk>
        <webguicss>pfSense.css</webguicss>
        <keephistory></keephistory>
        <bcrypt-hash>Some Hashed Password</bcrypt-hash>
      </0>
      <1>
        <scope>user</scope>
        <bcrypt-hash>Some Hashed Password</bcrypt-hash>
        <descr>User Name</descr>
        <name>UserID</name>
        <expires></expires>
        <dashboardcolumns>2</dashboardcolumns>
        <authorizedkeys></authorizedkeys>
        <ipsecpsk></ipsecpsk>
        <webguicss>pfSense.css</webguicss>
        <keephistory></keephistory>
        <uid>2000</uid>
      </1>
    </user>

As noted, I'm not expert here. With that said, it would seem to me that the goal would be to mirror--as closely as possible--what an actual OPNSense XML file looks like. The construct of the user area is a pretty big departure.

When I embarked on implementing OPNSense a couple of years ago, I manually copied over the XML areas of my pFsense that were most important to me (DHCP static leases, VLAN configurations, Interface definitions (inclusive of a bridge), and DNS overrides)--sometimes converting when needed. There was a lot of profanity and hair pulling, but ultimately I figured out how to do it easily. The other areas--no so much.

Most do and I've seen some pretty complex ones (5000+ line config files).

That's great for them. With many years in IT behind me, and several recent in InfoSec, I can't bring myself to do that. I might spot check areas later and provide feedback. If you do build something that would show exceptions, I'll run that and send sanitized output. Out of curiosity, I looked at the number of lines in my pFsense file; it's 12,378 lines.

[mwood77]

The indexing of K user entities was an issue I was unaware of. I've included a fix for this, along with other items in #11

Regarding your example, your OPNSense Actual is not representative of a valid configuration file, as it contains invalid XML; <authorized keys/> and <group-name>admins</groupname> are both invalid. The XML parser/builder will attempt to normalize them (groupname -> group-name) or remove them.

Comparing the generated output against a few of opnsense's example configs shows there is no difference.

If you do build something that would show exceptions, I'll run that and send sanitized output

I won't be building this, but you're welcome to do so and open a PR. I'd love to add it.

You need to provide more concrete examples of what is breaking and where; saying "this simply doesn't work" is unsatisfactory when it comes to debugging.

[jbhorner]

Regarding your example, your OPNSense Actual is not representative of a valid configuration file, as it contains invalid XML; <authorized keys/> and <group-name>admins</groupname> are both invalid. The XML parser/builder will attempt to normalize them (groupname -> group-name) or remove them. Those do not match the XML I have, either. I don't know if that is an issue with the "code" button in github or not. In my XML, it is authorizedkeys (one word) and groupname (oneword). The only modification I made was to replace identifying information.

You need to provide more concrete examples of what is breaking and where; saying "this simply doesn't work" is unsatisfactory when it comes to debugging.

I'm not sure how to respond to that. If I knew where it was breaking, I'd provide more context. OPNSense doesn't tell me "where" something breaks; it either works or it doesn't. Or so I thought. It appears that even when it works it isn't certain that it really worked.

I just spent a bit of time building a VM test environment for this in two hardware-identical (with the exception of the MAC addresses for the network adapters). It's a simple config. Four interfaces--LAN, WAN, Secondary, VLAN. I added a user so that there were two users. I ran this configuration through your converter.

The good news for this configuration is that I did see the user issue is fixed in the generated XML. Additional good news is that it imported--with a caveat. No more good news, just challenges.

The caveat noted is that when the import was finished OPNSense complained that it was not going to reboot (as would be normal after a configuration restore) as there was a difference in interfaces. The link to the interfaces was in the message. When I clicked on the link, the interfaces looked right to me from a UI perspective. I clicked each interface, saved it, and then saved the overall assignments.

Yet, curiously, when the system rebooted it came up with assignments that did not map to what I'd just reviewed and saved in OPNSense. (As shown on the console.) I would normally re-assign the interfaces through the console. But none of the username/passwords work. I'll forward both configs to you so that you can ascertain what might be going on.

For reference, I am using the latest version of pFsense CE and OPNSense, with both current with all updates as of today.

[jbhorner]

Regarding your example, your OPNSense Actual is not representative of a valid configuration file, as it contains invalid XML; <authorized keys/> and <group-name>admins</groupname> are both invalid. The XML parser/builder will attempt to normalize them (groupname -> group-name) or remove them.

I clearly haven't mastered replies on github. I had a reply to this as well. In my XML, that differs as well. In my XML groupname is one string, as is authorizedkeys. I didn't hand-type that information in, so I do not have an explanation as to what decided to "correct" something for me.

[mwood77]

Yet, curiously, when the system rebooted it came up with assignments that did not map to what I'd just reviewed and saved in OPNSense. (As shown on the console.) I would normally re-assign the interfaces through the console. But none of the username/passwords work. I'll forward both configs to you so that you can ascertain what might be going on.

OK, this is a common issue I've seen. I suspect it might be a duplicate configuration of some sort. I noticed pf2opn was applying the following tags on conversion:

    <config-apply>
        <uuid>12345678-1234-1234-1234-1234567890ab</uuid>
    </config-apply>

This uuid is arbitrary and generated at conversion. This is likely causing some kind of configuration mismatch in opnsense...which might be wrecking your login credentials.

I've deployed a temporary branch of pf2opn here, if you'd like to give it a try:

• https://add-more-fields.d2lqzfjit528nm.amplifyapp.com/

As an alternative, remove the above blocks in the generated output from www.pf2opn.com

I hope this may also fix the interface mismatch too.

---

code change: https://github.com/mwood77/pf2opn/pull/12