ignition.first parameter exists post deployment

[lethedata]

Bug Description

After Simplified Provisioner is provisioned with Ignition, the first boot contains the ignition.firstboot parameter causing broken tree mounts.

To Reproduce

1. Create butane file that adds a user with password (password_hash = pass)

   variant: fiot
   version: 1.0.0
   passwd:
   users:
   - name: user
     password_hash: $y$j9T$xwWR4bvlM.oju.xHz5hYl0$z.vuMRdMTx.rLSw8LETGVS1MX0uHK0eUZ1j7J5dPIr.
     groups:
       - wheel

2. Convert to ignition
3. Host ignition file on accessible httpd server
4. Boot Simplified Provisioner

   virt-install   --name "fedora-iot-device" \
              --os-variant "fedora40" \
              --boot uefi,loader.secure=false \
              --vcpus 1 --memory 3072 \
              --network=bridge=virbr0 \
              --disk pool=default,size=30 \
              --cdrom Fedora-IoT-provisioner-40.*.iso

5. Edit "Install Fedora 40" boot parameter appending ignition parameters to linux

   coreos.inst.append=ignition.config.url=http://192.168.122.1/fiot.ign coreos.inst.append=rd.neednet=1

6. Boot system
7. Wait for provisioning to complete and system reboot
8. Login to the system - No home folder exists

Expected behavior

System boots without ignition.first after configuration allowing configured tree to mount with user's home folder existing.

Screenshots

localhost login: user 
Password: 
Last login: Wed Jun  5 18:18:43 on ttyS0
 -- user: /var/home/user: change directory failed: No such file or directory
Logging in with home = "/".
[user@localhost /]$ ls -lR / 2>/dev/null | grep var/home/user
[user@localhost /]$ ls -lR /sysroot/ 2>/dev/null | grep var/home/user
/sysroot/ostree/deploy/fedora/deploy/5bcc01335c19de1117841d2334062196fc40369bcbe6f71055ac3a2ddaa013f3.0/var/home/user:
[user@localhost /]$

OS version:

State: idle
BootedDeployment:
● fedora-iot:fedora/stable/x86_64/iot
                  Version: 40.20240422.3 (2024-04-22T19:43:59Z)
                   Commit: 5bcc01335c19de1117841d2334062196fc40369bcbe6f71055ac3a2ddaa013f3
             GPGSignature: Valid signature by 115DF9AEF857853EE8445D0A0727707EA15B79CC

Additional context

• Running rpm-ostree kargs --delete-if-present='$ignition_firstboot' only works if ignition.firstboot is removed during the first boot. If ran after first boot, proper trees do not mount.

Work Around

1. Create butane file that adds a user with password (password_hash = pass)
2. Convert to ignition
3. Host ignition file on accessible httpd server
4. Boot Simplified Provisioner
5. Edit "Install Fedora 40" boot parameter appending ignition parameters to linux
6. Boot system
7. Wait for provisioning to complete and system reboot
8. Edit boot parameter removing ignition_firstboot during boot
9. login
10. run rpm-ostree kargs --delete-if-present='$ignition_firstboot'
11. reboot