Using Batch with Just Once causes unexpected results. It appears that a list of records created during the first batch of a run is lost as a reference to later batches.
Using the following recipe (my test run was on NPSP, but the main behavior should impact all orgs just with slight differences):
If I run this with a command like:
cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrg
I will get 450 Contacts in my org, all linked to one of 5 Accounts generated in the recipe.
If I run this with a command that uses batch size:
cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrg --batch_size 20
The first 20 contact will match that pattern. But the next 430 will not have an Account from the initial list of 5. In NPSP they will have households, in a basic Sales Cloud org I would expect them to fail entirely since AccountId is not set.
Reproduction steps
Setup CCI Project, and connect to an org (only tested with a static org, I expect a scratch will have the same behavior)
Create a file with the Snowfakery recipe above.
Run: cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrg --batch_size 20
The first batch should work, subsequent batches will fail (or use default accounts if there is a package that creates them)
It's hard to be sure if this is a Snowfakery issue or CCI issue. I opened it as a CCI issue because batch-size is an option related to the CCI task used to generate the data.
Describe the bug
Using Batch with Just Once causes unexpected results. It appears that a list of records created during the first batch of a run is lost as a reference to later batches.
Using the following recipe (my test run was on NPSP, but the main behavior should impact all orgs just with slight differences):
If I run this with a command like:
cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrgI will get 450 Contacts in my org, all linked to one of 5 Accounts generated in the recipe.
If I run this with a command that uses batch size:
cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrg --batch_size 20The first 20 contact will match that pattern. But the next 430 will not have an Account from the initial list of 5. In NPSP they will have households, in a basic Sales Cloud org I would expect them to fail entirely since AccountId is not set.
Reproduction steps
cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrg --batch_size 20Your CumulusCI and Python versions
Operating System
Windows 11
Windows environment
PowerShell
CumulusCI installation method
pipx
Error Gist
No response
Additional information
It's hard to be sure if this is a Snowfakery issue or CCI issue. I opened it as a CCI issue because batch-size is an option related to the CCI task used to generate the data.