Closed KonstantinHebenstreit closed 1 year ago
Much better now with loop for API Errors in "generate". But will still throw the error after five tries.
Generate is called per split of every dataset now. So the given split is lost but if generate is called a Collection of multiple datasets with multiple splits, everything up to the split where an Error occurs multiple times is saved.
The loop inside generate which catches API Errors is done five times (per item). But the Error is raised after that, since we do not want it to loop endlessly.
The generate function saves the generated cots, when it finished it for all examples. (see script of data-loader, which calls the function) If any error happens, all the generated cots from before are not saved and are therefor lost.