Open Nielsbishere opened 3 years ago
i am not following, did you create multiple jobs for a temp? number of job definitions and number of temp drives should match.
Basically a faster drive for temp1 and a slower bigger one for temp2. Wanted to see if it's faster but unfortunately it runs out of space since you can only limit phase1
@Nielsbishere, max concurrent is limiting per job.
Thing is, that's not enough for this. If you use a larger temp2 disk, you want more jobs to be spawned in phase >2 than in phase 1&2 combined
sorry i misunderstood. I see using the temp2 is now getting popular. so you want to have max_conncurent for phase 1 and 2 combined. seems useful. hard part is how to expose this option since the config structure is getting messy.
I think this could allow for interesting optimizations with faster SSDs on temp1 and then slower ones on temp2. Currently that's really hard because the ones in phase2 could be holding some of the space needed for the other threads.