search

zstephens / neat-genreads

NEAT read simulation tools
Other
95 stars 27 forks source link

This job id has no regions to process, exiting... #11

Closed muppetjones closed 8 years ago

muppetjones commented 8 years ago

I keep getting this error, but I cannot tell if this is an issue or not. Looking at the code, the regions should be spread evenly across the threads, and if there are too many threads, it will lower the number of threads.

This job id has no regions to process, exiting...

I wonder--is this check working? (And does it happen before the reference is loaded in memory?)

zstephens commented 8 years ago

Hi there,

This is not necessarily an error, but an indicator that the number of jobs specified exceeds the number of regions the reference can be split into for processing (see my previous message about how the parallel jobs work, splitting up ref regions separated by N-regions). For example, if the reference has 50 possible isolated regions but you specify 60 job ids, the last 10 jobs will exit immediately with that message because they have no regions to process.

I’m amenable to eventually making the parallel job options more intuitive (i.e. not dependent on reference, no maximum number of jobs), but the current implementation is as it is because of how comparatively simple it was to implement.

-Zach

On Oct 17, 2016, at 11:48 AM, Stephen J Bush notifications@github.com<mailto:notifications@github.com> wrote:

I keep getting this error for some reason...

This job id has no regions to process, exiting...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3Agithub.com_zstephens_neat-2Dgenreads_issues_11&d=DQMCaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=iSEBNMfCHfS5bLIS7AqI6wkxgRNv8-2dMwRF43ddddU&m=XIku5YQnpJvYSwK37STjUbNkIp1PbkjswtniSa3C1WA&s=eQqySeOvzo7grmWT671YTOziQfeCONsktUgv-lNoUGU&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3Agithub.com_notifications_unsubscribe-2Dauth_AIHT1Wgn-2DFhb-5F4tC3WK90kh7ZUmFCyeUks5q06bJgaJpZM4KY2Xl&d=DQMCaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=iSEBNMfCHfS5bLIS7AqI6wkxgRNv8-2dMwRF43ddddU&m=XIku5YQnpJvYSwK37STjUbNkIp1PbkjswtniSa3C1WA&s=OZeLAesrjB6BdHSQEiMjJLIBfsUo6pMI6yRmWHl-a2c&e=.

muppetjones commented 8 years ago

Makes sense. I wonder if the memory errors (separate issue) are due to parsing the reference or a high coverage. If it's the reference, I think the only way to fix it would be to pause all jobs while one calculates the regions and writes them to a file, then have each job read the regions...which would be a headache.